For historical reasons, astronomers measure the brightness of celestial objects in rank order. The smaller the rank number, aka magnitude, the brighter the object. Thus, a star of the first magnitude is much brighter than a star of the sixth magnitude, and it would take exceptionally good eyes and a very dark sky to see a star of the seventh magnitude. Now, it turns out that the human eye perceives brightness on a log scale, so magnitudes are numerically similar to log(brightness). And because they are a ranking list, it is always with reference to a standard. After some rough calibration to match human perception to true brightness of stars in the night sky, we have a formal definition for magnitude,
$$m = – \frac{5}{2}\log_{10}\left(\frac{f_{object}}{f_{standard}}\right) \,,$$
where f_object is the flux from the object and f_standard is the flux from a fiducial standard. In the optical bands, the bright star Vega (α Lyrae) has been adopted as the standard, and has magnitudes of 0 in all optical filters. (Well, not exactly because Vega is not constant enough, and as a practical matter there is nowadays a hierarchy of photometric standard stars that are accessible at different parts of the sky.) Note that we can also write this in terms of the intrinsic luminosity L_object of the object and the distance d to it,
$$m = – \frac{5}{2}\log_{10}\left(\frac{L_{object}}{4 \pi d^2}\frac{1}{f_{standard}}\right) \,.$$

Because astronomical objects are located at a vast variety of distances, it is useful to define an intrinsic magnitude of the object, independent of the distance. Thus, in contrast to the apparent magnitude m, which is the brightness at Earth, an absolute magnitude is defined as the brightness that would be perceived if the object were 10 parsecs away,
$$M \equiv m|_{d={\rm 10~pc}} = m – \frac{5}{2}\log_{10}\left(\frac{d^2}{{\rm (10~pc)}^2}\right) \equiv m – 5\log_{10}d + 5$$
where d is the distance to the object in [parsec], and the squared term is of course because of the inverse square law.

There are other issues such as interstellar absorption, cosmological corrections, extent of the source, etc., but let’s not complicate it too much right away.

Colors are differences in the magnitudes in different passbands. For instance, if the apparent magnitude in the blue filter is m_B and in the green filter is m_V (V for “visual”), the color is m_B-m_V and is usually referred to as “B-V” color. It is the difference in magnitudes, and is related to the log ratio of the intensities.

For an excellent description of what is involved in the measurement of magnitudes and colors, see this article on analyzing photometric data by Star Stryder.