How We See Color
The cones are wired in such a way that their activity is compared with that of the surrounding cones. If there were only one type of cone, this comparison would tell us whether one area was brighter or darker than another.
People with normal color vision have three varieties of cones, which we will call red, green, and blue cones. Because these different types of cones respond differently to different wavelengths of light, comparing their activity also provides information about color.
The activity of a cone is compared with the average activity of many surrounding cones. A red or green cone will therefore signal not simply how strongly it is responding, but whether it is responding more or less strongly than the mixture of red and green cones around it. This creates red-green color contrast. In a similar way, the activity of blue cones is compared with the combined activity of surrounding red and green cones, creating blue-yellow color contrast.
It is these differences between cone responses, rather than simply the absolute response of each cone, that are transmitted forward to the visual processing centers. The retina therefore acts as a color contrast detector.
More Detail
Each cone connects to two types of bipolar cells, creating two opposing pathways. One becomes more active when the center cone is more active than its surroundings—this is called a center-ON response. The other becomes more active when the center cone is less active than its surroundings—this is called a center-OFF response.
The “surround” signal is not taken from a single neighboring cone. Instead, it represents the combined activity of many surrounding cones. Horizontal cells collect information laterally across the retina and help create this surround signal.
For red and green cones, the surround contains a mixture of signals from both red and green cones. A red cone in the center can therefore be compared with the average red-green activity around it. A green cone can be compared with the same type of surround. If the center cone responds differently from the surrounding mixture, the difference creates a red-green contrast signal.
Blue cones are much less numerous, making up only about 5% of the cone population. The surround signal is therefore dominated by red and green cones. Comparing the activity of a blue cone with this predominantly red-green surround creates a second color contrast channel. Because yellow light stimulates both red and green cones, this comparison produces a blue-yellow contrast signal.
The retina therefore converts the original responses of the red, green, and blue cones into two major color-contrast signals: red versus green and blue versus yellow. A third signal, produced mainly by the combined activity of the red and green cones, provides information about brightness.
This is conceptually similar to converting an RGB video signal into YUV. An RGB signal describes the amount of red, green, and blue at each pixel. YUV reorganizes this information into a brightness signal and two color-difference signals. The retina performs a similar transformation: instead of simply sending the individual red, green, and blue cone responses to the brain, it sends information about brightness and the differences between colors. Amazingly, color television engineers arrived at a similar way of organizing color information—not to imitate the retina, but to make color television compatible with existing black-and-white sets.”