“... “We have found it fruitful, ... to suggest that these receptor systems exist in sets.  We would propose that all of the receptors with maximum sensitivity to the long waves in the spectrum, for example, operate as a unit to form a complete record of long-wavelength stimuli from objects being observed. (For convenience of reference let us call this suggested retinal-cerebral system a "retinex.") ...   

Our theory would propose that the receptor sets act independently to isolate each of the different images -or, as we shall call them, lightness scales-which exist within the mixed pigments of the natural world.  Although the images viewed through filters ... exemplify the type of lightness scale formed by the retinex, it is only under uniform lighting conditions... that the lightness scale on the retinex has any correspondence with measured reflected light.

...For our given object there exists in super-position in the retinal-cerebral tract three independently drawn conclusions-conclusions about rank order of lightness on separate retinexes. Essentially, our hypothesis is first, that the retinexes exist; second, that they draw their conclusions independently; and third, that it is the comparison of these three separate conclusions for the single object that gives the sense of color. Color is the correlation number for several rank orders of lightness. If we assign a number, for example, to each lightness position in each image one each of the retinexes, then the color of an object can be stated in terms of three digits, each of which expresses the lightness of a given object on its respective retinex.

At first, this hypothesis does not seem remarkably different from that which would be classically expected from the elementary Young-Helmholtz approaches to color.  It is only when one examines the extraordinary stability of the lightness scale for each of the retinexes, that one comes to realize how this idea diverges entirely from the classical point-by- point study of color vision. In classical studies, one is concerned with the mixing of energies at various wavelengths.  In this approach, one is concerned not with mixing energies, but with correlating lightnesses - whatever their associated energies happen to be.”

“... whereas the function of colorimetry is to classify reflectance into categories with similar visual properties, the function of retinex theory is to tell how the eye can ascertain reflectance in a field in which the illumination is unknowable and the reflectance is unknown.” [page 11] 

... “This experiment led Land to propose that something fundamental was wrong with the idea that the biological system used the physical stimulus at a point. Instead of the long-, middle-, and short-wave receptors comparing responses at a point, Land proposed that all the long-wave receptors interacted to compute a biologic equivalent of reflectance from the long-wave flux. Similarly the middle- and short-wave receptors acted as independent sets to compute biologic reflectances for middle- and for short-wave flux.24  This biologic correlate of reflectance integrated over a band of wavelengths is called lightness..25  It is what we see if we look at the world in a single band of wavelengths. Land proposed that there exist systems which he called Retinexes that generate the lightnesses in each of the long-, middle-, and short-wave bands. Color sensations are generated by comparing the three different biologic reflectances or lightnesses.

... Namely, color sensations correspond to the reflectance of objects, even though the light in the natural environment coming to the eye from these objects is determined by an extremely variable and "unknowable" illumination.

The idea of an "unknowable" illumination is worth studying for a moment. Since the flux at the eye is the product of the reflectance times the illumination, one cannot determine the illumination from the total flux unless one knows the reflectance of the object. In studying the physics of light today, determining reflectance is a trivial matter. But remember, we are discussing the environmental problem confronting the evolving animal. The illumination varies in spectral composition, such as sunlight vs. skylight; in overall quantity, such as a bright day vs. a dark day; and in uniformity of illumination across the field of view, such as shadows cast by clouds or trees, etc. The animal visual system cannot use the same techniques the physicist would. The animal has neither reflectance standards nor illumination standards. His estimation of the reflectance of objects must be accomplished both instantaneously and internally. If within these constraints the illumination is "unknowable," then the animal must have developed a mechanism that can determine reflectance without separating illumination from the flux at the eye. This is the job of each Retinex. It must generate lightnesses that correspond to reflectances. Land and McCann (1971) proposed a visual mechanism for computing reflectance regardless of the non-uniformities in illumination. This mechanism uses the ratio of flux at closely spaced points and sequentially multiplies them to form a product.  This product is the mathematical model's approximation of reflectance. It is beyond the scope of this chapter to discuss the details of this mechanism, however the basic hypothesis is that taking the ratio of flux at closely spaced points eliminates the effect of gradual changes of illumination. The change in flux is so small from gradual illumination changes that it cannot be detected by the ratio measuring cells. Abrupt changes in flux are caused by edges between objects of different reflectance. The magnitude of the change in reflectance across an edge is measured by the ratio operation, thus establishing the relative reflectance of two adjacent areas. The ratio of reflectances is then sequentially multiplied to obtain the relative reflectance of each object in the field of view. Lightness, in principle, is also independent of overall changes in the level of illumination. This follows naturally from the fact that lightness correlates with reflectance and the reflectance of an object does not change with changes in illumination. The human system approaches total independence; however, there exists enough of a dependence on overall illumination to tell a bright day from a dark day.”

“There has been remarkably little change in the fundamental operation of Retinex model since first proposed in 1967 at Land’s Ives Medal Address to the Optical Society of America. The original proposal used the Ratio Product, Reset and Average. The original proposal also used a threshold operation on the Ratio step. The argument then was that reflectances had sharp edges and illumination edges were gradual. A threshold that removed small gradual changes in radiance would be of great value in modeling the B&W

Mondrian (Figure 1).

Extensive psychophysical experiments have shown three important changes in theory. First, in real life scenes, illumination can have sharp edges and gradual changes in reflectance.  The original hypothesis that the model could separate illumination from reflectance was wrong. Second, extensive quantitative experiments 12 showed that there is no single rate of change in radiance on the retina at visual threshold. In other words, we could not find psychophysical support for the threshold

mechanism. Third, extensive experiments with models showed that the reset, “normalization” process was the mechanism predicting appearance in B&W Mondrians. These experiments showed the importance of controlling the number of iterations at each multi-resolution level. If the calculation uses one iteration at each level the results show very strong local interactions If the calculation uses hundreds of iterations the results show very strong global interactions, stronger than the data on enclosure (Figure 4) and separation require.  The key factor in a successful Retinex model is the selection of the number of iterations at each level, which controls the degree of spatial interaction, that, in turn, mimics human vision. 13  In 1980 Frankle and McCann introduced the multi-resolution version that made real-time image processing possible. It is illustrated in Figure 5 (right).

Lesson: A Few Biological Operators

In reviewing the operation shown in the description of

the Retinex model (Figure 5), we see that there are only three operations: ratio, product, reset and average. In implementing these calculations we have always converted the input to log radiance. The consequence is that ratio and product operations are simplified to subtraction and addition. Reset is another biologically friendly operation. Difficult operations such as multiplication and exponentiation have not been a part of the model. There has been a desire from the beginning in 1963 to restrict the operator to simple biologically friendly mechanisms.”