How Digital Cameras Capture Color

A digital camera sensor is color blind. Each photosite, the light-collecting well etched into a sensor's silicon, measures only how many photons landed on it, not what wavelength those photons carried. Every color photo taken on a phone, a mirrorless body, or a professional DSLR is a reconstruction: a mosaic of single-color brightness readings, covered in tiny colored filters, mathematically stitched into something that looks like continuous color. Understanding that gap between what a sensor actually records and what a viewer actually sees explains almost every quirk of digital photography, from moiré shimmer on a striped shirt to why megapixel counts overstate real resolution.

What a photosite actually measures

A photosite is the individual light-sensing element on a digital camera's image sensor, typically built from a photodiode that converts incoming photons into an electrical charge. The charge is proportional to how much light hit that specific location during the exposure, nothing more. A photodiode has no built-in mechanism for distinguishing a red photon from a blue one; physically, all it registers is intensity.

This is worth sitting with, because it cuts against how most people picture a camera. The intuitive model treats a sensor like a grid of tiny eyes, each independently perceiving red, green, or blue. The real hardware is closer to a grid of light meters, each reporting only a single number: how bright was it here. Color has to be added on top of that grid through an external structure, and reconstructed afterward through calculation. Neither step is optional, and neither step is a "measurement" in the strict sense: the color your eye associates with any given pixel is an inference, not a direct readout.

Two different engineering paths solve the "how do we get color out of a colorblind grid" problem. The dominant path, and the one built into the overwhelming majority of cameras sold since the late 1970s, places a patterned filter directly over the photosite grid so that each individual photosite only receives light of one color. The other path, pursued by a handful of sensor designs, tries to make every single location capture full color information at once, either by stacking color-sensitive layers vertically or by scattering the color pattern into a more elaborate, less repetitive mosaic. Both approaches trade something to get there, and both are still shipping in cameras today.

The Bayer filter: the pattern behind almost every camera

The dominant solution was patented by Bryce Bayer, an engineer at Eastman Kodak, as U.S. Patent 3,971,065, "Color Imaging Array," filed on March 5, 1975, and granted on July 20, 1976 (U.S. Patent and Trademark Office, via Google Patents). The patent describes a checkerboard-style mosaic of red, green, and blue filters laid directly over a solid-state imaging array, with each filter registered to sit precisely above one photosite.

The pattern repeats in blocks of four filters: one red, one blue, and two green. Bayer's patent explicitly labeled the green-filtered elements "luminance-sensitive" and the red- and blue-filtered elements "chrominance-sensitive," reflecting a deliberate decision to weight the sensor's most abundant filter color toward the wavelength range where human vision is most sensitive to fine detail (Google Patents, US3971065A; Imaging Resource obituary of Bryce Bayer, 2012). This arrangement is now universally called a Bayer filter array, or RGGB, after the order of the four filters within each repeating block.

Before Bayer's design reached mass production, the alternative for capturing color electronically was a three-sensor rig: a beam splitter divided incoming light into three separate paths, each striking its own dedicated red-, green-, or blue-filtered sensor, which then had to be optically and electronically recombined (Imaging Resource, 2012). That approach worked, and still exists in specialized three-chip video cameras, but the extra optics, alignment tolerances, and cost made it impractical for a consumer device meant to fit in a pocket. A single sensor with a printed filter mosaic collapsed the problem into something that could be manufactured at consumer-electronics scale, which is the real reason the Bayer pattern won out over the fifty years since.

A camera sensor rated at 24 million photosites does not record 24 million complete color pixels. Under a standard Bayer filter it records roughly 12 million single-channel green readings, 6 million red, and 6 million blue, and the camera's processor computes the rest through interpolation. The "megapixel" number on a spec sheet describes total photosites, not directly measured color pixels.

Because every photosite sits behind exactly one color filter, the raw data streaming off a Bayer-filtered sensor is literally a mosaic: a grid where each cell knows only its own red, green, or blue brightness, and two-thirds of the color information at that exact location is simply absent. Nothing about the sensor's raw output is a usable photograph yet. That transformation happens in the next stage.

Why does the Bayer filter use twice as many green filters?

Human color vision relies on three types of cone photoreceptors in the retina, tuned to different but overlapping ranges of the visible spectrum, with the medium-wavelength cones (loosely called "green-sensitive") occurring in greater numbers and contributing most heavily to the luminance signal the visual system uses for perceiving detail and sharpness (Wikipedia summary of Bayer filter design rationale). By doubling the green-filtered photosites relative to red or blue in every four-photosite block, the Bayer pattern concentrates real, directly measured data exactly where human perception is most sensitive to fine spatial detail, rather than spreading data evenly across three channels where the visual system would use two-thirds of it for detail perception anyway.

Demosaicing: turning a mosaic into a photograph

Demosaicing is the computational process a camera's image processor uses to reconstruct a full-color image from the single-color mosaic a Bayer-filtered sensor produces. For any given photosite, the processor already knows one color value directly (say, red, if that's the filter sitting over it) and has to estimate the other two (green and blue) using the readings from neighboring photosites of those colors.

The simplest version of this, called bilinear interpolation, averages the nearby same-color readings surrounding a target pixel to fill in the missing channels. A red-filtered photosite's missing green value, for instance, gets estimated by averaging the green readings from the photosites directly above, below, left, and right of it, since in a standard RGGB block a red pixel is always adjacent to green pixels on at least two sides. Do this systematically for every single photosite across the sensor, and a mosaic where each cell holds one color number becomes a full image where every cell holds an estimated red, green, and blue triplet, which is what a photo file actually needs to display normal color.

Modern cameras rarely use plain bilinear interpolation anymore, because it performs poorly at sharp edges and fine detail. Camera manufacturers instead use adaptive algorithms, sometimes described in the imaging literature under names like AHD (adaptive homogeneity-directed) or VNG (variable number of gradients), which detect the direction of an edge in the image before interpolating, so the algorithm blends color along the edge rather than across it. This produces noticeably sharper, less smeared results on complex real-world scenes than a simple four-neighbor average, at the cost of more processing per frame.

It helps to think of demosaicing like a jury reconstructing an event from partial witness statements. No individual photosite "saw" the full color at its own location, in the same way no single witness saw an entire incident from every angle. But by cross-referencing overlapping partial accounts from immediately neighboring positions, both a jury and a demosaicing algorithm can produce a coherent, high-confidence reconstruction of something that was never directly and completely observed at any single point. It's an inference built from correlated partial evidence, not a record of ground truth.

A second, complementary way to picture it: imagine painting a wall using only a paint-by-numbers grid where two out of every three squares were left blank, and you had to guess each blank square's color from the squares touching it. Get close enough neighbors and the guess is nearly always right, because real-world scenes tend to have smoothly varying color over small distances. Get a scene with fine repeating detail, and the guess starts breaking down, which is exactly the failure mode covered in the next section.

Where the reconstruction breaks down

Demosaicing is a statistical estimate built on the assumption that color tends to vary smoothly across small neighboring areas of a photo. Most real scenes cooperate with that assumption. Fine, tightly repeating patterns do not, and this is where demosaicing artifacts become visible to the naked eye.

Moiré is a false, shimmering color pattern that appears in a photo when a fine repeating detail in the subject, such as the weave of a dress shirt, a screen door mesh, or closely spaced roof shingles, has a spatial frequency close to the spacing of the sensor's own Bayer grid. The interference between the two regular patterns generates a third, entirely artificial pattern, often showing up as rainbow-colored bands where none exist in the real subject. For decades, most cameras with Bayer sensors included a dedicated optical low-pass filter (sometimes called an anti-aliasing filter) mounted directly in front of the sensor, whose only job was to slightly blur incoming light just enough to suppress moiré before it could be recorded. Many newer, very high-resolution sensors now skip this filter deliberately, trading a small increase in moiré risk for a meaningful gain in raw sharpness, on the bet that finer sensor resolution reduces how often real-world detail lines up with the grid frequency that causes the problem in the first place.

Color fringing, sometimes visible as faint purple, green, or magenta outlines along high-contrast edges (a bright sky behind a dark tree branch is the classic example), is a related but distinct demosaicing artifact. It shows up specifically at boundaries where color and brightness both change abruptly between neighboring photosites, forcing the interpolation algorithm to make its least confident guesses exactly where errors are most visible to a viewer. Some of what looks like fringing in a photo is actually a lens optical defect called chromatic aberration rather than a sensor-side demosaicing artifact, and distinguishing the two requires examining whether the fringing shifts with aperture and focal length (a lens problem) or stays fixed relative to the sensor's pixel grid (a demosaicing problem).

Noise compounds both issues. In low light, a sensor's signal-to-noise ratio drops, and demosaicing algorithms interpolating from noisy neighboring readings produce color artifacts, sometimes visible as random flecks of red, green, or blue speckling in shadow areas, that a well-lit exposure of the same scene would never show. This is one reason high-ISO photos in dim rooms often look not just grainy but oddly colored in the noise itself, distinct from the grain pattern seen in monochrome film.

Alternatives to the Bayer filter

Bayer's design has dominated for fifty years, but it is not the only way to solve the color problem, and understanding the alternatives clarifies what the Bayer approach is actually trading away.

Fujifilm's X-Trans sensor, introduced in its X-series mirrorless cameras, replaces the simple four-photosite Bayer repeat with a more elaborate 6-by-6 photosite pattern. The larger, less regular pattern is specifically designed to reduce the odds that a fine repeating real-world detail will line up with the sensor's own filter pattern, which lowers susceptibility to moiré and, according to Fujifilm's stated rationale, lets many X-Trans cameras skip the optical low-pass filter entirely without the moiré penalty that omitting it would normally carry on a standard Bayer sensor (PetaPixel comparison testing, 2017; Wikipedia, Fujifilm X-Trans sensor). Independent blind testing by photography outlets has found the real-world difference between X-Trans and Bayer output is often subtle and scene-dependent, with each pattern interfering differently depending on the specific orientation and frequency of the detail being photographed, rather than one pattern being categorically sharper than the other (PetaPixel, 2020 blind test).

Sigma's Foveon X3 sensor takes a fundamentally different approach, skipping the filter-mosaic idea entirely. It exploits the physical fact that different wavelengths of light penetrate silicon to different depths, stacking three photodiode layers directly on top of each other at a single location, so that blue light is absorbed nearest the surface, green in the middle layer, and red deepest, giving every single pixel location a directly measured red, green, and blue value with no interpolation required. That eliminates demosaicing artifacts like moiré and color fringing by construction, but historically has traded off against noise performance, low-light sensitivity, and total resolution compared to Bayer sensors of similar size, which is why Foveon-based cameras have remained a niche rather than a mainstream choice.

A third, simpler alternative sidesteps the color problem altogether: dedicated monochrome sensors remove the color filter array entirely, letting every photosite capture the full intensity of all visible light that reaches it, unfiltered. This produces sharper, cleaner black-and-white images than a Bayer sensor's demosaiced-and-then-desaturated equivalent, because nothing is being estimated or interpolated, every pixel is a direct, full-resolution brightness measurement. Some manufacturers pair a monochrome sensor with a separate full-color sensor shooting the same scene simultaneously, then computationally merge the sharp luminance detail from the monochrome sensor with the color information from the Bayer sensor, a technique used in some multi-camera smartphone systems to boost apparent sharpness beyond what either sensor could deliver alone.

Sensor approach	How it gets color	Demosaicing needed	Main strength	Main tradeoff
Bayer filter (RGGB)	Single filter mosaic over photosites, 2x green weighting	Yes, on every pixel	Cheap to manufacture, decades of mature processing algorithms	Vulnerable to moiré and color fringing
Fujifilm X-Trans	Larger 6x6 pseudo-random filter mosaic	Yes, more complex pattern	Reduced moiré, often skips anti-aliasing filter	Heavier processing, marginal real-world gain over Bayer in blind tests
Sigma Foveon X3	Three stacked photodiode layers per pixel location, using wavelength-dependent silicon penetration depth	No, full RGB per pixel directly	No demosaicing artifacts by construction	Historically weaker noise performance and resolution at comparable sensor size
Monochrome sensor	No filter at all; captures full-spectrum intensity only	Not applicable, no color captured	Maximum sharpness and light sensitivity for black-and-white	Captures no color information on its own

Every approach to sensor color capture trades processing complexity, sharpness, noise performance, or cost against the others; none captures color for free.

Common misconceptions

A frequent misunderstanding is that a camera advertised at "24 megapixels" delivers 24 million pixels' worth of directly measured, full-color detail. Under a standard Bayer sensor, it delivers roughly 12 million directly measured green values, 6 million red, and 6 million blue, then computes a full RGB triplet for every one of the 24 million locations through interpolation. This is why raw photosite count and perceived color resolution are related but not identical measures, and why two cameras with the same megapixel rating can produce visibly different image quality depending on sensor size, filter design, and processing.

Another common assumption is that every digital camera uses the same Bayer pattern, or that Bayer is the only viable design. As covered above, Fujifilm's X-Trans arrays and Sigma's Foveon sensors both depart from the standard Bayer approach for different engineering reasons, and dedicated monochrome sensors skip the color-filtering problem entirely by not attempting to capture color at all. None of these alternatives has displaced Bayer as the mainstream default, largely because manufacturing cost, decades of optimized processing software, and adequate real-world image quality have kept the original 1976 design commercially dominant even as sensor resolution and camera capability have advanced enormously since Bayer's patent was filed.

A third misconception worth flagging: demosaicing artifacts like moiré are not a flaw unique to cheap cameras. Because moiré results from the interaction between a specific subject's fine repeating detail and a specific sensor's specific filter grid spacing, even professional, expensive cameras can produce visible moiré on the right subject (a common on-set problem with televised sports commentators wearing fine-striped suits, for instance), while a budget camera photographing a subject without fine repeating patterns may show none at all. It's a property of the interaction, not simply a marker of sensor quality.

What this means

A digital photograph is not a direct capture of color; it is a statistical inference built from a mosaic of single-color brightness readings, filled in by an algorithm that assumes nearby areas of a scene tend to share similar color. Bryce Bayer's 1976 patent solved the practical problem of getting usable color out of a single silicon sensor cheaply enough for mass production, and the uneven 2:1:1 mix of green, red, and blue filters in his design exists because it mirrors an uneven mix already built into human vision. Every camera built since has either adopted that solution directly, modified it (X-Trans), replaced it with a physically different mechanism (Foveon), or abandoned color capture altogether (monochrome sensors) to chase the sharpness that comes from never having to guess.

---

Read next:

• How fiber-optic cables transmit data
• What is a photon, explained
• How lasers work: coherent light explained

FAQ

Q: Why can't camera sensors detect color directly?

A photosite is a photodiode that converts incoming light into an electrical charge proportional only to brightness, with no physical mechanism to distinguish wavelength. Color has to be added externally, typically through a filter placed in front of each photosite, or by exploiting how different wavelengths penetrate silicon to different depths.

Q: What is a Bayer filter?

A Bayer filter is a repeating grid of red, green, and blue filters placed over a camera sensor's photosites, patented by Bryce Bayer at Eastman Kodak in 1976, with twice as many green filters as red or blue, matching human vision's greater sensitivity to green light.

Q: What is demosaicing?

Demosaicing is the computational process that reconstructs full-color images from a Bayer-filtered sensor's mosaic of single-color readings, estimating the missing color values at each pixel using data from neighboring photosites, using either simple averaging or edge-aware adaptive algorithms.

Q: Do all digital cameras use the same color filter design?

No. Most consumer and professional cameras use a Bayer-pattern filter, but Fujifilm's X-Trans sensors use a larger, less regular filter pattern to reduce moiré, Sigma's Foveon X3 sensors stack three photodiode layers to capture full color at every pixel without a filter mosaic, and dedicated monochrome sensors skip color filtering entirely.

Q: What causes moiré in digital photos?

Moiré appears when a fine, repeating detail in the photographed subject, such as woven fabric or a screen mesh, has a spatial frequency close to the spacing of the sensor's Bayer filter grid, producing a false interference pattern that does not exist in the real subject. Optical low-pass filters mounted over the sensor historically suppressed this effect by slightly blurring incoming light before it was recorded.

Q: Does a higher megapixel count mean better color accuracy?

Not directly. Megapixel count measures total photosites, but under a Bayer filter only about a third of those directly measure red, a third measure blue-adjacent readings, and the remainder measure green; the rest of each pixel's color is computed through demosaicing regardless of resolution, so color accuracy depends more on filter design, processing algorithm, and sensor size than on raw pixel count alone.