Save the image highlights!

A friend pointed out an interesting background article on ISO in digital cameras by Richard Butler, a reviewer and technical writer for DPReview. It deals with ISO, dynamic range and the options that camera makers have to avoid blown highlights. The article may be interesting if you have an engineering degree, a camera fetish or both.

Blown highlights

The wedding dress still looks OK (Canon's Highlight Tone Priority feature on). Note that the image was taken on an overcast day.

The problem

By default, a CMOS sensor – unlike film – clips highlights abruptly : when the analog-to-digital converter exceeds its 10- to 14-bit range in a bright part of the image, you no longer see variation in brightness and color. Essentially you hit a hard ceiling on the output: there is no visible difference anymore between bright and brighter-than-bright. So snow, wedding dresses and clouds become flat blobs of white.

Photographers hate blown highlights (unless it gives a fully white background) more than they hate noise or featureless shadows. You may even get corrupt colors if the sky is bright, but not quite bright enough: one or two of the color channels may clip before the remaining channels, leading to an unnatural-looking cyan sky. Essentially blown highlights are comparable to a nasty form of distortion called clipping in the audio world: the equipment can’t generate more output on loud sounds and thereby introduces readily audible artifacts.

Limited dynamic range is one of the main remaining weaknesses of digital imaging nowadays – especially now that many cameras have more megapixels than is good for them. The topic is not getting a lot of marketing attention because the available improvements are limited, and the benefits is these solutions are still too difficult to explain to average consumers.

In photography terms: we are quite aware that the required exposure of our photos can easily vary by say 20 stops (1/8000s at f/16 in bright sun down to 1 second at f/1.4 by candle light). But people are less aware that the range of brightness inside a single image can easily vary more than the 10 stops – being the typical limit of a modern sensor or film.

The goal

So the trick is to change linear response (rising line that suddenly turns horizontal) into an S-shaped response curve. Like a car whereby pressing the pedal at high speeds increases the speed slightly to make you feel you are still in control, but not more than the car or driver can handle.

A sensor-to-image version of the story

The story by Richard Butler starts more or less from the end result and works backward. An engineer tends to work from the fundamental problem towards the solution. So this is what happens if you turn the story inside out…

The ISO settings used for a photo essentially tells you nothing about the sensor (for digital). It mainly tells you about the postprocessing in camera or PC.

So a sensor’s fundamental output (expressed in charge or milliVolt) is only a function of light brightness (aperture, scene) and exposure time (shutter speed). Manually or automatically varying the ISO on cameras is essentially an indication of how the sensor output is scaled to reach normal on-screen or print brightness levels.

So to get to higher ISO values (=take pictures with less light), there are two basic options:

  1. you increase the amplification before converting the analog voltage to digital using an (analog) amplifier. This increases the signal seen by the analog-to-digital (A2D or ADC) converter, but also amplifies noise. In audio terms, you are increasing the volume when there isn’t enough signal. The dynamic range or signal-to-noise ratio will be low because the original signal was low. Amplification (in the best case) increases noise and signal by a similar ratio.
  2. you can alternatively digitally “amplify” by multiplying the output of the A2D. This gives roughly the same result as long as the A2D is accurate enough compared to the original noise level. Hence the interest in 14-bit A2D converters: they allow the processing to be moved to “the digital domain”. Note that it doesn’t really make a difference if you do the scaling within the camera (done for JPEG) or in post-processing (an option in RAW) as long as you don’t lose information on the way.

Modern cameras do a mix of both. A variable gain amplifier before the A2D converter can boost the signal (typically in steps of 2x) without adding significant noise of its own. This reduces the need to make 16-bit high-speed A2D converters (increasing A2D resolution tends to increase the measurement time, and this is a problem when you have a lot of pixels to sample). Smaller steps in ISO are typically done in the digital domain and lead to a fraction of  a stop loss in dynamic range. I doubt this small loss in dynamic range is measurable in normally detectable.

So both options look equivalent, and the trade-off seems to be mainly a subtle optimization of sampling speed versus cost versus noise around the A2D converter.

BUT when you do digital amplification, you can easily make the digital amplification non-linear: you can use the amplification to create an S-shaped response curve by multiplying more at low- and mid tones than in the highlights. This reduces the chance of clipping in the final result and thus stretches the dynamic range.

To some degree it is normal to tweak this tonal response curve to compress the response curve to a nice S-shape that avoids clipping in the shadows or clipping in the highlights. The second option just allows you do this a bit more when you need it.

How does this relate to the DPReview article on Olympus?

The quoted article by Richard Butler shows that the Olympus E-620 SLR normally underexposes by 1 stop and compensate by boosting the output (except for the high tones). This gives an S-shaped response curve. The point of the article is that DPReview notices that this doesn’t occur at 100 ISO: the dynamic range is one stop less, and the 100 ISO behavior is closer to previous Olympus cameras. The point is that at 100 ISO, the sensor gets enough light to utilize the full intrinsic range of the photo sites. So at 100 ISO, you have more risk of losing the highlights than at 200 ISO or above. In fact, it is suggested that if you need 100 ISO (e.g. to get a slow shutter speed) you may be better off using the 200 ISO’s response curve and decreasing the light intensity (you needed 100 ISO didn’t you) using a 2x gray filter.

Or you can go manual

Alternatively, you can still use 100 ISO on the Olympus (assuming that’s what you need) and underexpose by say 1 stop. That uses the sensor in the same range as if you use 200 ISO, and you can then brighten the dark picture manually using whatever response curve you feel like in post-processing. With the underexposed picture, you can actually have fun solving the puzzle yourself on the PC. In say Adobe Lightroom, you can increase the exposure (gain), or tweak tone curve related parameters. Unfortunately, Lightroom has a lot of overlapping sliders you can play with and no single graph showing what they do except for the rendered image and the histogram. So you can play with “fill-in light” or alternative can turn up the dark tones, but only the latter will show a change in the tone curve.

Canon’s Highlight Tone Priority

Canon has a feature you can enable and disable called Highlight Tone Priority. Again, it underexposes, and compensates in the camera by boosting the non-highlight output levels.
Again it doesn’t work at 100 ISO. This actually has as side effect that Canon cameras, with HTP enabled and in auto-iso mode will avoid 100 ISO and select settings of 200 ISO or higher. This can be a good thing because, unless you have a very bright scene, the extra quality you get at 100 ISO doesn’t outweight other improvements you can get if you if you make the sensor look twice as sensitive: you can opt for 2x less motion blur or reduce lens aberrations by moving away from full aperture. So in my case, I tend to leave HTP on all the time and essentially avoid too extreme “expose to the right behaviour”.

Nikon has a similar feature called D-Lighting (pronounced “delighting”) which also doesn’t work at 100 ISO.

This entry was posted in Photo Technology and tagged , , , . Bookmark the permalink.

4 Responses to Save the image highlights!

  1. Risto Savolainen says:

    Good wrap-up. I am still not sure whether the Canon HTP affects RAW or not. It would be interesting to know. Here is a link to one explanation of Nikon´s D-lighting:


    • pvdhamer says:

      Good point: does Hightlight Tone Priority
      a) change the raw image data
      b) automatically get applied on most post-processing software (obviously the case for Canon, reports are that it does something, but less than on JPEGs)
      c) get ignored by typical post-processing software (with the exception of Canon’s).
      Not sure yet…


  2. Risto Savolainen says:

    I just discuused with a Canon representative here in Finland. He said, that HTP affects the RAW also. There is a patented, not fully public procedure to preserve the highlights in the RAW. He asked me to make some tests not caring about the` burning indicators´ in the back of canera LCD. Of course there is a limit, but it is higher than without HTP.


  3. Pingback: Lightroom 3 «

Leave a Reply

Your email address will not be published. Required fields are marked *