Category Archives: Photo Technology

Technical aspects of photography.

Lightroom 5

Photo Technology, Software, ,

Lightroom 5 was released on June 9th 2013. It is a solid, but less spectacular upgrade to Adobe’s main software package for photographers. The limited number of changes may be because it took 1 year to develop (compared to Lightroom 4 that took 2.5 years). The version of Lightroom may have been needed to get a clean baseline of tools that coincided with the release of Adobe Creative Cloud.

Lightroom

Despite its official name

Adobe® Photoshop® Lightroom® 5.0

Lightroom does not create modified image files from original files like Photoshop does. Instead it stores the adjustments you used to modify the image (e.g. “crop the image”) on disk. This is done in a so-called catalog. It automatically re-applies these adjustments whenever you view, print, edit or export the image. This saves storage space, simplifies file management and is convenient if you afterwards change your mind about any of the settings (e.g. “make the crop larger”).

From Photoshop to Lightroom

In general, Lightroom is meant for photographers who want to post-process their images, while Photoshop is nowadays mainly for graphics professionals who want to create new images. Sometimes a photographer does need Photoshop, but usually Lightroom is easier and faster to use because it is designed for photographers.

In each new Lightroom release the needs for Photoshop decreases a bit. The new licensing model for Photoshop may further accelerate the migration of photographers to Lightroom: you can still buy Lightroom, but you need to pay a yearly fee to use Photoshop (and a bunch of other Adobe tools).

Main new features in Lightroom 5

Radial gradient – This is a way to adjust a circular or elliptic area with soft edges in a number of ways. It is comparable to “dodge and burn” in the darkroom days.

rabbit_gradient

Circular gradient used to add fill light, color temperature and clarity.

Previous Lightroom versions had a few tools that are somewhat comparable:

  • Graduated Filter – for modifying one end of the image. Straight.
  • Adjustment Brush – for modifying an area which you paint in with a brush. Flexible.
  • Post-crop vignetting – light/dark adjustments only. Always in the middle of the crop.

Smart previews – These are essentially about 4 MPixels versions of the images that can be used if the full images are temporarily not available. You can choose which images or directories have smart previews. The software uses available smart previews if the USB drive or (in my case) NAS containing the images is not online. If you have access to smart previews, you can edit the images just like you could edit your original image – although you cannot see the image at full resolution. In my case that would mean editing at max 2540 × 1693 resolution without being able to zoom in to see the full 21 Mpixel resolution of my Canon 5D2. The feature is useful for laptop use, but also has benefits if you need to send somebody a Lightroom catalog to view or even edit.

Smart Preview2

Smart Preview controls are below the histogram.

Smart Preview

Smart previews are internally 2540 pixel raw files in Adobe’s DNG raw format.

Straightening – The “Upright” feature corrects tilted horizons and fixes problems with perspectives in architectural photography. Unlike DxO’s Viewpoint, you don’t have to indicate a set of lines or a rectangle that are supposed to be straightened. The software detects this automatically and gives a few options within the Lens Corrections section. Note that the image is warped to get this result.

Original image

Original image

2013_Vienna_09-2

Automatically straightened

Fully straightened (automatically)

Fully straightened (automatically)

Note that there are parts missing parts in the final image. These would normally be shown in black and cropped off by the user.

You could argue that this feature is the poor man’s Tilt and Shift lens: with a camera on a level tripod a wide-angle lens should get you similar results. The wide-angle should be enough if you keep the camera level, but shifting the lens up can get the horizon below the middle of the photo if required. This was not required for the above photo.

Other new features

PNG file support – I have some PNG files in my archive that I used for creating photo books. “Synchronizing” my storage folders using Lightroom 5 now brings these files into Lightroom. The files are not too relevant as the format is seldom used for photos, but here is an example of a PNG file that I used as an illustration in a photo book:

Continuous_happinessSpot removal – Lightroom 4′s circular spot removal tool has been extended to allow you to paint away any bobs that needs replacements.

In addition, a tool is provided to make sensor dust spots show up better. Here is part of the original image before any corrections. If you look carefully while slowly scrolling the image you may discover several dust spots:

Original image (with spots)

Original image (with spots)

While using the Spot Removal (Q) tool you can enable Visualize Spots mode. This enhances edges and shows them as white pixels. Black pixels represent lack of local detail. The ten or so small circles in the sky are dust spots:

Spots highlighted by Lightroom 5

Spots highlighted by Lightroom 5

The next screenshot shows eleven spots that I had manually discovered (using Lightroom 4) and had fixed earlier. In this case, you can see that I had manually discovered (using my motion trick at 100% magnification and slightly increased contrast) pretty much the same spots that Lightroom 5 could capture. Note that finding spots in a cloudless blue sky would be easier.

Spots repair locations

Spots repair locations

 

DxOMark Camera Sensor article (v2.0 and v2.0.1)

Photo Technology, , ,

I have been working on an update to my original DxOMark article. That update has just been published on Luminous Landscape, a well-known photography site operated by the Canadian landscape photographer and publicist Michael Reichmann.

A slightly newer version of the article is available at the DxOMark website. It features four extra cameras and almost identical text.

Sample DxOMark results

The article covers various aspects of image sensor size and its impact on image quality. The article is built around original benchmark data measured by DxO Labs. I have rehashed their data (with permission) to stress basic trends and highlight a few topics:

  • Benchmark data for over 180 high-end cameras (starting at about $400).
  • Which benchmark numbers by www.dxomark.com are most relevant for your needs?
  • The technical relationships between sensor noise, dynamic range and resolution.
  • A comparison of what noise does at low ISO and at high ISO (this is trickier than “doubling the ISO reduces the signal-to-noise ratio by 2×”).
  • The implications of using “mirrorless” cameras (and associated smaller sensors) on image quality.
  • The image quality of the new wave of cameras that use Sony’s new Exmor sensor with its excellent low ISO dynamic range performance. With a bit of speculation of whether Canon (that normally doesn’t use Sony sensors) can catch up with Nikon (that does regularly use Sony sensors).

You can contact me about the article via comments on this website. I will also try to keep an eye on comments on the LuLa and the DxOMark fora (forums).

Magic Lantern for the Canon 5D Mark II

Photo Technology, ,

What is Magic Lantern?

If you own a Canon 600D, a Canon 60D, Canon 5D Mark II, or certain of their predecessors, you might be interested to hear that you can extend the capabilities of your camera for free (although a donation is requested). This is not by replacing the camera’s internal software by a newer version (recommended, but this mainly fixes bugs), but by adding software from a bunch of non-Canon developers. This Magic Lantern software extends the existing Canon software with many new features that target technically inclined videographers and photographers.

Features for photographers

Magic Lantern was originally created mainly for those who use Canon DSLRs for serious video work. I don’t know much about video, so I will only describe features that help photographers.

The features are somehow largely centered around Liveview and likely benefit photographers most who sometimes need to do “slow” photography: they use a tripod, use tethering in a studio to check focus, have a complex setup or simply want to have maximum control. Having said that, Magic Lantern states that it has benefits as well for photographers that are in a constant hurry: it gives you the option of putting certain options that you use a lot under a particular button.

A few of the key features:

  • focus peaking – whereby the Liveview image displays which parts of the image are in focus. Useful when you want to carefully control what is in focus. This can be seen as an alternative to tethering your camera to a computer via USB in the studio.
  • exposure clipping – the Liveview image can show which parts of the image will be too light and too dark using overlaid zebra stripe patterns.
  • more on-screen data – for example the current main camera mode (e.g. M), focal length and focus distance.
  • focus loupe – you can see part of the image zoomed in 2x or 3x to check sharpness. This feature is fancier that Canon’s counterpart and can even simulate what a split screen focus aid used to look like.
  • interval timer – you can take 100 pictures at 60 second intervals to show a flower opening. Or 1000 pictures at 1 hour intervals of a construction site – all providing you can get your battery to last.
  • triggering exposures – the shutter can automatically fire if the scene brightness or content changes significantly. Essentially a makeshift motion sensor.
  • automatic HDR – not only can the camera take a series of images at different exposures automatically, but it can take the entire series at one press of the button. It can even determine how many exposures are required automatically (or manually) and give you a rough preview of the merged image. Pretty cool. Essentially this gives your 5D2 a feature found in the 5D3, but without the artsy options: you do your real HDR merging afterwards on a computer.
  • improved mirror lockup – flip up the mirror a few seconds before taking the picture to reduce vibrations. The Canon equivalent is relatively tedious to operate.

Display when shooting video using Magic Lantern

The actual list of features is about as long as the list of features that your camera originally came with. So some people only use 2 or 3 of the new features. Others actually do read the software manual and experiment around (takes an evening – just like Canon’s firmware).

Installation and risk

There are risks involved in tinkering with complex equipment. My feeling is that the risk is comparable to opening up PCs to upgrade memory. If you never did something similar, you can get someone else to install Magic Lanterns (ML) and show you the basics.

The risk is lower than you might expect because ML doesn’t simply overwrite Canon’s software: it runs as an add-on and (in most cases) you will not see changes to the menus provided by Canon. There is a simple procedure to uninstall ML entirely.

This is essentially how ML works under the hood:

  1. A minor modification to Canon’s software makes the camera Magic Lantern aware. Comparable to a boot loader on a PC. ML is incidentally not the only party that does this (there seems to be a USB remote controller that uses the same trick to extend Canon’s software).
  2. Whenever you activate the camera, the firmware first checks for the presence of special non-image files on your flash card. If found, it loads Magic Lantern from the flash card. This does not visibibly delay camera operation. The ML software sits alongside the Canon software in camera memory (RAM). If the ML files are not found on the flash card (or you hold  down a button while turning it on), Magic Lantern is not loaded and you get  unmodified camera behavior. Alternatively, you can choose to carry memory cards with and without ML.
  3. Running ML,
    • the optical viewfinder information display is unchanged
    • the LCD viewfinder for LiveView displays significantly different information
    • Canon’s own menus (Menu button) are for 99% unchanged
    • you can view ML’s own menus by pressing the Erase button while in Liveview mode
  4. Whenever you make changes to ML settings this is written to the flash card for the next session. Some changes are also stored in the camera’s non-volatile memory (e.g. when ML menu’s interact with existing Canon features?)
  5. The ML files stay on the flash card, even if you erase the card using the camera. Actually ML formats the card and then writes the ML files back from memory. If you erase or format the card entirely using a PC, you need to reinstall the ML files onto the card. Until then, you will be operating without ML when you use that card.

Quality and stability

I cannot give you hard numbers, but since version 2.3 the stability seems to be close to that of Canon’s own software. Both have occasional bugs and both try to fix these bugs as soon as possible. ML is an open source project, so anyone with (considerable) programming skills can contribute.

All this doesn’t mean you can never run into a problem: ML software adds complexity to the entire setup, and strange combinations of features may give strange results. But if you stick to mainstream usage of the features (= use them more or less as documented) you should be alright.

Some features are clearly marked as “for very advanced users”. One example is the ability to take pictures in a low-res format while in Liveview mode without any shutter motion or sound whatsoever. A bit weird, and it actually seems to work, but you won’t be using this unless you are a video technician or are motivated enough to figure out how to deal with these “422″ encoded frames.

A final example is a menu item called “Don’t press this”. The user manual just says not to press it. Actually it probably doesn’t do any harm (otherwise why give it such a tempting name), but I don’t want to press it just yet. I suspect it contains a game that is totally not camera related. After all, your camera is just a computer with an industrial strength webcam attached as a peripheral (at least that is how geeks tend to see it).

Drawbacks

So far, things are going well with my own use. And ML has thousands of heavy users who rely on it on a daily basis. The documentation is actually pretty good – including the description of the risks involved. But…

  • It will only install on the latest version of Canon’s firmware. So you need to upgrade a 5D2 to v2.12 before you can install ML. A sensible choice by ML to minimize risk.
  • Running ML will slightly increase battery drain. Essentially because it gives the ARM processor more work to do because of extra features. It will increase batter drain a lot if you start using Liveview more than you previously did.
  • ML increases overall system complexity somewhat: it is like upgrading from a 5D Mark II to a 5D Mark III – more features which you may or may not use.
  • ML is not available on all currently Canon cameras (notably not the 7D or 5D Mark III so far). ML is written by volunteers and all this is a lot of work.
  • Something could go wrong. But the manual explains how to get the camera up and running again in the more common cases. As far as I can tell, the risk of loosing images stored on the flash card is absent, but there is a risk that you may need to briefly remove the batter to recover. A quote from the Magic Lantern FAQ:

In practice, we are doing our best to prevent these situations, and thousands of users are enjoying it without problems. However, this does not represent a guarantee – use it at your own risk.

Lightroom 4′s PV2012 image enhancement algorithm

Photo Technology, Software, , ,

In 2009 I wrote a small article here about a new class of image enhancement algorithms. Such algorithms made pictures look sharper by added local contrast and brought out details in both shadows and highlights. And they did this without adding halos around high-contrast edges such as the transition between foreground and sky. The article focused on a research paper by Farbman, Fattal, Lischinski and Szeliski (FFLS, 2008).

The brand new Lightroom 4 (and the associated versions of Adobe Camera Raw 6.7) has now incorporated similar technology that is based on a newer research paper entitled “Local Laplacian Filters: Edge-aware Image Processing with a Laplacian Pyramid” by Paris, Hasinoff and Kautz (PHK, 2011).

The implications of this new Lightroom feature for photographers are significant enough that I will gradually add more details here as I upgrade to Lightroom 4 (possible since March 5th) and get hands-on experience with it.

What’s so important about this technology?

On superficial examination, the two mentioned research papers (and there are many more where that came from) seem similar enough. They are both trying to give users the ability to make details in images more striking (local contrast enhancement) without creating undesirable artifacts such as halos.

The weird thing about this research is that they don’t distinguish between minor image tweaking (e.g. typical raw converters sharpen images to some default degree) and image modifications like HDR which also sharpen the image but can lead to unnatural results if overused. This is probably because there is no sharp boundary between enhancing a little and going overboard with your the settings.

Thus many of the example images in the research papers look like what we photographers would consider HDR photography: in HDR photography, you want to show a broad range of lightings without making the local contrast look flat, and without making the overall picture look artificial. Probably the authors would answer that HDR is an application area where this kind of algorithm is needed – but they can also be used for “normal” images shot with a single exposure in a Normal Dynamic Range situation.

Something similar applies to the Lighting module in DxO Lab’s “DxO Optics Plus”. It probably uses a similar type of approach to boost local contrast. I tend to consider it an HDR-like technique.

Alternative algorithms

A claimed key benefit PHK as chosen by Adobe is that the algorithm is simpler and thus takes less processing time. In fact, the paper’s introduction starts off by saying that the widely known (in the right circles) Laplacian Pyramid technology was underrated, leading previous researchers (including FFLS) to develop more complex algorithms to compensate for its shortcomings.

It is worth noting that the first author in PHK works for Adobe. But these are scientifically reviewed papers published in respectable conferences – meaning the authors have to be very rigorous about their claims, when these hold and what evidence and even counter-evidence there is for these claims.

Where do I find this in Lightroom 4?

Now shipping in Lightroom 4 (and Photoshop CS6 Camera Raw) – the tools for adjusting shadows, highlights, and clarity are based on a fast version of the local Laplacian filters we introduced at SIGGRAPH 2011

Source: https://plus.google.com/116168853211931020305/posts/id3W5HZcdFN

Should we care?

Fuji’s Organic Sensor Technology

Photo Technology, , ,

In 2009, Fujifilm was granted a patent on an organic layer to improve CMOS image sensor performance. The thin organic layer converts incoming light to electrical charge (like a solar cell) that is then detected by more or less conventional CMOS image sensor circuitry.

Fujifilm’s “organic sensor” technology started to receive attention recently due to speculation that it would be featured in a future high-end compact camera (expected in early 2012; possibly called the “LX10“). This camera is expected to have an interchangeable lens and target the same market as the highly successful Fujifilm X100.

Update: the Fujifilm X-Pro1 (March 2012) turned out to not contain this particular technology. Instead it has a color filter array layout which avoids the need for an anti-aliasing filter.

Although Fujifilm has not confirmed the use of their organic conversion layer in the forthcoming camera, a Fujifilm executive claimed that the interchangeable lens camera (possibly with an APS-C sized sensor) will outperform the current generation of full-frame sensors in terms of noise.

The technology

Fuji sensor with "panchomatic" organic layer

Fuji sensor with "panchromatic" organic layer (taken from Fuji's 2009 white paper)

A color sensor using this technology uses a conventional color filter array to make individual pixels sensitive to green, red and blue light. In a 2009 white paper, Fuji states that the layer is insensitive to infrared light. Therefore no IR absorption filter is needed, which is in itself a minor benefit.

The organic layer is labeled “Panchromatic Photoelectric Conversion Layer” in the above diagram by Fuji. The layer is 0.5 μm thick, and converts almost all visible light to an electrical charge (electrons and holes). The organic layer, which closely resembles a solar cell technology, is sandwiched between a negatively charged transparent electrode (like in LCDs) and an array of positively charged square “Pixel Electrodes” that form the actual imaging pixels. The latter are not transparent.

Electron microscope image of the upper surface of an array of 3 μm (aka um) pixel electrodes. The small circle within (below) each square the metal conductor that connects each pixel electrode to the transistors below.

In the 2009 prototype the pixels are spaced 3 μm apart. This would result in a resolution of roughly 40 MPixels for an APS-C size sensor or 22 MPixels for a Four-Thirds size sensor. Due to the gaps between the pixel electrodes, 85% of the surface area is used to capture charge. The possible loss of 15% of the incoming light can probably be neglected anyway, but these gaps could be further reduced if Fuji wants to and may not play any measurable role at all if the free electrons generated above the gaps reach the pixel electrodes anyway due to the applied electric field.

So a key benefit of this technology is that it is efficient compared to alternative ways to make arrays of relatively small pixels. With small pixels in a conventional CMOS sensor, the gaps between pixels (or the inverse “fill factor”) becomes a problem, especially because the wiring needing to access the pixels lies on the upper (outside) layers of the sensor. Backside illumination (BSI) addresses the latter by grind down the chip and illuminating the sensor from the back.

Fuji stresses that their sensor doesn’t need a micro lens array to funnel the light to the pixels. Apart from a reduction in component count, this avoids color errors at the edges of the sensor. This occurs with small pixels due to crosstalk between the color channels when light hits the sensor at an extreme angle.

The Benefits

One key benefit of this sensor technology should be its sensitivity to light: Fuji measured a quantum efficiency of 65% in 2009 (for green light at 550 nm) but stated that this can be improved by adding anti-reflection coatings. The quantum efficiency figure seems competitive compared to relatively expensive backside-illuminated sensors as used in recent high-end sensors with small pixel dimensions (such as the Nikon P-100, Canon S100, iPhone 4 and iPhone 4s).

This translates to a slight sensitivity gain – and would behave like removing a very light gray neutral-density filter. Using the available data, I am unable to calculate how big this improvement is expressed in ISO. Note that the prototype sensor described in 2009 will obviously differ from Fuji’s current technology anyway.

A more interesting question is what impact the technology change has on dynamic range. The ability to funnel more light to the sensor helps sensitivity and noise. But dynamic range depends on the ratio between the maximum amount of charge the pixel can hold (at saturation) compared to the noise level. Given the high expectations set by Fuji for the 2012 camera, the benefits of the organic layer must somehow translate to a dynamic range improvement (as differences in quantum efficiency between modern sensors tend to be limited).

Impact

It is a bit early to speculate on the impact of an unannounced camera of only leaked images seem to exist. Especially when we can only guess whether it utilizes Fuji’s organic photoelectric conversion layer. But what would happen if the technology delivers? In particular, would it really close the image quality gap between say full frame and APS-C sensors?

The key question for strategists is thus whether an organic layer would improve all sensor sizes and sensor resolutions to the same degree. If so, it could simply improve the quality level of what could be manufactured, but doesn’t change the landscape of Four-Thirds versus APS-C versus Full Frame versus Medium Format sensors. Fuji’s white paper starts off with

We proposed a new CMOS image sensor with a thin overlaid panchromatic organic photoelectric conversion layer as the best candidate for sensors with reduced pixel size.

and its title also mentions reduces pixel size as the highlight of this technology.

This suggests that the organic layer helps increase the design of high-resolution sensors regardless of their size, by improving their light gathering efficiency. This implies improved performance of tiny compact camera sensors with mere 1 μm pixels, and would provide an alternative to the cumbersome back-side illumination technology used in some compact cameras and camera phones.

It could also pave the way for 20 MPixel Four-Thirds sensors and 40 Mpixel APS-C sensors, and 80 MPixel full-frame sensors. Such (relatively speaking) extreme resolutions when using 3 μm pixels might be overkill for many lenses, but would indirectly increase image sharpness by promoting cameras designed without an anti-aliasing filter. Such an AA filter softens the image to get rid of details that are too fine to be resolved. But in doing so, AA filters also blur the finest details which the sensor can capture (in essence an anti-aliasing filter is a sheet of very finely frosted glass).

Because extreme resolution is not always needed and is sometimes even useless, a high-end camera could conceivably provided modes whereby the full pixel resolution can be downscaled within the camera. For example a full-frame camera 80 MPixel camera might normally record at 40, 20 or 10 MPixel resolution. But can be set to resolve in 80 MPixels if the user wants this. Current Canon SLRs already have a similar resolution scaling feature that Canon calls SRAW, but the option is not used very much because it gives loss of detail without shrinking the file size much. Note however that this route should give fine image quality, but likely costs more energy than the equivalent lower resolution sensor (more samples to digitize, more digital operations).

[ last modification: 26-Nov-2011 ]