Tag Archives: RAW

Lightroom 5

Lightroom 5 was released on June 9th 2013. It is a modest upgrade to Adobe’s main software package for photographers. The limited number of changes may be because it took 1 year to develop (Lightroom 4, by comparison, took 2.5 years). A new version of Lightroom may have been needed to get a clean baseline of tools that coincided with the release of Adobe Creative Cloud.

Software aspects

The new version still contain “encounterable” minor software bugs. Although these seem mainly related to the user interface, less eager adopters may want to wait for a 5.1 upgrade.

A Lightroom 5 bug
The above error appears to have popped up in Lightroom versions as long ago as 2008. But I hadn’t seen worthwhile errors in earlier Lightroom versions, so maybe LR5 was brought to the market somewhat rushed.

Lightroom 5 incidentally requires Windows 7 or 8. Windows Vista (which I still run on my laptop) is no longer supported.

The Lightroom concept

Despite its official name

Adobe® Photoshop® Lightroom® 5.0

Lightroom does not create modified image files from original files like Photoshop does. Instead it stores the adjustments you used to modify the image (e.g. “crop the image”) on disk. This is done in a so-called catalog. It automatically re-applies these adjustments whenever you view, print, edit or export the image. This saves storage space, simplifies file management and is convenient if you afterwards change your mind about any of the settings (e.g. “make the crop larger”).

From Photoshop to Lightroom

In general, Lightroom is meant for photographers who want to post-process their images, while Photoshop is nowadays mainly for graphics professionals who want to create new images. Sometimes a photographer does need Photoshop, but usually Lightroom is easier and faster to use because it is designed for photographers.

In each new Lightroom release the needs for Photoshop decreases a bit. The new licensing model for Photoshop may further accelerate the migration of photographers to Lightroom: you can still buy Lightroom, but you need to pay a yearly fee to use Photoshop (and a bunch of other Adobe tools).

Main new features in Lightroom 5

Radial gradient – This is a way to adjust a circular or elliptic area with soft edges in a number of ways. It is comparable to “dodge and burn” in the darkroom days.

rabbit_gradient

Circular gradient used to add fill light, color temperature and clarity.

Previous Lightroom versions had a few tools that are somewhat comparable:

  • Graduated Filter – for modifying one end of the image. Straight.
  • Adjustment Brush – for modifying an area which you paint in with a brush. Flexible.
  • Post-crop vignetting – light/dark adjustments only. Always in the middle of the crop.

Smart previews – These are essentially about 4 MPixels versions of the images that can be used if the full images are temporarily not available. You can choose which images or directories have smart previews. The software uses available smart previews if the USB drive or (in my case) NAS containing the images is not online. If you have access to smart previews, you can edit the images just like you could edit your original image – although you cannot see the image at full resolution. In my case that would mean editing at max 2540 × 1693 resolution without being able to zoom in to see the full 21 Mpixel resolution of my Canon 5D2. The feature is useful for laptop use, but also has benefits if you need to send somebody a Lightroom catalog to view or even edit.

Smart Preview2

Smart Preview controls are below the histogram.

Smart Preview

Smart previews are internally 2540 pixel raw files in Adobe’s DNG raw format.

Straightening – The “Upright” feature corrects tilted horizons and fixes problems with perspectives in architectural photography. Unlike DxO’s Viewpoint, you don’t have to indicate a set of lines or a rectangle that are supposed to be straightened. The software detects this automatically and gives a few options within the Lens Corrections section. Note that the image is warped to get this result.

Original image

Original image

2013_Vienna_09-2

Automatically straightened

Fully straightened (automatically)

Fully straightened (automatically)

Note that there are parts missing parts in the final image. These would normally be shown in black and cropped off by the user.

You could argue that this feature is the poor man’s Tilt and Shift lens: with a camera on a level tripod a wide-angle lens should get you similar results. The wide-angle should be enough if you keep the camera level, but shifting the lens up can get the horizon below the middle of the photo if required. This was not required for the above photo.

Other new features

PNG file support – I have some PNG files in my archive that I used for creating photo books. “Synchronizing” my storage folders using Lightroom 5 now brings these files into Lightroom. The files are not too relevant as the format is seldom used for photos, but here is an example of a PNG file that I used as an illustration in a photo book:

Continuous_happinessSpot removal – Lightroom 4′s circular spot removal tool has been extended to allow you to paint away any bobs that needs replacements.

In addition, a tool is provided to make sensor dust spots show up better. Here is part of the original image before any corrections. If you look carefully while slowly scrolling the image you may discover several dust spots:

Original image (with spots)

Original image (with spots)

While using the Spot Removal (Q) tool you can enable Visualize Spots mode. This enhances edges and shows them as white pixels. Black pixels represent lack of local detail. The ten or so small circles in the sky are dust spots:

Spots highlighted by Lightroom 5

Spots highlighted by Lightroom 5

The next screenshot shows eleven spots that I had manually discovered (using Lightroom 4) and had fixed earlier. In this case, you can see that I had manually discovered (using my motion trick at 100% magnification and slightly increased contrast) pretty much the same spots that Lightroom 5 could capture. Note that finding spots in a cloudless blue sky would be easier.

Spots repair locations

Spots repair locations

 

Caption

Technical tips for amateur photographers

Sometimes there are tips I don’t offer to experienced photography enthusiasts unless they ask the right question first. This is because I feel they should already know all this, and might be offended that I could imagine they didn’t – especially if the issue is pretty important.

But… in practice nobody can know everything. Especially when some things in the photography world have changed gradually or are non-trivial. So many photographers still use best practices that were suitable years ago, but may be outdated. So, pssst… here is a checklist of several such issues. Where applicable, I tried to add evidence.

1. Good cameras and photographers deserve Raw

The question is whether to set the camera so that it generates Raw image files. Your camera’s default is to generate JPG files. Raw files can be seen as the camera’s native format, and may need to be converted to JPG at some point.

Adobe called their open Raw format “Digital Negatives” or DNG. So, to borrow their analogy, the underlying question is whether to save or to automatically discard your digital negatives. If you currently generate JPG files, you are automatically developing and discarding those “negatives“. As the analogy suggests, this is automatic and thus convenient. But it has certain drawbacks.

Originally Raw files were considered a tool for pixel fetishists: the digital photo that you really needed (for printing, for mailing, for websites, for displaying) was in JPG format because most software could only handle JPG files. So Raw files added an extra step and were thus considered by many to be a waste of time, energy and storage space. Furthermore the JPG file format is well standardized, while Raw files were vendor-specific image formats that might become obsolete in the future.

This situation has changed. By now JPG is a 20 year-old image format that is still good enough as an output format, but isn’t really sophisticated enough to store the images which current camera sensors can capture. So assuming you have a good camera, one could argue that you lose a few year’s worth of camera industry innovation if you choose to exclusively work in Raw. Obviously not everybody will care, but it is at least worth knowing what your choice is.

In more technical terms, current sensors have 12 or 14 bit of accuracy and JPG was designed in the time that 8 bit was considered enough. Furthermore classic JPG was designed to discard fine detail and color nuances in order to save storage space: JPG is essentially a form of “lossy compression”. To JPG‘s credit, the tradeoff between image quality and file size is adjustable, but cameras only give you limited control over this tradeoff. Raw files, on the other hand, use “lossless compression”, store all 12-14 bits of information and produce files that may be 1.5-4× larger (this depends on the ISO setting).

Here is an example of the image quality lost by using JPG. These are 100% crops of an image of a bee keeper’s working clothes taken with my Canon 5D Mark II. The Raw and JPG images were actually simultaneously recorded in-camera and are thus automatically generated from the same exposure. I selected the camera’s highest quality setting. No changes were applied to the resulting Raw and JPG files in post-processing: these are pretty much the defaults.

Hold the mouse over the images to exchange the Raw and JPG images. Loading may take a few seconds.

The JPG version clearly has less contrast (but that could be fixed in post-processing) but a lot of details are lost: at this extreme magnification, the photo starts to look like a water-color painting.

The full image (100 ISO, 70-200mm f/4L IS lens, 106 mm, f/6.7, 1/500s).

The claim that “Raw is an extra intermediate step” is a misconception: when you print or view or zoom into an image, you are generating a new derived image – generally at a lower resolution. So in a way, JPG is a detour rather than the short route: your camera natively speaks Raw, compresses the image because storage space was formerly a major concern, after which the image is decompressed so that it can be used (viewed, edited, printed), can be sent (e-mail) or can be shared (web).

In modern software like Lightroom or Google’s Picasa, Raw and JPG both serve as valid input formats. When you adjust an image (e.g. change brightness, crop it, remove dust, adapt the contrast…) no output image file is generated. Only the (tiny) adjustment commands themselves are saved.

Photoshop and its alternatives also support all major Raw formats and can generate many different output formats. If you may edit the image again in the future, JPG is seldom recommended as the intermediate storage format because every additional detour via JPG causes more loss of quality.

Thus, although Raw has few drawbacks nowadays, here are some legitimate excuses to still use JPG:

  1. If you have a camera phone or compact camera and it probably only supports JPG. This means the camera is probably not good enough to worry about subtle differences in image quality.
  2. In general, if quality is not an issue, JPG is good enough. An extreme example: I use JPG to take pictures of street names, etc instead of writing them down ;-)
  3. If you take studio images, you may have the time and skills to tune the lighting, composition and camera settings so that you don’t need to adjust the image at all in post-processing. If you never change your image (no dust, ideal exposure, ideal contast…) JPG may be good enough. You still loose some sharpness, but for portraits sharpness may even be somewhat undesirable.
  4. If your camera has an obscure proprietary Raw format, you may want to use something else for archiving. This is seldom the case today: small manufacturers use Adobe’s well-documented (but not really open) DNG Raw format and major manufacturers get enough software support for their current and older formats anyway.

To strengthen the case for Raw, Your Honor, here are some more comparisons:

Hold the mouse over the image to swap the Raw and JPG images.

Again, the JPG version largely loses skin texture and fabric detail. You won’t really notice this on small prints, but it will limit your ability to crop or enlarge your pictures.

The corresponding full image (100 ISO, 111 mm, f/6.7, 1/500, 70-200 f/4L IS)

As a last example, here is a macro image of bees taken with a macro lens (while wearing the protective clothing you saw in the first picture).

Hold the mouse over the images to swap the Raw and JPG images.

The JPG has less sharpness for the hairs, less texture in the smooth bits and darker shadows (although the latter can likely be fixed).

The corresponding full image (320 ISO, 100 mm f/2.8 macro lens, f/5, 1/250)

More examples at higher ISO values and using different cameras can be found on here on the www.DPReview.com site. Note that when you compare different cameras, you are often also varying resolution and varying the choice of lens. But the a quick look at different cameras confirms the conclusion that a good photo made with good equipment deserves Raw.


2. Photoshop or Lightroom?

Adobe’s Photoshop dates back to the late 1980′s. It is one of the most famous software tools and even brands in the world. Unfortunately it has also grown into a huge program with many features and large add-ons (e.g. Bridge). This is partly because it is used by very diverse types of users. And this is partly because it was designed to be the single, ultimate tool for modifying or generating images.

It is consequently reasonably tricky to learn, and requires a pretty disciplined approach to manipulating images.

In 2006 Adobe launched a new product, Adobe Photoshop Lightroom, that only targets the basic needs of (serious) photographers. It is thus designed to cover pretty much everything a photographer does with photos. It focuses on helping the photographer do common tasks efficiently – rather than on providing an ultimate toolbox which can do everything… providing that you can find out how.

The main benefits of Lightroom (compared to Photoshop CS are):

  1. It is lightweight, but still targeted at professional photographers and serious amateurs. “Lightweight” implies easier to use, easier to learn and significantly less expensive.
  2. Lightroom also keeps track of your files (thus covering Bridge functionality). It does keywording, searching, browsing, etc. The files themselves can be stored with normal file names in a normal directory structure. There is support for having different versions of a file (“virtual copies”).
  3. You never create output images unless you need to send (“export”) a file to someone/somewhere else. You only store the original image. And information is recorded on what modifications you selected. This allows you to change your mind and adapt the image later without any loss of quality. It also makes it irrelevant in what order you do modifications, and it avoids having to store and track multiple intermediate or alternative versions of the same image.
    This approach actually works faster on large images because the computer only calculates changes at the resolution or the crop that you are viewing at that moment: on a screen you either see a low resolution overview image, or a high-resolution partial image. This is because screens are typically 1 to 2 MPixels while your camera is likely between 10 and 20 MPixels.
  4. There is no support for layers. Some uses of layers are handled by the previous point. But there are things you can do in Photoshop which you simply cannot do in Lightroom. Many professional photographers thus actually own both, but spend most of their time in Lightroom.
  5. Lightroom covers the entire workflow in one user interface: importing and managing collections of images, common and some less common image enhancements (“Develop”), professional quality printing, and exporting images to a website or web service.

So if the final output of your work is typically still essentially a single photo, Lightroom (or its competitor Aperture) may give you all you need in an elegant, efficient but professional-strength tool.

Here is a rather extreme example of how far Lightroom can adjust a (Raw) image.

Original version of 2011_Paris_249 (100 ISO, 105 mm, f/5.6, 1/200).

Edited version that stresses the shadows above (!) the tower.

The changes made to this particular image were:

  1. Reduced the exposure by 3.5 (!) stops. Yes, the original was a Raw file.
  2. Removing dust spots
  3. Rendering as black and white.
  4. Cropping off the bottom of the image
  5. Increasing the contrast
  6. Applying automatic lens corrections for distortion, vignetting, etc

The order in which such changes are made is irrelevant – unlike Photoshop, the enhancements are applied in a fixed order determined by Lightroom. This “fixed order” may sound inflexible, but actually allows you to apply the required changes in any order you like and backtrack on earlier decisions without having to start over.

Below is a cropped version of the original image superimposed with the final image. The images move slightly due to the applied lens correction.

Cropped original version of 2011_Paris_249. Place mouse over image to see edited version.

Obviously many images need less or even no editing. But, on the other hand, Lightroom even supports some enhancements that are fancier than those shown above: gradient filters and brush-based local enhancements.

Some of the main things you cannot do in Lightroom 3.x:

  • no layer support (although there is a form of masking)
  • no hundreds of creative filters
  • no HDR or panorama stitching (Lightroom can invoke Photoshop to do both)
  • no soft proofing of print jobs (this has been added in Lightroom 4)

So … for photographers, using Photoshop Lightroom as your main tool saves you time. And the end result is likely to be a bit better. This is simply because Lightroom was design for photographers. Photoshop nowadays targets graphic artists, website designers, engineers, print ad developers, etc. You may find that you still need Photoshop occasionally. But Lightroom provides integration support for extra tools and plugins – including Photoshop.


Candidates for additional topics in this article or series:

  • Calibrate the colors and brightness of your screen.
  • I recommend using adding artificial vignetting to many photos
  • image stabilizers as “digital tripods”. 10x slower shutter speed.
  • Beginners should never use the flash.

File size on the Canon 5D2

RAW file size

At its highest settings, the Canon EOS 5D Mark II produces 21 Mpixel RAW images. These are about 26 MByte according to the manual (page 55). This article will concentrate on RAW rather than on JPG.

File sizes versus sensor resolution for various Canon cameras

File sizes versus sensor resolution for various Canon cameras

The diagram compares the file sizes of various Canon digital SLR cameras. The solid red line is the 5D Mark II. At its highest resolution, it  produces 26 MByte files, but it also has 2 lower resolution sRAW modes (10 Mpixel and 5 Mpixel) which will be discussed later. 26 Mbyte files is slightly larger than the RAW files from the Canon 1Ds Mark III (also 21 MPixel), and significantly larger than images from Canon’s lower resolution models.

26 Mbytes for 21.1 Mpixel equals 10.3 bit/pixel. The sensor itself produces more data: 14 bit/pixel (remember that a camera pixel captures only one color per pixel – essentially a marketing decision to get you large numbers). The compression from 14 bit/pixel to 10 bit/pixel is in principle lossless – meaning that typical image files will shrink compared to the original data without any visible loss of quality (“lossless”). As usual for lossless encoding, some types of images will compress better than others. Some un-photolike images may even grow rather than shrink.

Information density of images generated by various Canon EOS camera

Information density of images generated by various Canon EOS camera

From the above data (based on the file sizes quoted in the Canon Instruction Manuals) we can conclude a few things:

  • A RAW file at the camera’s native resolution is roughly 10 bit per pixel, or about 1.25 MBytes per Mpixel.
  • JPEG files are (at their higher quality setting) about 2.5 bit per pixel, or 4× smaller than RAW. You can opt to make them 8× smaller instead of 4× smaller by using a lower quality setting (not shown) although most photographers will prefer the highest quality JPGs these cameras can produce.
  • The per-pixel quality of JPEG files seems to shrink with increasing resolution. Maybe simply because there is less difference between adjacent pixel readings in a high-res image (due to the subject, or due to lens limitations).
  • The size of RAW files (at the camera’s native resolution) actually increased in terms of bit/pixel (or MByte/MPixel). The camera models 10D → 30D → 40D → 450D → 5D2 show an increase in bit/pixel which goes 7.5 → 8.9 → 10.4 → 10.7 → 10.4.  All-in-all, more than a 35% increase, which can be partly explained by the transition from 12-bit to 14-bit pixel accuracy. So it might also be an increasing emphasis on quality. JPEG files, in contrast, are said to be limited to 8-bit values due to the JPEG standard.
  • The new lower resolution sRAW files require more storage per (Mega)pixel than the full RAW files. This is because they have use a very different internal technology despite the use of the same .CR2 file extension. Thus, the sRAW 2 format is more than half the size of the full resolution RAW while having 4× fewer pixels.

Storing 26 Mbyte images

26 MByte images can become a problem for processing (get a faster CPU with more RAM), storage (get a larger hard disk) and communications (get a faster network).

The storage part is easiest to solve: 1 TByte drives cost below €100 and hold 40 thousand 25 MB images. Large drives are often faster than smaller ones (more bit per track). I already took measures to prepare for the storage part: dual Samsung SpinPoint F HD103UJ 1Tb in a NAS.

Moving 26 Mbyte images

Ideally faster communication involves using local drives rather than network drives, or using a gigabit LAN to central storage (my choice). The LAN still doesn’t achieve 1 Gbit/s speeds, but say at 200 Mbit/s it still takes 1 seconds to download each image. Avoid wireless links as they are 10× slower. At 200 Mbit/s, the CPU rather than the LAN may be the bottleneck. A drive attached to USB 2.0 or Firewire will also give you about 200 Mbit/s.

Processing 26 Mbyte images

Processing is the tricky part: Lightroom and Aperture (Mac) show you the modifications you make but using lower resolution previews of images. In fact, they do not generate high-resolution images of any image files at all:  they just record what modifications you did, and repeat these when needed. They rely on lower-resolution “cached” images generated once in a while from the full resolution images. In Lightroom you can see this happening: when the full resolution image is fetched it display “Loading” or “Loading preview”. In contrast, typical image processing in Photoshop or intermediate processing in DxO Optics Pro involve reading and writing of full-size images.

There is obviously a lot to say about performance differences between different computers. For modern image processing software, having multiple cores helps. Having enough RAM (cheap nowadays) always helps. I am currently using a 1.8 GHz dual core Intel system with 2 GBytes of RAM.

The sRAW option

Photographers who feel that 10 or less Mpixel gives enough image quality have the option of creating and storing in “sRAW 1” (10 Mpixel -> 15 MB). This sRAW option is newish and not supported by some software that does support regular RAW. Photoshop Lightroom and Canon’s own software support sRAW. DxO and Aperture currently do not support sRAW. Photomatix Pro (HDR software) plans to support sRAW soon.

Although sRAW files have the same .CR2 file extension as traditional Canon RAW, they are technically very different. Analysis by Gao YangDave Coffin and Laurent Clevy has shown that sRAW does not store the original Bayer RGB matrix, but instead contains a 15-bit luminosity value per each pixel location, and 2 or 4 color components per block of 2×2 pixels (in the reduced resolution).

sRAW 1 (half the pixels) is suitable for many applications (e.g. portraits), but the limited third-party software support can be a problem. It is also not known yet how sRAW 1 compares in quality to a hypothetical RAW file from a camera with the same resolution. sRAW 1 files from the Canon 5D Mark II are on average 15 MBytes (12 bit/pixel) – meaning that you get a 1.7× reduction in file size in exchange for a 2× fewer pixels. The 2× pixel reduction should give 2× faster processing in programs like Photoshop because these internally use their own data formats.