Lightroom 5

Lightroom 5 was released on June 9th 2013. It is a modest upgrade to Adobe’s main software package for photographers. The limited number of changes may be because it took 1 year to develop (Lightroom 4, by comparison, took 2.5 years). A new version of Lightroom may have been needed to get a clean baseline of tools that coincided with the release of Adobe Creative Cloud.

Software aspects

The new version still contain “encounterable” minor software bugs. Although these seem mainly related to the user interface, less eager adopters may want to wait for a 5.1 upgrade.

A Lightroom 5 bug
The above error appears to have popped up in Lightroom versions as long ago as 2008. But I hadn’t seen worthwhile errors in earlier Lightroom versions, so maybe LR5 was brought to the market somewhat rushed.

Lightroom 5 incidentally requires Windows 7 or 8. Windows Vista (which I still run on my laptop) is no longer supported.

The Lightroom concept

Despite its official name

Adobe® Photoshop® Lightroom® 5.0

Lightroom does not create modified image files from original files like Photoshop does. Instead it stores the adjustments you used to modify the image (e.g. “crop the image”) on disk. This is done in a so-called catalog. It automatically re-applies these adjustments whenever you view, print, edit or export the image. This saves storage space, simplifies file management and is convenient if you afterwards change your mind about any of the settings (e.g. “make the crop larger”).

From Photoshop to Lightroom

In general, Lightroom is meant for photographers who want to post-process their images, while Photoshop is nowadays mainly for graphics professionals who want to create new images. Sometimes a photographer does need Photoshop, but usually Lightroom is easier and faster to use because it is designed for photographers.

In each new Lightroom release the needs for Photoshop decreases a bit. The new licensing model for Photoshop may further accelerate the migration of photographers to Lightroom: you can still buy Lightroom, but you need to pay a yearly fee to use Photoshop (and a bunch of other Adobe tools).

Main new features in Lightroom 5

Radial gradient – This is a way to adjust a circular or elliptic area with soft edges in a number of ways. It is comparable to “dodge and burn” in the darkroom days.

Circular gradient used to add fill light, color temperature and clarity.

Previous Lightroom versions had a few tools that are somewhat comparable:

  • Graduated Filter – for modifying one end of the image. Straight.
  • Adjustment Brush – for modifying an area which you paint in with a brush. Flexible.
  • Post-crop vignetting – light/dark adjustments only. Always in the middle of the crop.

Smart previews – These are essentially about 4 MPixels versions of the images that can be used if the full images are temporarily not available. You can choose which images or directories have smart previews. The software uses available smart previews if the USB drive or (in my case) NAS containing the images is not online. If you have access to smart previews, you can edit the images just like you could edit your original image – although you cannot see the image at full resolution. In my case that would mean editing at max 2540 × 1693 resolution without being able to zoom in to see the full 21 Mpixel resolution of my Canon 5D2. The feature is useful for laptop use, but also has benefits if you need to send somebody a Lightroom catalog to view or even edit.

Smart Preview2
Smart Preview controls are below the histogram.
Smart Preview
Smart previews are internally 2540 pixel raw files in Adobe’s DNG raw format.

Straightening – The “Upright” feature corrects tilted horizons and fixes problems with perspectives in architectural photography. Unlike DxO’s Viewpoint, you don’t have to indicate a set of lines or a rectangle that are supposed to be straightened. The software detects this automatically and gives a few options within the Lens Corrections section. Note that the image is warped to get this result.

Original image
Original image
Automatically straightened
Fully straightened (automatically)
Fully straightened (automatically)

Note that there are parts missing parts in the final image. These would normally be shown in black and cropped off by the user.

You could argue that this feature is the poor man’s Tilt and Shift lens: with a camera on a level tripod a wide-angle lens should get you similar results. The wide-angle should be enough if you keep the camera level, but shifting the lens up can get the horizon below the middle of the photo if required. This was not required for the above photo.

Other new features

PNG file support – I have some PNG files in my archive that I used for creating photo books. “Synchronizing” my storage folders using Lightroom 5 now brings these files into Lightroom. The files are not too relevant as the format is seldom used for photos, but here is an example of a PNG file that I used as an illustration in a photo book:

Continuous_happinessSpot removal – Lightroom 4’s circular spot removal tool has been extended to allow you to paint away any bobs that needs replacements.

In addition, a tool is provided to make sensor dust spots show up better. Here is part of the original image before any corrections. If you look carefully while slowly scrolling the image you may discover several dust spots:

Original image (with spots)
Original image (with spots)

While using the Spot Removal (Q) tool you can enable Visualize Spots mode. This enhances edges and shows them as white pixels. Black pixels represent lack of local detail. The ten or so small circles in the sky are dust spots:

Spots highlighted by Lightroom 5
Spots highlighted by Lightroom 5

The next screenshot shows eleven spots that I had manually discovered (using Lightroom 4) and had fixed earlier. In this case, you can see that I had manually discovered (using my motion trick at 100% magnification and slightly increased contrast) pretty much the same spots that Lightroom 5 could capture. Note that finding spots in a cloudless blue sky would be easier.

Spots repair locations
Spots repair locations


Lightroom 4’s PV2012 image enhancement algorithm

In 2009 I wrote a small article here about a new class of image enhancement algorithms. Such algorithms made pictures look sharper by added local contrast and brought out details in both shadows and highlights. And they did this without adding halos around high-contrast edges such as the transition between foreground and sky. The article focused on a research paper by Farbman, Fattal, Lischinski and Szeliski (FFLS, 2008).

The brand new Lightroom 4 (and the associated versions of Adobe Camera Raw 6.7) has now incorporated similar technology that is based on a newer research paper entitled “Local Laplacian Filters: Edge-aware Image Processing with a Laplacian Pyramid” by Paris, Hasinoff and Kautz (PHK, 2011).

The implications of this new Lightroom feature for photographers are significant enough that I will gradually add more details here as I upgrade to Lightroom 4 (possible since March 5th) and get hands-on experience with it.

What’s so important about this technology?

On superficial examination, the two mentioned research papers (and there are many more where that came from) seem similar enough. They are both trying to give users the ability to make details in images more striking (local contrast enhancement) without creating undesirable artifacts such as halos.

The weird thing about this research is that they don’t distinguish between minor image tweaking (e.g. typical raw converters sharpen images to some default degree) and image modifications like HDR which also sharpen the image but can lead to unnatural results if overused. This is probably because there is no sharp boundary between enhancing a little and going overboard with your the settings.

Thus many of the example images in the research papers look like what we photographers would consider HDR photography: in HDR photography, you want to show a broad range of lightings without making the local contrast look flat, and without making the overall picture look artificial. Probably the authors would answer that HDR is an application area where this kind of algorithm is needed – but they can also be used for “normal” images shot with a single exposure in a Normal Dynamic Range situation.

Something similar applies to the Lighting module in DxO Lab’s “DxO Optics Plus”. It probably uses a similar type of approach to boost local contrast. I tend to consider it an HDR-like technique.

Alternative algorithms

A claimed key benefit PHK as chosen by Adobe is that the algorithm is simpler and thus takes less processing time. In fact, the paper’s introduction starts off by saying that the widely known (in the right circles) Laplacian Pyramid technology was underrated, leading previous researchers (including FFLS) to develop more complex algorithms to compensate for its shortcomings.

It is worth noting that the first author in PHK works for Adobe. But these are scientifically reviewed papers published in respectable conferences – meaning the authors have to be very rigorous about their claims, when these hold and what evidence and even counter-evidence there is for these claims.

Where do I find this in Lightroom 4?

Now shipping in Lightroom 4 (and Photoshop CS6 Camera Raw) – the tools for adjusting shadows, highlights, and clarity are based on a fast version of the local Laplacian filters we introduced at SIGGRAPH 2011


Should we care?

Technical tips for amateur photographers

Sometimes there are tips I don’t offer to experienced photography enthusiasts unless they ask the right question first. This is because I feel they should already know all this, and might be offended that I could imagine they didn’t – especially if the issue is pretty important.

But… in practice nobody can know everything. Especially when some things in the photography world have changed gradually or are non-trivial. So many photographers still use best practices that were suitable years ago, but may be outdated. So, pssst… here is a checklist of several such issues. Where applicable, I tried to add evidence.

1. Good cameras and photographers deserve Raw

The question is whether to set the camera so that it generates Raw image files. Your camera’s default is to generate JPG files. Raw files can be seen as the camera’s native format, and may need to be converted to JPG at some point.

Adobe called their open Raw format “Digital Negatives” or DNG. So, to borrow their analogy, the underlying question is whether to save or to automatically discard your digital negatives. If you currently generate JPG files, you are automatically developing and discarding those “negatives“. As the analogy suggests, this is automatic and thus convenient. But it has certain drawbacks.

Originally Raw files were considered a tool for pixel fetishists: the digital photo that you really needed (for printing, for mailing, for websites, for displaying) was in JPG format because most software could only handle JPG files. So Raw files added an extra step and were thus considered by many to be a waste of time, energy and storage space. Furthermore the JPG file format is well standardized, while Raw files were vendor-specific image formats that might become obsolete in the future.

This situation has changed. By now JPG is a 20 year-old image format that is still good enough as an output format, but isn’t really sophisticated enough to store the images which current camera sensors can capture. So assuming you have a good camera, one could argue that you lose a few year’s worth of camera industry innovation if you choose to exclusively work in Raw. Obviously not everybody will care, but it is at least worth knowing what your choice is.

In more technical terms, current sensors have 12 or 14 bit of accuracy and JPG was designed in the time that 8 bit was considered enough. Furthermore classic JPG was designed to discard fine detail and color nuances in order to save storage space: JPG is essentially a form of “lossy compression”. To JPG‘s credit, the tradeoff between image quality and file size is adjustable, but cameras only give you limited control over this tradeoff. Raw files, on the other hand, use “lossless compression”, store all 12-14 bits of information and produce files that may be 1.5-4× larger (this depends on the ISO setting).

Here is an example of the image quality lost by using JPG. These are 100% crops of an image of a bee keeper’s working clothes taken with my Canon 5D Mark II. The Raw and JPG images were actually simultaneously recorded in-camera and are thus automatically generated from the same exposure. I selected the camera’s highest quality setting. No changes were applied to the resulting Raw and JPG files in post-processing: these are pretty much the defaults.

Hold the mouse over the images to exchange the Raw and JPG images. Loading may take a few seconds.

The JPG version clearly has less contrast (but that could be fixed in post-processing) but a lot of details are lost: at this extreme magnification, the photo starts to look like a water-color painting.

The full image (100 ISO, 70-200mm f/4L IS lens, 106 mm, f/6.7, 1/500s).

The claim that “Raw is an extra intermediate step” is a misconception: when you print or view or zoom into an image, you are generating a new derived image – generally at a lower resolution. So in a way, JPG is a detour rather than the short route: your camera natively speaks Raw, compresses the image because storage space was formerly a major concern, after which the image is decompressed so that it can be used (viewed, edited, printed), can be sent (e-mail) or can be shared (web).

In modern software like Lightroom or Google’s Picasa, Raw and JPG both serve as valid input formats. When you adjust an image (e.g. change brightness, crop it, remove dust, adapt the contrast…) no output image file is generated. Only the (tiny) adjustment commands themselves are saved.

Photoshop and its alternatives also support all major Raw formats and can generate many different output formats. If you may edit the image again in the future, JPG is seldom recommended as the intermediate storage format because every additional detour via JPG causes more loss of quality.

Thus, although Raw has few drawbacks nowadays, here are some legitimate excuses to still use JPG:

  1. If you have a camera phone or compact camera and it probably only supports JPG. This means the camera is probably not good enough to worry about subtle differences in image quality.
  2. In general, if quality is not an issue, JPG is good enough. An extreme example: I use JPG to take pictures of street names, etc instead of writing them down 😉
  3. If you take studio images, you may have the time and skills to tune the lighting, composition and camera settings so that you don’t need to adjust the image at all in post-processing. If you never change your image (no dust, ideal exposure, ideal contast…) JPG may be good enough. You still loose some sharpness, but for portraits sharpness may even be somewhat undesirable.
  4. If your camera has an obscure proprietary Raw format, you may want to use something else for archiving. This is seldom the case today: small manufacturers use Adobe’s well-documented (but not really open) DNG Raw format and major manufacturers get enough software support for their current and older formats anyway.

To strengthen the case for Raw, Your Honor, here are some more comparisons:

Hold the mouse over the image to swap the Raw and JPG images.

Again, the JPG version largely loses skin texture and fabric detail. You won’t really notice this on small prints, but it will limit your ability to crop or enlarge your pictures.

The corresponding full image (100 ISO, 111 mm, f/6.7, 1/500, 70-200 f/4L IS)

As a last example, here is a macro image of bees taken with a macro lens (while wearing the protective clothing you saw in the first picture).

Hold the mouse over the images to swap the Raw and JPG images.

The JPG has less sharpness for the hairs, less texture in the smooth bits and darker shadows (although the latter can likely be fixed).

The corresponding full image (320 ISO, 100 mm f/2.8 macro lens, f/5, 1/250)

More examples at higher ISO values and using different cameras can be found on here on the site. Note that when you compare different cameras, you are often also varying resolution and varying the choice of lens. But the a quick look at different cameras confirms the conclusion that a good photo made with good equipment deserves Raw.

2. Photoshop or Lightroom?

Adobe’s Photoshop dates back to the late 1980’s. It is one of the most famous software tools and even brands in the world. Unfortunately it has also grown into a huge program with many features and large add-ons (e.g. Bridge). This is partly because it is used by very diverse types of users. And this is partly because it was designed to be the single, ultimate tool for modifying or generating images.

It is consequently reasonably tricky to learn, and requires a pretty disciplined approach to manipulating images.

In 2006 Adobe launched a new product, Adobe Photoshop Lightroom, that only targets the basic needs of (serious) photographers. It is thus designed to cover pretty much everything a photographer does with photos. It focuses on helping the photographer do common tasks efficiently – rather than on providing an ultimate toolbox which can do everything… providing that you can find out how.

The main benefits of Lightroom (compared to Photoshop CS are):

  1. It is lightweight, but still targeted at professional photographers and serious amateurs. “Lightweight” implies easier to use, easier to learn and significantly less expensive.
  2. Lightroom also keeps track of your files (thus covering Bridge functionality). It does keywording, searching, browsing, etc. The files themselves can be stored with normal file names in a normal directory structure. There is support for having different versions of a file (“virtual copies”).
  3. You never create output images unless you need to send (“export”) a file to someone/somewhere else. You only store the original image. And information is recorded on what modifications you selected. This allows you to change your mind and adapt the image later without any loss of quality. It also makes it irrelevant in what order you do modifications, and it avoids having to store and track multiple intermediate or alternative versions of the same image.
    This approach actually works faster on large images because the computer only calculates changes at the resolution or the crop that you are viewing at that moment: on a screen you either see a low resolution overview image, or a high-resolution partial image. This is because screens are typically 1 to 2 MPixels while your camera is likely between 10 and 20 MPixels.
  4. There is no support for layers. Some uses of layers are handled by the previous point. But there are things you can do in Photoshop which you simply cannot do in Lightroom. Many professional photographers thus actually own both, but spend most of their time in Lightroom.
  5. Lightroom covers the entire workflow in one user interface: importing and managing collections of images, common and some less common image enhancements (“Develop”), professional quality printing, and exporting images to a website or web service.

So if the final output of your work is typically still essentially a single photo, Lightroom (or its competitor Aperture) may give you all you need in an elegant, efficient but professional-strength tool.

Here is a rather extreme example of how far Lightroom can adjust a (Raw) image.

Original version of 2011_Paris_249 (100 ISO, 105 mm, f/5.6, 1/200).
Edited version that stresses the shadows above (!) the tower.

The changes made to this particular image were:

  1. Reduced the exposure by 3.5 (!) stops. Yes, the original was a Raw file.
  2. Removing dust spots
  3. Rendering as black and white.
  4. Cropping off the bottom of the image
  5. Increasing the contrast
  6. Applying automatic lens corrections for distortion, vignetting, etc

The order in which such changes are made is irrelevant – unlike Photoshop, the enhancements are applied in a fixed order determined by Lightroom. This “fixed order” may sound inflexible, but actually allows you to apply the required changes in any order you like and backtrack on earlier decisions without having to start over.

Below is a cropped version of the original image superimposed with the final image. The images move slightly due to the applied lens correction.

Cropped original version of 2011_Paris_249. Place mouse over image to see edited version.

Obviously many images need less or even no editing. But, on the other hand, Lightroom even supports some enhancements that are fancier than those shown above: gradient filters and brush-based local enhancements.

Some of the main things you cannot do in Lightroom 3.x:

  • no layer support (although there is a form of masking)
  • no hundreds of creative filters
  • no HDR or panorama stitching (Lightroom can invoke Photoshop to do both)
  • no soft proofing of print jobs (this has been added in Lightroom 4)

So … for photographers, using Photoshop Lightroom as your main tool saves you time. And the end result is likely to be a bit better. This is simply because Lightroom was design for photographers. Photoshop nowadays targets graphic artists, website designers, engineers, print ad developers, etc. You may find that you still need Photoshop occasionally. But Lightroom provides integration support for extra tools and plugins – including Photoshop.

Candidates for additional topics in this article or series:

  • Calibrate the colors and brightness of your screen.
  • I recommend using adding artificial vignetting to many photos
  • image stabilizers as “digital tripods”. 10x slower shutter speed.
  • Beginners should never use the flash.

Review of Fine Art Printing video tutorial

After two technical postings, here is some lighter reading. Over the holidays, I bought the downloadable tutorial video from the Luminous Landscape site called

From Camera To Print – Fine Art Printing Tutorial

The tutorial is presented by two accomplished photographers, Michael Reichmann and Jeff Schewe, who talk you through the steps involved in making gallery-quality “fine art” prints. This means prints for sale, prints for collectors or prints for exhibitions. Or maybe just great prints to hang on your own wall.

Printing at this level is about controlling numerous details, but also about fussing about subtle nuances and achieving repeatability. Unfortunately, even the larger differences (like the difference between prints on glossy and matte paper) are as hard to show directly on a video. But the video manages to explain all this anyway, partly by showing enlarged or exaggerated versions (e.g. for sharpening workflow) on a computer monitor.

Overall Impressions

I am a bit of a fan of the tutorial style of Reichmann, Schewe and Christopher Sanderson (the invisible videographer). I tend to be pretty selective about what I read and watch: especially when the total series of 24 files takes a little under 7 hours to watch in its entirity. But this video tutorial is un-American in the good sense of the word – it doesn’t remind you of US talk show hosts.

One of their strengths, apart from their occasional comic Statler/Waldorf moments, are that together the two presenters cover the artsy side of fine-art photography (mainly Michael Reichmann) all the way to the technical side (mainly Jeff Schewe).

Until July 2010, Michael Reichmann operated a gallery in Toronto (yes, Canada) where his work was on display. He now still sells his prints via the Internet (I own a small one). Reichmann is best known for his reviews on the Luminous Landscape website, for his landscape photography, and apparently also for training, workshops and the like.

Jeff Schewe‘s background includes a career in advertising photography. He also consults for Epson’s professional printing division (and in return gets all the printers and supplies that “he can eat”). To avoid an Epson bias, the competing printers from Canon and Hewlett-Packard are presented by Michael. Jeff also has close ties to the Adobe development teams for Photoshop and Lightroom and has co-developed a notable sharpening plug-in and has written several Photoshop-related books.

What works really nicely, is that these video sessions have a general plan but are still pretty much authentic dialogs: Michael stresses why he and gallery visitors like the feel of matte paper, Jeff reminds him that that is less relevant once the print is behind glass, etc. So it is just like hearing two experts improvising to summarize their views and experiencee in a clear and accurate manner.


You might expect a tutorial about printing to be about printers. In reality, printers themselves do not feature prominently in the tutorial. In fact, believe it or not, you never see someone loading paper into a printer or a printer actually printing. Instead the tutorial covers everything that it takes to make high quality art prints:

  • printer, ink, paper – briefly
  • color management & profiling (camera, screen, scanner, printer) – extensively
  • sharpening techniques – reasonably extensively
  • print settings (mainly Photoshop and Lightroom, both Mac & PC) – extensively
  • soft proofing in Photoshop – extensively
  • matteing and framing – extensively
  • workflow – extensively

Interestingly, in the final wrap-up, Jeff acknowledges that getting all the details right (in terms of calibration and particularly software settings) is still a “gross inconvenience” and expresses the hope that things will get simpler in the near future.

Comments and suggestions

  • The video was made in 2007. At that time, for example, the Epson 3800 and the HP B9180 were both new. The former has been replaced by the 3880 and HP discontinued it’s 17″ product line. Back then Lightroom was at version 1.1. In general,  the fact that the video shows its age on such details is not too disturbing. But maybe a text file with some notes might help until there is a newer version of the tutorial. Examples:
    • LR didn’t have soft proofing in 2007. The authors expected this glaring omission to be resolved soon. That still hasn’t happened (thank you, Adobe).
    • LR 1.1 had little control for output sharpening. Tot what degree are the new options in LR 2.3 and 3.3 good enough? Jeff has done some consultancy towards Lightroom to get their sharpening techniques more leading edge. But the viewer cannot tell to what degree that has helped to simplify the overall process in the subsequent years: should you still do output sharpening in Photoshop (possibly using Jeff’s plug-in) as explained in the video?
    • The video demonstrates what Jeff calls the “old way” where you generate a new version of the image that is optimized for a particular print setup. Obviously Lightroom (with it’s parametric philosophy) was already available, but what workflow to use for output sharpening now?
    • ICC profiles from 3rd party paper manufacturer’s were described by Jeff as being generally less reliable. Is that assessment still justified? In fact, was the assessment fair at the time? For me it is not obvious that say Epson is better at creating profiles than paper specialists like Ilford or Hahnemuhle.
  • The tutorial shows how to cut mattes (= “passe-partouts” in other major global languages like Dutch, Finnish and even French) using a wall-mounted $2000+ Speed Matte system. This is fine for high volume, but it would be nice to at least get some tips on how/whether to use a low-cost matte cutter based on a ruler and sliding matteing knife.
  • At the very end, the tutorial points you to one of the Luminious Landscape Video Journals for a list of brands of the stuff used in the tutorial. These are largely brands of minor things like adhesive tape or foam board. That list is probably only useful if you live in the US or Canada, but the video should be self-contained without this loose end, even though it is optionally sold in combination with that extra video (which I didn’t order).
  • I wouldn’t mind a discussion of how to load paper into a printer. In particular, printers have 2 or 3 ways to load sheet paper. Some have roll feeds. Many have fuzzy restrictions about what feed to use for what type of paper. Or the ability to adjust (argggh) “platten gaps” in the printer driver. Nothing is quite as simple as it seems in “fine art” printing, especially when every sheet wasted comes at quite a cost. And high-end printers don’t come with much documentation. Incidentally, why not bundle the Luminous Landscape video in DVD form with a $1000+ fine-art printer…

Notes on the main things that I learned

The video essentially covered what I was looking for in a useful and enjoyable fashion. Here are the notes I made on key points that I personally learned and need to remember (note that I was pretty up to speed on color management and Lightroom already):

  1. The black density (DMax) of matte paper is lower than of glossy paper: maybe 1.7 versus 2.4. I have seen this myself, but a confirmation that this is normal is nice.
  2. Borderless printing is strongly discouraged (because of framing implications).
  3. Really glossy papers are strongly discouraged (despite giving a better contrast). Not sure I agree. In the darkroom days, glossy was quite mainstream.
  4. Lightroom contains calibration profiles for cameras. This is automated/hidden from the user. You can’t change it (actually a workaround is demonstrated if you need to calibrate your specific camera instance).
  5. Even if your photo is sharp, you need to add a degree of output sharpening. It isn’t explained why this level of manual intervention is needed in the first place. But it is important to know that this is normal and needed.
  6. You should print at resolutions between 180 and 480 Pixels/Inch. Lower requires special upscaling tricks (and consider adding noise) which can get horribly complicated. Higher resolutions don’t help and only make life difficult for the printer driver.
  7. There is a nice demo on how to do high-end matteing. Including signing the matte in pencil, and signing the print in ink (even though that will presumably never be seen again). And particularly mounting the actual print in such a way that it can be removed if ever needed. Fancy.
  8. Use “simulate paper” and “simulate black level” when soft proofing. I still have my doubts about the “simulate paper” one though (as it predicts much more yellow results than the actual print).


If you own a serious (pigment-based, 17″ or better) printer and create or want to create gallery-quality prints, I can recommend this video tutorial: “optimal” and repeatable printing is simply a tricky business. Even if you find only a few tips for things you then do differently in the future, the time is well-spent. And it is almost as fun to watch as a live course (although you can’t see detailed print samples here). The video is obviously much cheaper than a one-day workshop, but you learn just as much. Unfortunately you can’t talk to the presenters, but you can replay parts is needed.

NAS and Lightroom performance

My digital photos are stored on an inexpensive NAS. This CH3SNAS consists of dual 3.5″ SATA drives of, in my case, 1 TByte each. Each drive contains a copy of each photo (RAID 1) for robustness. Lightroom 3 maintains a catalog of these photos (with associated keywords, metadata and a cache of previews) on the computer’s local hard disk. Unfortunately, although the NAS is fine for archiving and backup tasks, Lightroom’s access to the stored photos is rather slow. The question is thus whether I can get better read performance by tweaking this setup, or need to upgrade to a fancier NAS.

The NAS and its drives

The equipment:

  • the NAS is a small Linux-based ARM9 box with 64 MBytes of memory
  • the drives in the NAS and in my desktop PC are all:
    • Samsung Spinpoint F1 HD103UJ
    • drive specs: SATA-300, 7200 RPM, 32 MB cache
  • the NAS is connected to the client PC via a switch. The router, the NAS and the PC are capable of running Gigabit Ethernet.
  • the relevant partition is formated as RAID 1 (although I don’t recommend that any more) meaning that each file is simply stored on both drives for safety

Basic drive performance

I benchmarked one of the Samsung Spinpoint F1 HD103UJ drives mounted inside a desktop PC using HD Tune Pro. This test thus tests what the drive can do under normal (non-NAS) conditions.

Detail: I restricted the part of the drive under test to 0.75 Terrabytes because the data on the NAS was confined to a 750 GB RAID 1 partition on each drive. This doesn’t change the measurements significantly.

Samsung Spinpoint F1 HD103UJ performance

The average transfer rate is thus 84 MB/s while the average latency was 13.5 ms. In other words, the drives themselves can sustain read speeds of 5 GBytes/minute if the files are big enough. I am ignoring write performance because it gives similar results and is less relevant for my usage (“read-mostly”).

Usage of the NAS

I currently have 26,000 digital photos (JPG, RAW, occasionally other stuff) requiring 225 GBytes of space. In RAID 1, this takes 225 GBytes per drive.

The average size of a single photo is roughly 17 MBytes (a mix of recent JPGs and two types of RAW). A worst case photo (Raw, full resolution, depends on compressibility) can exceed 30 MBytes.

NAS performance across the network

Copying a 16 GByte directory consisting of 902 photos (Egypt) from the NAS to the local disk:

  • took 26.5 minutes = 11 MBytes/s = 0.6 GBytes/minute = 34 photos/minute
  • generating an average network traffic of 10.1 MByte/s (received). Note sure where the 10% discrepancy comes from.
  • CPU load on the NAS (log in with ssh, top command) is about 50%

To check this, I copied the same directory from the NAS to the NAS (to a non-RAID partition):

  • took 53.5 minutes = 4.9 MBytes/s = 0.3 GBytes/minute = 17 photos/minute
  • generating network traffic of 5 MBytes/second per direction

This is consistent (enough): the NAS now needs to read and write the data. It incidentally shows that the time needed to store data on the local hard disk in the 11 MBytes/s test case was apparently negligible.

So the problem is that the drives can read (or write) data at 5 GBytes/minute, but the NAS is only reading at 0.6 GBytes/minute. The “34 photos/minute” also implies that the NAS performance can easily limit the performance of browsing of photos that are not stored in the cache.

One reviewer, however, measured 21 MBytes/s rather than my 11 MBytes/s. So this gives hope that performance can be tweaked.

Optimizations and errors found

  • I found I had 100% CPU load on the NAS, even when running on a 100 Mbps link: 50% went to samba, 25% to inotify_itunes and 25% to inotify_upnp. Disabling iTunes and univeral Plug-and-Play using the Web interface thus got the CPU load down to about 50%. Apparently this is a bug in older versions of the CH3SNAS firmware that causes these two processes to eat all remaining idle time. Apart from wasting power, they undoubtedly don’t help performance.
  • The NAS was still configured to run at 100 Mbps, despite having a 1000 Mbps Ethernet link to the router (and beyond).
  • Adobe itself just announced that the imminent version of Lightroom would fix “Library: Sub-optimal preview rendering performance could impact application performance“. Whatever that means, it is always welcome.

Checking the NAS performance locally

[ coming ]

So where is the bottleneck?

[ coming ]

Lightroom 3 review

Adobe released Lightroom 3.0 on June 8th, after eight months of public beta testing.

I simply kept using LR 2.7 during the beta testing period as the beta version didn’t allow you to easily import LR 2.x databases, and I didn’t want to run any risk with my existing catalog data. So, even though official releases can still have some bugs, I upgraded to LR 3.0 as soon as the final version was available.

For an overview of what’s new in Lightroom 3.0, see for example Adobe’s own site.

Upgrade process

Lightroom 3.0 installs itself alongside the LR 2.x version. When the software is first run, a new LR v3 catalog (=metadata database) is generated alongside the old version 2 catalog. This means you can go back to the old version if necessary. For my catalog containing 25000 images (and over 100,000 keywords), the conversion took about 15 minutes.

Lens correction modules

Lightroom 3 has the option to correct vignetting, lens distortion and lateral chromatic aberration for

  • Canon (26 lenses and 2 point-and-shoots),
  • Nikon (7 lenses and 1 point-and-shoot),
  • Sigma (2 APC-C lenses and 3 full-frame lenses)
  • Sony (Sony DT 18-200mm only), and
  • Tamron (Tamron DI 28-75mm only)

This can be seen as a simple equivalent to DxO’s Optics Pro lens correction modules: the program automatically corrects these defects based on calibration data provided by Adobe (in some cases with support from the lens maker). I still hope that Adobe will acquire DxO’s technology – but this seems less likely now that Lightroom 3.0 does the low-hanging fruit part of what DxO does.

There are Canon lenses supported at present (2 of my 3 lenses; 100mm f/2.8 macro missing):

Canon lens support in Lightroom 3

The first lens is incidentally the Canon PowerShot G10/G11 point-and-shoot camera, but can also be used for the Canon PowerShot S90. The number of supported lenses will likely continue to grow: modules can be provided by Adobe, third parties, or even by end-users (Adobe provides software for this).

Support for managing video files

My directory tree containing all my pictures also contain a few dozen short HD video fragments made using my Canon 5D Mark II. It is a good idea to run “Synchronize Folder…” on the file system because this allows Lightroom 3.0 to find and import these videos. In my case, it also picked up some JPGs that my daughter had made with her camera and had manually placed in the directory tree. The support for videos is currently pretty basic: you can see a thumbnail, can view it using an external application (e.g. Windows Media Player), can add keywords, and can (obviously) export the file – which in this case just means copying the file as-is.

This isn’t much, but should be enough for now to prevent the following scenario that was easily possible with Lightroom 2 with a newer model camera:

  1. you take hundreds of pictures (JPG or Raw) with your DSL, but also a video (.MOV)
  2. you use Lightroom 2.x to import the pictures from your flash card. It warns that there are some movie files, but it doesn’t do anything with them.
  3. you are eager to see your pictures, so you start running Lightroom (adding keywords, deleting the weak images, etc.).
  4. you put your flash card back in the camera and… reformat the flash card: the video files are now lost.

Lightroom 3, in contrast, imports both the pictures and any file format that it recognizes as videos (I have seen .mov, .mp4, .avi work).

The 2003 versus the 2010 “process”

Adobe’s original image improvement “flow” or “process” was getting a bit out of date (it hasn’t fundamentally changed since Lightroom was introduced in 2003). So the Lightroom engineers needed a way to improve this without causing old photo’s to suddenly start looking slightly different. Thus by default, Lightroom 3 still uses the “2003 process” for existing images in catalogs and uses a new “2010 process” for anything that is newly imported. You have full control over which of process you want – these are just the defaults. The main improvement in the 2010 process is supposedly the handling of high-ISO images.

Below is a 100% crop of a raw image taken with a Canon 5D Mk II using a 24-104mm f/4L IS USM lens. The lens is good, but not great, so we can see some lens artifacts when we zoom in all the way.

Image EXIF data: ISO 200 with Highlight Tone Protection enabled (essentially underexposed!), 32mm, f/6.7, 1/250, tripod, raw @ 21 MPixels. In addition, the original image was by 1 stop too dark (probably due to spot metering). The HTP and the underexposure together mean that the dark parts of the image exhibit chroma noise – even at 200 ISO. Warning: the differences between the images is very small. I will point them out, but if you want to compare them, you can download the files and compare them in a slideshow-like tool.

100% crop of a 21 MPixel image using Lightroom 3.0 using 2010 process without lens correction

In this image (remember that you are seeing only 1.1% of the surface area of the full image – the full image is 10× wider and 10× higher), look for:

  • the purple fringe at border between sleeve and Leigh’s arm. This is chromatic aberration. It is not too visible here because we are not too far from the center of the image and the image quality would have been visibly worse at 24mm.
  • the purple color noise in the lady’s gray sweater and my blue sweater. Surprising in a 200 ISO image, but this is again because of the HTP setting and accidental underexposure.
  • the moiré in the striped pink blouse. A resolution of 21 MPixel may sound more than high enough, but it is actually not too high by modern standards: it corresponds to the same pixel pitch as an 8 MPixel APS-C camera.
Test image using Lightroom 3.0, 2010 process and lens correction

This image is very similar to the previous one. But look for:

  • There is less/no purple fringe at border between sleeve and arm. The lens correction module for the 24-105mm lens has automatically fixed this.
  • When you compare this image to the one without lens correction, you see slightly different cropping on the left side. Check out the pearl necklace. This is due to the circa 1% distortion: you lose a few pixels.
Test image using Lightroom's 2003 process with lens correction

This image is again quite similar to the previous two – but things to look for:

  • there seems to be less purple color noise in the lady’s gray sweater and my blue sweater.

Adobe obviously intends the 2010 process to outperform the 2003 process. In this case, the 2003 process does a better job (argggh; the first comment by a reader incidentally seems to confirm this). Adobe demonstrates the differences between the two processes mainly using high-ISO images.

Test image using DxO v6.2 with lens correction (and black level and exposure tweaking to match the overall appearance of the Lightroom output)

The same file processed using DxO Optics Pro version 6.2. DxO specializes in noise reduction and correction of lens aberations. Using default settings, I would say it has gotten rid of the chroma noise, but at the cost of detail in the sweaters. This essentially means that when your image is too noisy, you use spatial low-pass filtering to reduce the noise – at some loss of detail. You can tune all these settings in both Lightroom and DxO, so you may be able to fix this by moving away from the default settings – after all it is a critical trade-off. Note also that the aliasing in the pink blouse is less than in the Lightroom images, suggesting more effective de-mosaicing filtering.

Lens correction benefits

The cropped image used above does show too much lens correction. So let’s look at another image shot using a Canon EF 24-105mm f/4L lens. This image with lots of straight lines and indirect lighting incidentally shows a hall where Belgian coal miners used to shower after their shift.

ISO 200, 24mm, f/7.1, 1/100, full image (move mouse over image to see result of lens correction)

If you compare the two images, you see clearly that the lens has quite some distortion at 24mm (and with a full frame sensor). The website even calls this 4.3% distortion “massive“. A direct comparison also shows that the light fall-off of 1.5 stops in the corners (this is at f/7.1; it would have been more at f/4).

100% crop of a corner (move mouse over image to see result of lens correction)

The cycan and magenta fringes are clearly visible (both around the roof and between the tiles) and are largely corrected using the lens correction module. In the crop, both images are distortion corrected for practical reasons. It is worth noting that although all of this is pretty advanced stuff, you only need to click on a checkbox to activate lens correction.

iPhone support

The iPhone 3G and 3GS are both supported with respect to the camera inside these phones. On the one hand these are popular camera’s and undoubtedly have medium quality optics. Possibly Adobe added this as a bonus for people with fancy cameras who also use iPhones. Anyone using the iPhone as a main camera probably doesn’t care too much about image quality.

I am also pretty sure that the Canon PowerShot S90 is also supported. In fact, the Adobe software silently did distortion correction for this model without informing the user or giving users the option to enable or disable the feature. This was a design decision by Canon: correct residual lens aberrations in Canon software and where possible also in major 3rd party software.

Tethering your camera

Connecting your camera via a USB cable to a laptop or desktop is easy and can be useful. Every picture you take is sent over the USB cable and shows up in Lightroom pretty much immediately. The picture that shows up in Lightroom can automatically be given some keyword or preset or get the same adjustments as the previous image.

You can either trigger the camera’s shutter using the camera’s shutter button(s), or trigger the shutter from the computer (using a mouse or keyboard). You can also see important camera settings like ISO/aperture/shutter_speed/white_balance, but you cannot adjust these from the computer. I didn’t manage to start a video this way, but that may not be terribly useful anyway.

So the tethering works (at least with a hand full of recent cameras) and is easy to use. But don’t expect the ability to really control the camera from your armchair.

Oddities and bugs

  • Lens correction and image resizing. The lens correction module fixes distortion, resulting in a warped picture that is then automatically cropped back to a rectangle. You lose some pixels at the edges. The resulting image size in pixels is, however, identical to the original image size. I guess that choice is ok for casual users, but how about demanding users?Another way to explain what I mean: if you straighten the image using Lightroom (e.g. rotate it by 1 degree), you get a different image size and different aspect ratio. Distortion correction is somewhat comparable, but behaves differently.
  • File count in the keyword hierarchy. I have a keyword hierarchy that includes Locations > Europe (1) > Belgium (32) > Brussels (61) > Manneke Pis (3). The numbers indicate the number of pictures. In Lightroom 2.x the number following Europe would show the total number of images classified as Europe, including images that had only Belgium as keyword. This is clearly no longer the case. I probably only have 1 image that is labeled Europe that is not attributed to any specific country. I can click to get all Europe images (11017), but how do I get the odd one? If I cannot find it easily, why show me the number?
  • Tethered shooting. Turning off the camera while it was still connected to the computer for tethered shooting, gave a weird colored animation on the LCD on the back of the Canon 5D Mark II: essentially the camera thought it still needed to store a file. The animation looked like it might be designed for the WiFi adapter (which I don’t have) because something similar happens when the camera starts messing with FTP and HTTP protocols to push files to a nearby server.
  • Editing video capture time. Lightroom 3 doesn’t claim to be able to edit the capture time of videos, but this is a bit of an inconvenience. I had previously (LR 2.x) adjusted the times of a set of images to match the local timezone in which they were shot. Now I wanted to do the same for the videos in LR 3.0. It can’t. So now my videos show up in the wrong locations when the files are sorted based on capture time. I guess the capture time can be edited: the time is stored somewhere in/with the file itself. And if absolutely necessary, you could edit just the capture time as stored in the database (and risk losing the change if you resynchronize metadata).
  • Lens correction and freedom of choice. By default, lens correction will use the lens the image was taken with. Surprisingly, Lightroom 3 also lets you use the correction models for other lenses. Even lenses from other brands and for lenses that don’t have the focal length you are using. This is nice for playing around with (“what if I select a fish-eye”) or to use a similar modular if the one you need is not available. But there is no warning if you select a “wrong” module: it gets stored in your catalog.
  • Deletion. When a single image is viewed within a directory, and the image is deleted, Lightroom loses track of where you were within the directory. This applies for both Development view and Library view. This behavior is different than Lightroom 2, and a bit of a nuisance.

Lightroom 3 beta

Adobe has released a public beta of Lightroom 3. It can be evaluated/tested for free until April 2010. The Beta is explicitly not intended for production use. A series of free short “Scott Kelby” videos demonstrating the main new features can be found here.

Lightroom 3 beta "About"
Lightroom 3 beta "About"

Installation and migration

You can install Lightroom 3 while keeping your installed copy of Lightroom 2.  Adobe, however, doesn’t recommend doing any production work with the Lightroom 3 Beta. In fact, importing of Lightroom 2 (or 1) databases is disabled in the Beta.

Although you can import new photos, you can import existing photos from your current Lightroom database along with their metadata by exporting the metadata from Lightroom 2 to the file system using: Metadata > Save Metadata to Files (Ctrl S). This generates .XMP sidecar files for  RAW files and inserts metadata into JPG images (changing the file system date while doing so). The .XMP sidecar files (which are readable in Notepad) is automatically read when the  images are imported into Lightroom 3. The end results is that you get the images registered with your keywords, crop settings, lighting adjustments, etcetera in Lightroom 3. The process  takes a while because as far as Lightroom 3 is concerned these are new pictures (e.g. new thumbnails and previews need to be generated).

You may be able to use the same trick to save “non-production” work done in Lightroom 3 back into Lightroom 2, but this will probably give problems with new features.

Main new features

  • Easier control of printing layout
    There is a new Custom Package mode whereby you can drag and resize you images. One nice feature there is to enable crop marks as lines. That allows you to align the image on the pages.
  • Improved noise reduction – to presumably reduce the need for DxO, Noise Ninja, etc. DxO incidentally also just claimed to have improved their noise reduction.
  • Ability to add a watermark to the image during export or printing.
  • Export of slide shows as a stand-alone PDF files
    The PDF file can contain an audio track, but the audio had some glitches. I don’t know if that is a Lightroom issue or an Acrobat Reader issue.
  • Export of slide show as a stand-alone video file
    The video is in the .mp4 format and a few resolutions up to full-HD are supported. There are no easy ways to provide video that can be burned to DVDs: this is for playback on computers or gadgets. The slide show can have an accompanying audio track and the user can easily adapt the slide show to match the duration of the audio track. Here is an example slide show with some of my pictures that lasts 99 seconds – I know the music doesn’t match the images.
  • Integration of Flickr online gallery
  • Fancier control of vignetting (but there was already quite a bit of control)
  • Ability to add grain (comparable to DxO’s Filmpack, but the grain is controlled using just two sliders (DxO tries to emulate a number of classic film types)

Main missing features

Soft-proofing, as provided in PhotoShop, is still missing. It is a well-known omission and may still show up in the final release.

Impression so far

The functional additions are not too shocking, but possibly more will be added before the final release. The program is still a bit buggy, although these were mainly user interface issues rather than anything particularly blocking.