It’s been quite a long time since my last post. I was busy with work and other stuff, but what is more important, i was on to port my photographic post-processing workflow to linux.
Workflow seems to be a popular word among photographers nowadays, and there are as many workflows as there are photographers. In this post I will introduce the first part of my (former) workflow to you in order to show you an interesting discovery.
My (former) workflow began like this:
Mainly, i shoot raw. Not because my camera does not make decent jpegs, but i like to be in control. I have a directory structure for my photos (actually, there are two, one for the original files, and one for the derivative files. I store them separately), what is extendible, and makes backing up to dvd-s easy, and i want applications to adapt to my habits (workflow) rather then force me to adapt to theirs.
After downloading the photos from memory card (or, as it is often in my case, from a portable hard drive, with built in card reader to wich i have copied the contents of my card on site) and immediately backing them up to a separate hard drive, i convert the raws to DNG-s.
There are web-wide arguments on DNG, if it is the answer or not to the question of the (photographic) life, the (photographic) universe and everything, and a very long but very interesting conversation on it (from Stuart Nixon, Peter Krogh, Kevin Connor, and others) can be found on openraw.org.
Why I have chosen DNG? Portability. Oh yes, i forgot to tell you, my wokflow was developed with portability, and application independence in mind.
Raw converters never write to the raw files. The raw files are from-camera originals, and shall remain so. Usually they are closed, not documneted standards, an in order not to corrupt them, raw-editors do not write them. They usually create a separate file, wich contains the alterations the user have made. And every raw-converter does it on its own way (even the tools the raw converters offer differ slightly). They even place theese files differently (some of them place them to the folder of the image files, while others make a separate directory), and none of them can read the sidecar files of the others (naturally
). If you have processed a photo with one type of converter, you won’t be able to view the result later on a different converter.
The only exception is DNG. DNG stores the original raw file + metadata about this file in a container. The metadata is about the alterations the user made (in a standardized form) and data on the photo (exif, iptc, xmp, and whatewer). And it is an open standard. Not a perfect one, but usable. And the applications, that implement the standard can read, interpret and write the DNG files (writing means altering the metada about the original embedded raw file, and not directly altering the embedded original raw file).
There is no DNG converter for linux yet (afaik), mostly because linux users (and developers) see the DNG move as a blatant attempt to take over the RAW format market by a jealous software house not currently a player (writing the two words ‘DNG’ and ‘linux’ to google, the first result will be this article).
So far, the only solution – in my view – is DNG. OpenRAW is a very respectable movement, but please, the article that pops up if you go to the openRaw site is 15 months old! And – again afaik – open documentation of the raw formats will not solve the problem of the sidecar files (and thus application independence). Even if a raw file is documented, it does not contain a place for additional metadata (on image alterations). And a little addition: The original title of the above linked article, the Notes on the future of Open RAW formats, and a look at DNG was “DNG is not the answer”, but it was renamed a few days after its publishing. And as the comments after the article shows, Mr Nixon stated things that were wrong, and were rather his assumptions than facts. Now who is blatant here? (more on what was right and wrong in that article and in its comments is here and here by Barry Pearson, (the page was checked for its accuracy by Juergen Specht from openRAW))
I feel that for the members of the openRAW, DNG is the Antichrist, while Adobe is the Satan itself. In my view, the purpose of the two initiations have several common points. DNG has a versioning system, and maybe if members of openRAW would suggest alterations on DNG, then the next release of DNG (if Adobe would have implemented theese theoretical alterations) could be “the” openRAW. DNG is very well accepted, and together, Adobe and openRaw would be able to persuade more manufacturers to implement a common standard.
October 23, 2007 at 8:12 pm
[...] more here [...]
November 24, 2007 at 6:03 pm
Hello vivosz,
I’ve found this blog very interesting. Thanks for sharing.
In a couple of your posts, you’ve talked about trying to replicate your photo workflow from windows to ubuntu. I would find it very interesting if you would share your current windows workflow and toolset with us. It’s always nice to see where people are coming from
JDG
November 25, 2007 at 2:49 pm
Ok, here is my workflow:
In most of the cases, i shoot raw. I also use a portable hard drive with a built in card reader. During travels, when i run out of cardspace, i copy the full cards to the hard drive, and use them again. From a security point of view, it is risky (all the photos on a single drive), but have no better solution yet.
When i copy the contents of cards/hdd to the computer, i copy them to two separate hard disks, to have a backup of the originals. Then I convert the raws to dng, and enter some metadata (creator, location, event, so on, what is valid for all the photos).
For this i used the Adobe DNG Converter, and later Adobe Lightroom.
Before the emergence of Lightroom I used Adobe Bridge for evaluation and sorting, and developing, but Lightroom was better in many ways (while in others it was – for me – weaker).
Initial sorting (deleting the technically flawed or redundant photos), marking the “good” photos (about one in every ten).
The came metadata tagging (people, location, and all information that is not valid for every photo).
Developing of the DNG-s, usually somewhat automated, but still manually. Usually only the starred photos get really much attention.
Creating of jpeg-s from the dng-s, with a moderate amount of sharping and noise reduction.
At this point come cataloging, for wich i used Iview Media Pro (now it is called Microsoft expression Media). I cataloged the original files (the developed dng-s) and derivative files in separate catalogs.
I make a backup to a separate hd, and delete the initial backups.
After it is done, if i want to have some prints, i examine the catalog of the dng-s, mark the ones i want to print, and convert them to tiffs with no sharping and noise reduction. For noise reduction i used NeatImage, but only in the case of photos shot on ISO 400 or above. Sharpening here is a bit tricky issue, it depends on where i wanted to print my photos. I don’t like to print large quantities at home, so after a longer trip my pictures usually ended up printed in a lab. But lab printing needs different sharpening than inkjet printing (both for the different technology and for the different resolution they require). Photokit Sharpener is a great tool for this.
After finishing i save the files as jpeg-s.
Usually i don’t do any retouching or stuff with the photos.
I organize my photos to two separate folder trees, one for dng-s, and one for derivatives. The folder trees are two level deep. On the lover level there are directories with a date stamp and an event or location identifier. These contain the photos. These folders are organized to DVD-r sized container folders (the name of these container folders have image id-s (DNG or derivative), number (the number of the disc in the collection) and date stamp (the date of the first folder in the container). This system makes it easy to back up my photos. Whenever a container folder gets full (its size reaches 4.3GB) i copy it to a disc (to two discs, to be proper). If i ever will use different discs (Blu ray or HDDVD), the system can remain unchanged, only the size of the containers will change.
I usually don’t delete the photos from the hard disk, but use a separate (external) hard drive to store them. If i want to have a photo for something, i don’t have to look for the dvd-s.
I have learnt much from the book of Peter Krogh: The DAM Book, Digital Asset Management for Photographers.
November 25, 2007 at 8:14 pm
Thank you.
May 9, 2008 at 1:57 pm
Hello vivosz,
I regularly use Linux wt work (completely not-related to photography), but at home I’ve being using my windows laptop, given up the Linux tools for photography ‘workflow’.
However lately I was wondering if there was something new around, considering maybe to give linux a try again.
That’s how I got to your blog, I really hope that your Sagittarius efforts will give great results!
Anyway, i am writing you to make you notice the blueMarine project: http://bluemarine.tidalwave.it/
I haven’t got chance to try it myself, but it looks promising and seems it could be exactly what a workflow software dependant needs (I need Lightroom non-destructive editing, so far)…
May 14, 2008 at 1:13 pm
Thanks for your comment.
About two months ago I promised the author (ore one of them) of BlueMarine to give the app a test, but haven’t done so far.
Your comment is a second sign for me, so i will take this business more seriously