Hi Pedro, Thanks for your suggestions! Improved RAW handling is a focal point for the next release of Shotwell, 0.11. For example, one of the enhancements you mentioned (#1771) is already marked to be implemented as part Shotwell 0.11. What's more, I'd like to add some of my thoughts. Sometimes the reason why we have done or haven't done certain things in Shotwell isn't immediately clear, so let me chime in:
> Mimics are a bad idea ... > Using JPEGs embedded in RAW files is also a bad idea You're right. Neither mimics nor embedded JPEGs provide full access to the underlying RAW image data. A lot of this has to do with nature of RAW: RAW files have more information in them per pixel than do JPEGs, so whenever you convert from RAW to JPEG, you lose information -- usually you get a smaller color gamut and a decreased dynamic range. That said, loading and decoding RAW files is an expensive operation. You can see this yourself by running Shotwell with the --no-mimicked-images command-line option. When this option is set, Shotwell ignores mimcs and instead loads and decodes the full RAW file when you open the photo. Setting this option makes Shotwell much less responsive. Since one of the things we value in Shotwell is its speed, we want JPEGs available for each RAW photo so that we can display it quickly. The only way to do get this JPEG data -- short of implementing RAW development in Shotwell itself -- is to either create a JPEG when a RAW photo is imported (i.e., to create a mimic) or to use the JPEG data embedded in the RAW file itself. For a variety of reasons (most notably that not all RAW files have embedded JPEGs) we chose the mimic approach. We're not alone in this. For example, Apple's iPhoto uses mimics as well. You also mentioned that you didn't like Shotwell "fixing the RAW conversion to a substandard default." I'm sure you're not alone in this. As I mentioned above, a JPEG file will always contain less information than the RAW file from which it was developed. Pro photographers like you, of course, want to control exactly what information gets preserved and what gets discarded. Indeed, that's what you're doing in ufraw when you manipulate "conversion curves." Like you, we would love to have fully parameterizable RAW to JPEG conversion inside Shotwell itself. But this would be a big feature to implement. For example, we'd have to come up with a UI for manipulating conversion curves through sliders, dials, etc. Alas, our development resources are limited so we can't implement every feature we'd like to (but hey, if you're a developer and want to give it a shot, patches are gladly accepted!). So for right now, we have to pick some "substandard defaults" to use when we convert RAW photos to JPEGs inside of Shotwell. As you've said, our defaults aren't perfect, but the curves and white-balance parameters we use tend to give good results for most photos shot in well-lit rooms or in daylight. Of course, if you're shooting in low-light or otherwise working at the extremes of the dynamic range, Shotwell's defaults will not give you the best results and you're better off developing the RAW image yourself in ufraw. I hope this makes clear why we did some of the things we did in implementing RAW support in Shotwell. Like you, we'd love to have much fuller, richer RAW support, including parameterizable conversion. We just don't have the resources to make this happen in the next release or two. Regards, Lucas On Sun, May 29, 2011 at 5:13 PM, Pedro Côrte-Real <[email protected]> wrote: > I went through 1500 RAW images in the last few weeks from two events > from a Sony A700 (12MP ARW files). I'm still using f-spot since > shotwell's current RAW handling abilities are not enough. Here are my > observations from doing it that may help in designing shotwell: > > - f-spot is generally slow at showing the thumbnails from my 20k > photos. shotwell already solves this > - f-spot is pretty decent at actually displaying the raw files. It > doesn't generate any mimics, doing a conversion on the fly to display, > that's fast enough to not be too much of an issue (for sony ARW files > at least). This could potentially still be sped up by using something > like rawspeed[1] and reading RAWs as half or quarter resolution[2]. > - Using an external program to then do raw conversions (ufraw in this > case) is very cumbersome. If I'm converting 100 photos out of the > original 1000, I have to open each of them in ufraw, do changes and > then wait for ufraw to do a full conversion and save the new image in > a new file. So most of my time is actually spent waiting for ufraw to > open (re-read the RAW and apply initial settings) or to close (do a > full conversion and save the file). > - I used to shoot RAW+JPEG to get a decent RAW conversion straight out > of the camera. I then realized that the embedded thumbnail was pretty > big (1080p in my camera) to use as a reference or web image and that I > ended up doing new raw conversions for everything anyway. Since the > raw converter can't really reproduce the in-camera conversion, using > the built-in thumbnail just introduces inconsistency. > > My conclusions for shotwell would be: > > - If versions/RAW+JPEG pairs are added to shotwell it will basically > have the same RAW handling abilities as f-spot, but that shouldn't be > the real solution. (i.e., bug #1772 is only really a way to handle > RAW+JPEG shot in camera and shouldn't be the general way to handle RAW > like f-spot currently does) > - Mimics are a bad idea. Changing the RAW conversion is a basic step > in any RAW workflow. I basically only ever do a conversion and > sometimes a crop. Fixing the RAW conversion to a substandard default > and then doing color/brightness/etc correction on the output throws > away a bunch of information. > - Using JPEGs embedded in RAW files is also a bad idea as they are > inconsistent with the RAW processing on the computer. RAW conversion > for display/thumbnailing can be made fast enough for this not to > matter. > - RAW conversions shouldn't be done until a file actually needs to be > generated. In my case my final output was to upload to Flickr. Ideally > shotwell would just store the RAW conversion settings when I do > changes and show me a fast preview. Only when I actually push export > to flickr should the files be generated (and they don't need to be > saved to disk, maybe a limited size cache would make sense though). > It's fine if that takes longer as at that point I can just leave the > computer to do it as a batch. > > Bug #1771 has a lot of discussion about this. Shotwell seems to > already have a great philosophy for JPEGs where it only stores > metadata about transformations and generates the output on the fly. > The same should apply to RAW, in fact that's how things like Apple's > Aperture work and even f-spot was planning on moving in that > direction[3]. > > Pedro > > [1] http://sh0dan.blogspot.com/2009/02/introducing-rawspeed.html > [2] I've seen that half is possible and have asked the rawspeed > developer if quarter is also possible. Even if it's not, just dropping > half the resolution after reading will reduce the amount of work for > all the rest of the steps (demosaic, etc) > [3] http://mail.gnome.org/archives/f-spot-list/2010-June/msg00051.html > _______________________________________________ > Shotwell mailing list > [email protected] > http://lists.yorba.org/cgi-bin/mailman/listinfo/shotwell > _______________________________________________ Shotwell mailing list [email protected] http://lists.yorba.org/cgi-bin/mailman/listinfo/shotwell
