Re: How large is too large...
Of course. All my backups are on external drives as are most of my working files. I also have Dropbox and iCloud backups of critical files. Paul > On Feb 17, 2023, at 11:16 AM, Larry Colen wrote: > > I hope that is only your first level backups, and you have a couple other > sets of backups on external drive. > >> On Feb 17, 2023, at 7:27 AM, Paul Stenquist wrote: >> >> I should clarify that about half of those are backups, so I have only about >> 3 terabytes of unique images. I don’t know how many that would be, but it >> includes a lot of unprocessed DNGs and a tiff and jpeg of each rendered >> image. So it’s irrelevant in comparison to a lightroom catalog. But the >> system works very well. >> >> On Feb 17, 2023, at 7:13 AM, Paul Stenquist wrote: >>> >>> I have approximately 6 terabytes of images on 10 drives, both spinny and >>> ssd. They’re all catalogued in Adobe Bridge, making them easy to access and >>> not dependent on software gymnastics. >>> >>> Paul >>> On Feb 17, 2023, at 7:28 AM, Rob Studdert wrote: Late to the thread, my main LR classic catalogue currently shows 689k images, I prefer it this way as I'm regularly needing to revisit previous edits and I shoot a lot of recurring jobs. I have no problems with speed during use, though it resides on a fast SSD (7GB/s), the problem is that with previews the cat occupes 143GB of storage space, it also takes a long time to check/compact and back-up the cat and when I revisit old folders where the previews have been purged already it can hang for a little. > On Thu, 12 Jan 2023 at 09:50, Rick Womer wrote: > > …for a Lightroom catalog? > > Mine has 54,590 images on it today. It seems to work fine (knocking > wood); the only problem is that the list of folders along the left side > is getting rather long (one folder per year, since 2005). It is backed up > to external drives and the cloud. > > Am I tempting fate in some way, or should I just roll along happily? > > Rick > > -- > %(real_name)s Pentax-Discuss Mail List > To unsubscribe send an email to pdml-le...@pdml.net > to UNSUBSCRIBE from the PDML, please visit the link directly above and > follow the directions. -- Rob Studdert (Digital Image Studio) https://linktr.ee/digitalimagestudio -- %(real_name)s Pentax-Discuss Mail List To unsubscribe send an email to pdml-le...@pdml.net to UNSUBSCRIBE from the PDML, please visit the link directly above and follow the directions. >>> -- >>> %(real_name)s Pentax-Discuss Mail List >>> To unsubscribe send an email to pdml-le...@pdml.net >>> to UNSUBSCRIBE from the PDML, please visit the link directly above and >>> follow the directions. >> -- >> %(real_name)s Pentax-Discuss Mail List >> To unsubscribe send an email to pdml-le...@pdml.net >> to UNSUBSCRIBE from the PDML, please visit the link directly above and >> follow the directions. >> > > -- > Larry Colen > l...@red4est.com. sent from Mirkwood > > > -- > %(real_name)s Pentax-Discuss Mail List > To unsubscribe send an email to pdml-le...@pdml.net > to UNSUBSCRIBE from the PDML, please visit the link directly above and follow > the directions. -- %(real_name)s Pentax-Discuss Mail List To unsubscribe send an email to pdml-le...@pdml.net to UNSUBSCRIBE from the PDML, please visit the link directly above and follow the directions.
Re: How large is too large...
I hope that is only your first level backups, and you have a couple other sets of backups on external drive. > On Feb 17, 2023, at 7:27 AM, Paul Stenquist wrote: > > I should clarify that about half of those are backups, so I have only about 3 > terabytes of unique images. I don’t know how many that would be, but it > includes a lot of unprocessed DNGs and a tiff and jpeg of each rendered > image. So it’s irrelevant in comparison to a lightroom catalog. But the > system works very well. > > >> On Feb 17, 2023, at 7:13 AM, Paul Stenquist wrote: >> >> I have approximately 6 terabytes of images on 10 drives, both spinny and >> ssd. They’re all catalogued in Adobe Bridge, making them easy to access and >> not dependent on software gymnastics. >> >> Paul >> >>> On Feb 17, 2023, at 7:28 AM, Rob Studdert wrote: >>> >>> Late to the thread, my main LR classic catalogue currently shows 689k >>> images, I prefer it this way as I'm regularly needing to revisit >>> previous edits and I shoot a lot of recurring jobs. I have no problems >>> with speed during use, though it resides on a fast SSD (7GB/s), the >>> problem is that with previews the cat occupes 143GB of storage space, >>> it also takes a long time to check/compact and back-up the cat and >>> when I revisit old folders where the previews have been purged already >>> it can hang for a little. >>> On Thu, 12 Jan 2023 at 09:50, Rick Womer wrote: …for a Lightroom catalog? Mine has 54,590 images on it today. It seems to work fine (knocking wood); the only problem is that the list of folders along the left side is getting rather long (one folder per year, since 2005). It is backed up to external drives and the cloud. Am I tempting fate in some way, or should I just roll along happily? Rick -- %(real_name)s Pentax-Discuss Mail List To unsubscribe send an email to pdml-le...@pdml.net to UNSUBSCRIBE from the PDML, please visit the link directly above and follow the directions. >>> >>> >>> >>> -- >>> Rob Studdert (Digital Image Studio) >>> https://linktr.ee/digitalimagestudio >>> -- >>> %(real_name)s Pentax-Discuss Mail List >>> To unsubscribe send an email to pdml-le...@pdml.net >>> to UNSUBSCRIBE from the PDML, please visit the link directly above and >>> follow the directions. >> -- >> %(real_name)s Pentax-Discuss Mail List >> To unsubscribe send an email to pdml-le...@pdml.net >> to UNSUBSCRIBE from the PDML, please visit the link directly above and >> follow the directions. > -- > %(real_name)s Pentax-Discuss Mail List > To unsubscribe send an email to pdml-le...@pdml.net > to UNSUBSCRIBE from the PDML, please visit the link directly above and follow > the directions. > -- Larry Colen l...@red4est.com. sent from Mirkwood -- %(real_name)s Pentax-Discuss Mail List To unsubscribe send an email to pdml-le...@pdml.net to UNSUBSCRIBE from the PDML, please visit the link directly above and follow the directions.
Re: How large is too large...
I should clarify that about half of those are backups, so I have only about 3 terabytes of unique images. I don’t know how many that would be, but it includes a lot of unprocessed DNGs and a tiff and jpeg of each rendered image. So it’s irrelevant in comparison to a lightroom catalog. But the system works very well. > On Feb 17, 2023, at 7:13 AM, Paul Stenquist wrote: > > I have approximately 6 terabytes of images on 10 drives, both spinny and ssd. > They’re all catalogued in Adobe Bridge, making them easy to access and not > dependent on software gymnastics. > > Paul > >> On Feb 17, 2023, at 7:28 AM, Rob Studdert wrote: >> >> Late to the thread, my main LR classic catalogue currently shows 689k >> images, I prefer it this way as I'm regularly needing to revisit >> previous edits and I shoot a lot of recurring jobs. I have no problems >> with speed during use, though it resides on a fast SSD (7GB/s), the >> problem is that with previews the cat occupes 143GB of storage space, >> it also takes a long time to check/compact and back-up the cat and >> when I revisit old folders where the previews have been purged already >> it can hang for a little. >> >>> On Thu, 12 Jan 2023 at 09:50, Rick Womer wrote: >>> >>> …for a Lightroom catalog? >>> >>> Mine has 54,590 images on it today. It seems to work fine (knocking wood); >>> the only problem is that the list of folders along the left side is getting >>> rather long (one folder per year, since 2005). It is backed up to external >>> drives and the cloud. >>> >>> Am I tempting fate in some way, or should I just roll along happily? >>> >>> Rick >>> >>> -- >>> %(real_name)s Pentax-Discuss Mail List >>> To unsubscribe send an email to pdml-le...@pdml.net >>> to UNSUBSCRIBE from the PDML, please visit the link directly above and >>> follow the directions. >> >> >> >> -- >> Rob Studdert (Digital Image Studio) >> https://linktr.ee/digitalimagestudio >> -- >> %(real_name)s Pentax-Discuss Mail List >> To unsubscribe send an email to pdml-le...@pdml.net >> to UNSUBSCRIBE from the PDML, please visit the link directly above and >> follow the directions. > -- > %(real_name)s Pentax-Discuss Mail List > To unsubscribe send an email to pdml-le...@pdml.net > to UNSUBSCRIBE from the PDML, please visit the link directly above and follow > the directions. -- %(real_name)s Pentax-Discuss Mail List To unsubscribe send an email to pdml-le...@pdml.net to UNSUBSCRIBE from the PDML, please visit the link directly above and follow the directions.
Re: How large is too large...
I have approximately 6 terabytes of images on 10 drives, both spinny and ssd. They’re all catalogued in Adobe Bridge, making them easy to access and not dependent on software gymnastics. Paul > On Feb 17, 2023, at 7:28 AM, Rob Studdert wrote: > > Late to the thread, my main LR classic catalogue currently shows 689k > images, I prefer it this way as I'm regularly needing to revisit > previous edits and I shoot a lot of recurring jobs. I have no problems > with speed during use, though it resides on a fast SSD (7GB/s), the > problem is that with previews the cat occupes 143GB of storage space, > it also takes a long time to check/compact and back-up the cat and > when I revisit old folders where the previews have been purged already > it can hang for a little. > >> On Thu, 12 Jan 2023 at 09:50, Rick Womer wrote: >> >> …for a Lightroom catalog? >> >> Mine has 54,590 images on it today. It seems to work fine (knocking wood); >> the only problem is that the list of folders along the left side is getting >> rather long (one folder per year, since 2005). It is backed up to external >> drives and the cloud. >> >> Am I tempting fate in some way, or should I just roll along happily? >> >> Rick >> >> -- >> %(real_name)s Pentax-Discuss Mail List >> To unsubscribe send an email to pdml-le...@pdml.net >> to UNSUBSCRIBE from the PDML, please visit the link directly above and >> follow the directions. > > > > -- > Rob Studdert (Digital Image Studio) > https://linktr.ee/digitalimagestudio > -- > %(real_name)s Pentax-Discuss Mail List > To unsubscribe send an email to pdml-le...@pdml.net > to UNSUBSCRIBE from the PDML, please visit the link directly above and follow > the directions. -- %(real_name)s Pentax-Discuss Mail List To unsubscribe send an email to pdml-le...@pdml.net to UNSUBSCRIBE from the PDML, please visit the link directly above and follow the directions.
Re: How large is too large...
Using LR Classic, currently at v.12.0.1 My All Photographs catalog is at about 276k images. There are about 262k in a Missing Photographs catalog. I suspect those “missing” images are the ones on my external drives, which I keep turned off mostly. Some of the “missing” are due to a recent renaming of some of my folders, it will be trivial to “find” them again when needed. My 2020, 2021, 2022, and 2023 images are in the Pictures folder on my internal drive. Currently showing 31k Every once in a while I turn on the external drives and/or plug in my backup SSDs, either to do a periodic backup of the items on my hard drive, or to open up an earlier year folder and thus “find” the missing images. 20+ years ago, before Lightroom, I was doing some SQL work on a volunteer basis, helping create and maintain large databases of names/address/etc. Working with a very fine, long since deceased, Mac-based program whose name escapes me. I toyed with the idea of creating my own image db program, but then one came on the market (pre LR still). I used it, I worried about what would happen as the catalog size grew, and made sure that I had extensive backups. Microsoft bought the program, killed it very soon after that, but by then LR was here. As Rob mentioned, the downside of a very large catalog is the speed hit you take when e.g. doing a catalog backup or when bringing in older images whose Previews have been purged. I mostly work on “recent” images. Having an “active” catalog of 20-30k images works just fine. Back early in the pandemic I started a project to organize and publish (as Blurb books) my images from our travels. Got as far as a Caribbean book (2002, 2013-2018) which required working across many older image sets. I do not recall LR being an issue. The experience did lead me to trade in my 2013 Mac for a much faster 2021 Mac but LR per se was never an issue. Stan > On Jan 11, 2023, at 5:50 PM, Rick Womer wrote: > > …for a Lightroom catalog? > > Mine has 54,590 images on it today. It seems to work fine (knocking wood); > the only problem is that the list of folders along the left side is getting > rather long (one folder per year, since 2005). It is backed up to external > drives and the cloud. > > Am I tempting fate in some way, or should I just roll along happily? > > Rick > > -- > %(real_name)s Pentax-Discuss Mail List > To unsubscribe send an email to pdml-le...@pdml.net > to UNSUBSCRIBE from the PDML, please visit the link directly above and follow > the directions. -- %(real_name)s Pentax-Discuss Mail List To unsubscribe send an email to pdml-le...@pdml.net to UNSUBSCRIBE from the PDML, please visit the link directly above and follow the directions.
Re: How large is too large...
Late to the thread, my main LR classic catalogue currently shows 689k images, I prefer it this way as I'm regularly needing to revisit previous edits and I shoot a lot of recurring jobs. I have no problems with speed during use, though it resides on a fast SSD (7GB/s), the problem is that with previews the cat occupes 143GB of storage space, it also takes a long time to check/compact and back-up the cat and when I revisit old folders where the previews have been purged already it can hang for a little. On Thu, 12 Jan 2023 at 09:50, Rick Womer wrote: > > …for a Lightroom catalog? > > Mine has 54,590 images on it today. It seems to work fine (knocking wood); > the only problem is that the list of folders along the left side is getting > rather long (one folder per year, since 2005). It is backed up to external > drives and the cloud. > > Am I tempting fate in some way, or should I just roll along happily? > > Rick > > -- > %(real_name)s Pentax-Discuss Mail List > To unsubscribe send an email to pdml-le...@pdml.net > to UNSUBSCRIBE from the PDML, please visit the link directly above and follow > the directions. -- Rob Studdert (Digital Image Studio) https://linktr.ee/digitalimagestudio -- %(real_name)s Pentax-Discuss Mail List To unsubscribe send an email to pdml-le...@pdml.net to UNSUBSCRIBE from the PDML, please visit the link directly above and follow the directions.
Re: How large is too large...
Am 11.01.23 um 23:50 schrieb Rick Womer: …for a Lightroom catalog? Mine has 54,590 images on it today. It seems to work fine (knocking wood); the only problem is that the list of folders along the left side is getting rather long (one folder per year, since 2005). It is backed up to external drives and the cloud. Lightroom doesn't appear to have problems with such large catalogs and directories, unlike ON1 and DxO which are slowed down to the point that they become almost unusable. Ralf -- Ralf R. Radermacher - Köln/Cologne, Germany Blog : http://the-real-fotoralf.blogspot.com Audio : http://aporee.org/maps/projects/fotoralf Fotos : https://www.fotocommunity.de/user_photos/770012 -- %(real_name)s Pentax-Discuss Mail List To unsubscribe send an email to pdml-le...@pdml.net to UNSUBSCRIBE from the PDML, please visit the link directly above and follow the directions.
Re: How large is too large...
My current catalog shows a shade over 140K with nary a hiccup, so I think you're good. On 1/11/23 5:50 PM, Rick Womer wrote: …for a Lightroom catalog? Mine has 54,590 images on it today. It seems to work fine (knocking wood); the only problem is that the list of folders along the left side is getting rather long (one folder per year, since 2005). It is backed up to external drives and the cloud. Am I tempting fate in some way, or should I just roll along happily? Rick -- %(real_name)s Pentax-Discuss Mail List To unsubscribe send an email to pdml-le...@pdml.net to UNSUBSCRIBE from the PDML, please visit the link directly above and follow the directions. -- %(real_name)s Pentax-Discuss Mail List To unsubscribe send an email to pdml-le...@pdml.net to UNSUBSCRIBE from the PDML, please visit the link directly above and follow the directions.
Re: How large is too large...
I will presume that you're talking about Lightroom Classic (or LR 6.14, the last perpetual license version) as they are local-storage based versions of Lightroom, unlike the later Lightroom CC which is Adobe cloud-storage based. Whenever I say LR, I mean Lightroom Classic. :) As context regards the size question: I divide my work in LR Classic into two main catalogs: "InProgress" and "CompletedWorks". - InProgress contains all the original files … raw, JPEG, TIFF, whatever the camera or device that made them used as an original file format … along with all the derivative products on the way to whatever I called "finished" or "completed". Whenever I complete a photo, I export it as a 16bit per component TIFF file at full resolution for archiving as well as at least two different JPEG renderings (one high resolution and one medium resolution) for my standard image posting operations. - CompletedWorks contains all the output products that I have deemed completed … in essence, the archived TIFF masters as well as the various sizes of JPEG output files. My InProgress catalog at this point in time contains 197,000+ image entities (originals, derivatives, and virtual copies) spanning all the time I've been using LR (since early 2004, with files imported from as far back as 1995). I use this catalog every day and LR Classic seems to have no issues with it. That's not the largest catalog I've worked with. I had a job some years back to sort and manage another professional photographer's 50 years worth of combined film image scans and digital captures. That catalog ended up having more than a half million image entities in it, and again had no operational issues. The base technology of the LR catalog is SQL Lite, which at its inception was intended to be able to contain an arbitrarily large number of records. Since then, SQL Lite by definition has acquired some limits but all of those limits are much much larger than an LR Classic catalog with a million records in it would ever achieve. The real key to how big a catalog can get and how useful it can be has much more to do with how much and how fast the storage that it is contained on is. Big catalogs build big libraries of previews and access them frequently in use, so a large catalog should be sited on a large, fast volume with LOTS of free space for quick and useful access. BTW: I saw someone posted that they store metadata back into the DNG files and therefore felt safe that they can reconstruct an LR catalog should the catalog crash. Be aware that the metadata that is stored into the DNG files is the editing and annotation data, but not anything about virtual copies or the organization of collections you might be using inside LR when editing. So, while you can reconstruct most of the editing that your original files contain that way, you cannot, for instance, reconstruct the five virtual copies of the same original with all of the different rendering efforts you put into them. You'll only get the one file with the rendering associated with the master file in that instance. The reason for my CompletedWorks catalog is that it contains exclusively completed works that take all the rendering and metadata associated with the original files and writes the pixels and metadata together, complete, as finished items. I can move those archive files to any editing system at all and have at my disposal the work that reflects all the editing time and annotations that I put into the originals, complete, independent of LR entirely. Backup frequently, and use LR's facility to optimize the catalog file when you do. You should have no problems with catalogs with a quarter million or more image entities in them. G > On Jan 11, 2023, at 2:50 PM, Rick Womer wrote: > > …for a Lightroom catalog? > > Mine has 54,590 images on it today. It seems to work fine (knocking wood); > the only problem is that the list of folders along the left side is getting > rather long (one folder per year, since 2005). It is backed up to external > drives and the cloud. > > Am I tempting fate in some way, or should I just roll along happily? > > Rick -- %(real_name)s Pentax-Discuss Mail List To unsubscribe send an email to pdml-le...@pdml.net to UNSUBSCRIBE from the PDML, please visit the link directly above and follow the directions.
Re: How large is too large...
do you store files as dng files with the lightroom settings built into the files? If so, almost nothing can go wrong, since if you somehow corrupted the catalog so that it lost files or had metadata conflicts, a fresh import of those dng files into a new catalog will reconstruct all the edits and keywords etc that you have given each image. Your previews will be gone, but they will be reconstructed too. That's the reason I embed the metadata into DNG - you are not reliant on the catalog being right. Alastair On Thu, 12 Jan 2023 at 11:50, Rick Womer wrote: > …for a Lightroom catalog? > > Mine has 54,590 images on it today. It seems to work fine (knocking wood); > the only problem is that the list of folders along the left side is getting > rather long (one folder per year, since 2005). It is backed up to external > drives and the cloud. > > Am I tempting fate in some way, or should I just roll along happily? > > Rick > > -- > %(real_name)s Pentax-Discuss Mail List > To unsubscribe send an email to pdml-le...@pdml.net > to UNSUBSCRIBE from the PDML, please visit the link directly above and > follow the directions. -- %(real_name)s Pentax-Discuss Mail List To unsubscribe send an email to pdml-le...@pdml.net to UNSUBSCRIBE from the PDML, please visit the link directly above and follow the directions.
Re: How large is too large...
First question, which version of Lightroom are you using? Adobe is taking obfuscation to new heights, since Lightroom became Lightroom Classic, something that seems mostly usable to people who have phones and fast internet, but no real camera or storage is now Lightroom, plus there are folks still on LR6 so they don’t have to pay the monthly ransom …. > On Jan 11, 2023, at 2:50 PM, Rick Womer wrote: > > …for a Lightroom catalog? > > Mine has 54,590 images on it today. It seems to work fine (knocking wood); > the only problem is that the list of folders along the left side is getting > rather long (one folder per year, since 2005). It is backed up to external > drives and the cloud. I currently run Lightroom Classic. I have two main catalogs, “Working” and “Everything”. Every so often I will export all of my “new” photos and recently edited photos to a “catchup” catalog, which then gets imported into my Everything catalog. When that is good, I then delete a bunch of stuff from my “Working” catalog so that it doesn’t get too big. I think my working catalog is somewhere between 20,000 & 80,000 images, and my everything is upwards of 800,000. Since I use jf_flickr to upload to flickr, and it doesn’t seem to talk between catalogs, I keep most of what I post to flickr in my working catalog, so that I can easily update files. I will also make special purpose catalogs as needed, and import the new/changed files back into Everything. > > Am I tempting fate in some way, or should I just roll along happily? Keep good backups. you can copy your catalog files to another drive and recover to them. You shouldn’t have problems with a catalog of that size. As to the length of the folder list, you could move the annual files into folders for each decade, so you’d have a folder for the oughts, one for the teens, and probably leave the twenties in the top level because they’re recent and more likely to be accessed. -- Larry Colen l...@red4est.com sent from ret4est -- %(real_name)s Pentax-Discuss Mail List To unsubscribe send an email to pdml-le...@pdml.net to UNSUBSCRIBE from the PDML, please visit the link directly above and follow the directions.
How large is too large...
…for a Lightroom catalog? Mine has 54,590 images on it today. It seems to work fine (knocking wood); the only problem is that the list of folders along the left side is getting rather long (one folder per year, since 2005). It is backed up to external drives and the cloud. Am I tempting fate in some way, or should I just roll along happily? Rick -- %(real_name)s Pentax-Discuss Mail List To unsubscribe send an email to pdml-le...@pdml.net to UNSUBSCRIBE from the PDML, please visit the link directly above and follow the directions.
