Hi Ben, Thanks for helping me think this through. I did some further digging and found that backup for the bundle I described would cost $10 more a month, and they make it sound like adding resources is a simple matter. I'm inclined to start at this level and then add on if we have performance issues. Our current database is probably fewer than 15,000 bibs, but I expect it to double in size within a year. We are still cataloging a good sized backlog.
Ciao, Bruce > On Nov 20, 2015, at 12:19 AM, Ben Shum <[email protected]> wrote: > > Hi Bruce, > > Theoretically, I think that ought to be enough resources in general > for a reasonable starting system. But I honestly do not have exact > metrics for system specifications to draw from. Other folks might > suggest more appropriate specs based on their experience. I'm going > to do some thinking aloud next... > > For a moment or so, I wondered on average how large the PostgreSQL > metabib indexes would look like for a library with 30k titles. Using > a totally, terribly rough example on this, I have access to an > Evergreen library's database that I know which has roughly 10k live > bibs equaling around 1.3 GB of metabib data content (using Galen > Charlton's method for analyzing Evergreen database size: > https://galencharlton.com/blog/2010/05/database-server-disk-space-usage-in-evergreen/). > So if you're three times that, that could mean you might use up 3.9 GB > (or more if your bibs are well defined). Not to mention all the other > parts of the database. *Note: these numbers are very rough and quite > possibly wrong in either direction.* > > Having a large database of contents could affect overall system > performance, because if you do not have enough memory to keep your > Evergreen database happy in memory, it will have to hit the disk to > retrieve necessary data more frequently. That said, because Linode > uses SSDs for disk storage, I wonder if PostgreSQL performance issues > would be mitigated anyways with the generally faster I/O with that > technology. Unfortunately, that is not something I have easy > experience measuring... though hey, I once ran a copy of our > production Evergreen database (with 1.2 million bibs) on a Thinkpad > laptop with 2 core i7 CPU, 16 GB of RAM, and 500 GB SSD and it felt > like I was searching faster than our old production environment (made > up of lots of clustered servers and regular disk storage). So you > never know what magic SSDs can deliver... :) > > I know from my own personal experience using Linode, that there are > also plans like the one you describe where you can add on extra RAM > with additional cost. Not sure if that's worth looking into long-term > either. In any case, it would seem that you may have some options to > alter the system specifications later if necessary to expand the > Hi Bruce, > > Theoretically, I think that ought to be enough resources in general > for a reasonable starting system. But I honestly do not have exact > metrics for system specifications to draw from. Other folks might > suggest more appropriate specs based on their experience. I'm going > to do some thinking aloud next... > > For a moment or so, I wondered on average how large the PostgreSQL > metabib indexes would look like for a library with 30k titles. Using > a totally, terribly rough example on this, I have access to an > Evergreen library's database that I know which has roughly 10k live > bibs equaling around 1.3 GB of metabib data content (using Galen > Charlton's method for analyzing Evergreen database size: > https://galencharlton.com/blog/2010/05/database-server-disk-space-usage-in-evergreen/). > So if you're three times that, that could mean you might use up 3.9 GB > (or more if your bibs are well defined). Not to mention all the other > parts of the database. *Note: these numbers are very rough and quite > possibly wrong in either direction.* > > Having a large database of contents could affect overall system > performance, because if you do not have enough memory to keep your > Evergreen database happy in memory, it will have to hit the disk to > retrieve necessary data more frequently. That said, because Linode > uses SSDs for disk storage, I wonder if PostgreSQL performance issues > would be mitigated anyways with the generally faster I/O with that > technology. Unfortunately, that is not something I have easy > experience measuring... though hey, I once ran a copy of our > production Evergreen database (with 1.2 million bibs) on a Thinkpad > laptop with 2 core i7 CPU, 16 GB of RAM, and 500 GB SSD and it felt > like I was searching faster than our old production environment (made > up of lots of clustered servers and regular disk storage). So you > never know what magic SSDs can deliver... :) > > I know from my own personal experience using Linode, that there are > also plans like the one you describe where you can add on extra RAM > with additional cost. Not sure if that's worth looking into long-term > either. In any case, it would seem that you may have some options to > alter the system specifications later if necessary to expand the > environment. > > Hope some of this proves useful and spurs thought and suggestions by others. > > Cheers, > > -- Ben > >> On Thu, Nov 19, 2015 at 9:45 PM, Bruce Willms <[email protected]> >> wrote: >> Hi Ben, et al., >> >> Thanks for the information. I thought I had sent a reply a long time ago, >> but I just discovered that it somehow got hung up in my outbox, and I can't >> dislodge it. So I'm sending another. >> >> Would a set-up like the one below be sufficient for a smallish instance of >> Evergreen, or do you think we would need something more robust? Our >> collection is small, probably less than 30,000 titles, and we don't expect >> to do much circulation, being essentially a research library. >> >> 4 GB RAM >> 4 CPU Cores >> 96 GB SSD Storage >> 4 TB Transfer >> 40 Gbps Network In >> 500 Mbps Network Out >> >> This bundle would cost $40 a month, which is quite reasonable in my book. I >> don't know yet what sort of backup or other support is provided for this >> price. >> >> Any advice is greatly appreciated. >> >> Best regards, >> Bruce >> >> >> -----Original Message----- >> From: Open-ils-general >> [mailto:[email protected]] On Behalf Of Ben >> Shum >> Sent: Wednesday, November 11, 2015 8:25 PM >> To: Evergreen Discussion Group <[email protected]> >> Subject: Re: [OPEN-ILS-GENERAL] Linux VPS >> >> Hi Bruce, >> >> I've used Linode for personal applications like IRC and testing web >> services, but have not applied it towards any live Evergreen systems. >> I have created test droplets with DigitalOcean too, back when they were one >> of the leaders in use of SSD disk storage, and that was certainly a nice >> hosted environment. The SSDs make things speedier when you don't >> necessarily have all the memory you'd like to run things fast. >> >> These are some areas I would be check before using any third-party host >> environment: >> >> 1) Making sure to spec out a powerful enough system (or more than one >> instance if needed). Getting the right amount of resources for CPU, RAM, >> disk space can be very important when putting together a proper Evergreen >> system. Even a test server could malfunction on less than 2 GB of RAM (with >> memory killers wrecking your installed instance). So having an appropriate >> sized instance is important. >> >> 2) Knowing what your data protection services are like. In the past, with >> groups like DigitalOcean, they had issues where the data in the instances >> could be leaked due to not properly deleting content between node rebuilds >> (which has since been fixed, as far as I last read). >> But how safe is your data "in the cloud"? What rules govern how your >> instances are accessed? With library data, I try to be cautious on this >> front. >> >> I'll let others chime in when they can, but hopefully this gets some >> conversation going for you. >> >> Good luck in your investigations, and feel free to ask further questions as >> applicable. >> >> -- Ben >> >>> On Thu, Nov 5, 2015 at 10:12 PM, Bruce Willms <[email protected]> >>> wrote: >>> We are just gearing up to set up an instance of Evergreen and are >>> wondering if anyone is running the system on a hosted VPS service like >>> Linode. It seems like a reasonable solution in our case since we >>> don’t have IT staff to support a server, etc. I’m curious to know if >>> others have done this and what their experience has been – good, bad, >>> or otherwise. Or if this is a recipe for disaster. >>> >>> >>> >>> I’m new to this list and to Evergreen, and looking for some pointers >>> from experienced hands. >>> >>> >>> >>> Bruce Willms >>> >>> East Side Freedom Library >>> >>> St. Paul, MN >>> >>> >>> >>> >>> >>> >>> >>> >> >> >> >> -- >> Benjamin Shum >> Evergreen Systems Manager >> Bibliomation, Inc. >> 24 Wooster Ave. >> Waterbury, CT 06708 >> 203-577-4070, ext. 113 >> > > > > -- > Benjamin Shum > Evergreen Systems Manager > Bibliomation, Inc. > 24 Wooster Ave. > Waterbury, CT 06708 > 203-577-4070, ext. 113
