Re: [CODE4LIB] Web Server Specs?
Thanks everyone for your responses. I don't think I was very clear with part of my communications. IT will actually be the ones configuring the hardware and likely installing the OS and perhaps even the web service. From there we will get superuser access and install our own dbs and packages. The hardware itself will be out of our control. It will likely be part of their Virtual Server farm. While this means it won't actually be set in stone, from a practical side, we don't want to have to go back and ask for more everytime we develop a new app. I like the idea ensuring we have resources to do both OS level imaging and docker. IT has the ability to snapshot the servers, so I think we could ask them to run this periodically on our server, but most of the changes wouldn't be happening on the OS level side, so I like the idea of Docker as well. I will definitely need to get more proficient with it. I have only played with it a bit on my local PC. Thanks, Julie. -Original Message- From: Code for Libraries On Behalf Of Kyle Banerjee Sent: Tuesday, August 7, 2018 2:07 PM To: CODE4LIB@LISTS.CLIR.ORG Subject: Re: [CODE4LIB] Web Server Specs? It might be worthwhile having a conversation to get a feel for what they're equipped to support and what the cost structure will look like for the library. Sometimes, resources that are easy and cheap to provision on the open market can be difficult and (very) expensive to obtain locally. Also find out what support looks like, what kind of disaster recovery options are available, what policies they have regarding connections/services, etc. As this is a dev environment, specific things you might want to ask about include imaging machines and rebuilding. When doing these things is easy, you can develop very aggressively. Whenever something works, take a new image you can work from in future. Whenever something blows up, dump the server and reconstitute from your latest good image. It's much easier to work if you don't have to worry about messing things up. kyle On Tue, Aug 7, 2018 at 12:08 PM Cary Gordon wrote: > Ideally, they should give you a mini-monster server so you could spin > up your own VMs or run Docker. Actually, Docker would be a great way > to go. I would want at least 8 GB RAM and 4 cores, although I would > ask for at least double that. > > Of course, in my world, I would just spin up nano and micro instances > on AWS as I needed them. I run scripts to turn them off when they are > not being used. If you need to test on bigger hardware, use spot instances. > > If hardware is free for you, then you'll be sticking to it, I presume. > > Thanks, > > Cary > > On Thu, Aug 2, 2018 at 1:56 PM, Julie Cole wrote: > > > Good point, although in a VM environment it is likely outside our > control, > > and might be hard to justify an exception for a sandbox server. > > Julie > > > > -Original Message- > > From: Code for Libraries On Behalf Of > > Edward Almasy > > Sent: Thursday, August 2, 2018 1:19 PM > > To: CODE4LIB@LISTS.CLIR.ORG > > Subject: Re: [CODE4LIB] Web Server Specs? > > > > On Aug 2, 2018, at 2:27 PM, Julie Cole wrote: > > > We plan to install a LAPP or LAMP stack on it. (maybe both > > > databases?) > > I don't see this as being a very heavy transaction oriented environment. > > Some thoughts of things we would like to do with it: > > > This will likely be a virtual server, so it won't be written in > > > stone, > > but I'm still unclear on how much to ask for in terms of CPU, Memory > > and RAM. > > > Any tips, pointers, gotchas? > > > > If possible, get an SSD for storage. In addition to the obvious > increased > > general responsiveness, it’ll make many development and maintenance > > tasks far less onerous. > > > > Ed > > > > > > -- > > Edward Almasy Director • Internet Scout > > Research Group Computer Sciences Dept • U > of > > Wisconsin - Madison > > 1210 W Dayton St • Madison WI 53706 • 3HCV+J6 > > 608-262-6606 (voice) • 608-265-9296 (fax) > > > > > > > -- > Cary Gordon > The Cherry Hill Company > http://chillco.com >
Re: [CODE4LIB] Web Server Specs?
It might be worthwhile having a conversation to get a feel for what they're equipped to support and what the cost structure will look like for the library. Sometimes, resources that are easy and cheap to provision on the open market can be difficult and (very) expensive to obtain locally. Also find out what support looks like, what kind of disaster recovery options are available, what policies they have regarding connections/services, etc. As this is a dev environment, specific things you might want to ask about include imaging machines and rebuilding. When doing these things is easy, you can develop very aggressively. Whenever something works, take a new image you can work from in future. Whenever something blows up, dump the server and reconstitute from your latest good image. It's much easier to work if you don't have to worry about messing things up. kyle On Tue, Aug 7, 2018 at 12:08 PM Cary Gordon wrote: > Ideally, they should give you a mini-monster server so you could spin up > your own VMs or run Docker. Actually, Docker would be a great way to go. I > would want at least 8 GB RAM and 4 cores, although I would ask for at least > double that. > > Of course, in my world, I would just spin up nano and micro instances on > AWS as I needed them. I run scripts to turn them off when they are not > being used. If you need to test on bigger hardware, use spot instances. > > If hardware is free for you, then you'll be sticking to it, I presume. > > Thanks, > > Cary > > On Thu, Aug 2, 2018 at 1:56 PM, Julie Cole wrote: > > > Good point, although in a VM environment it is likely outside our > control, > > and might be hard to justify an exception for a sandbox server. > > Julie > > > > -Original Message- > > From: Code for Libraries On Behalf Of Edward > > Almasy > > Sent: Thursday, August 2, 2018 1:19 PM > > To: CODE4LIB@LISTS.CLIR.ORG > > Subject: Re: [CODE4LIB] Web Server Specs? > > > > On Aug 2, 2018, at 2:27 PM, Julie Cole wrote: > > > We plan to install a LAPP or LAMP stack on it. (maybe both databases?) > > I don't see this as being a very heavy transaction oriented environment. > > Some thoughts of things we would like to do with it: > > > This will likely be a virtual server, so it won't be written in stone, > > but I'm still unclear on how much to ask for in terms of CPU, Memory and > > RAM. > > > Any tips, pointers, gotchas? > > > > If possible, get an SSD for storage. In addition to the obvious > increased > > general responsiveness, it’ll make many development and maintenance tasks > > far less onerous. > > > > Ed > > > > > > -- > > Edward Almasy > > Director • Internet Scout Research Group Computer Sciences Dept • U > of > > Wisconsin - Madison > > 1210 W Dayton St • Madison WI 53706 • 3HCV+J6 > > 608-262-6606 (voice) • 608-265-9296 (fax) > > > > > > > -- > Cary Gordon > The Cherry Hill Company > http://chillco.com >
Re: [CODE4LIB] Web Server Specs?
Ideally, they should give you a mini-monster server so you could spin up your own VMs or run Docker. Actually, Docker would be a great way to go. I would want at least 8 GB RAM and 4 cores, although I would ask for at least double that. Of course, in my world, I would just spin up nano and micro instances on AWS as I needed them. I run scripts to turn them off when they are not being used. If you need to test on bigger hardware, use spot instances. If hardware is free for you, then you'll be sticking to it, I presume. Thanks, Cary On Thu, Aug 2, 2018 at 1:56 PM, Julie Cole wrote: > Good point, although in a VM environment it is likely outside our control, > and might be hard to justify an exception for a sandbox server. > Julie > > -Original Message- > From: Code for Libraries On Behalf Of Edward > Almasy > Sent: Thursday, August 2, 2018 1:19 PM > To: CODE4LIB@LISTS.CLIR.ORG > Subject: Re: [CODE4LIB] Web Server Specs? > > On Aug 2, 2018, at 2:27 PM, Julie Cole wrote: > > We plan to install a LAPP or LAMP stack on it. (maybe both databases?) > I don't see this as being a very heavy transaction oriented environment. > Some thoughts of things we would like to do with it: > > This will likely be a virtual server, so it won't be written in stone, > but I'm still unclear on how much to ask for in terms of CPU, Memory and > RAM. > > Any tips, pointers, gotchas? > > If possible, get an SSD for storage. In addition to the obvious increased > general responsiveness, it’ll make many development and maintenance tasks > far less onerous. > > Ed > > > -- > Edward Almasy > Director • Internet Scout Research Group Computer Sciences Dept • U of > Wisconsin - Madison > 1210 W Dayton St • Madison WI 53706 • 3HCV+J6 > 608-262-6606 (voice) • 608-265-9296 (fax) > > -- Cary Gordon The Cherry Hill Company http://chillco.com
Re: [CODE4LIB] Web Server Specs?
Good point, although in a VM environment it is likely outside our control, and might be hard to justify an exception for a sandbox server. Julie -Original Message- From: Code for Libraries On Behalf Of Edward Almasy Sent: Thursday, August 2, 2018 1:19 PM To: CODE4LIB@LISTS.CLIR.ORG Subject: Re: [CODE4LIB] Web Server Specs? On Aug 2, 2018, at 2:27 PM, Julie Cole wrote: > We plan to install a LAPP or LAMP stack on it. (maybe both databases?) I > don't see this as being a very heavy transaction oriented environment. Some > thoughts of things we would like to do with it: > This will likely be a virtual server, so it won't be written in stone, but > I'm still unclear on how much to ask for in terms of CPU, Memory and RAM. > Any tips, pointers, gotchas? If possible, get an SSD for storage. In addition to the obvious increased general responsiveness, it’ll make many development and maintenance tasks far less onerous. Ed -- Edward Almasy Director • Internet Scout Research Group Computer Sciences Dept • U of Wisconsin - Madison 1210 W Dayton St • Madison WI 53706 • 3HCV+J6 608-262-6606 (voice) • 608-265-9296 (fax)
Re: [CODE4LIB] Web Server Specs?
On Aug 2, 2018, at 2:27 PM, Julie Cole wrote: > We plan to install a LAPP or LAMP stack on it. (maybe both databases?) I > don't see this as being a very heavy transaction oriented environment. Some > thoughts of things we would like to do with it: > This will likely be a virtual server, so it won't be written in stone, but > I'm still unclear on how much to ask for in terms of CPU, Memory and RAM. > Any tips, pointers, gotchas? If possible, get an SSD for storage. In addition to the obvious increased general responsiveness, it’ll make many development and maintenance tasks far less onerous. Ed -- Edward Almasy Director • Internet Scout Research Group Computer Sciences Dept • U of Wisconsin - Madison 1210 W Dayton St • Madison WI 53706 • 3HCV+J6 608-262-6606 (voice) • 608-265-9296 (fax)
Re: [CODE4LIB] Web Server Specs?
If you're planning to run postgres/mysql on the same box as your webserver, you might want to consider increasing Buddy's RAM recommendation to 4GB and constraining the DB to a fixed percentage of that total. Allocating a larger-than-typical chunk of your storage for swap (4-8GB perhaps in your case) can also buy you some additional flexibility with a sandbox VM if requests for additional memory cannot be accommodated. Typically, I would not encourage such a concession for a production box but this sounds like it is for internal-use-only. There's quite a bit you can do with 2 vCPUs / 4GB RAM (which happens to be our minimum VM deployment size these days). Low traffic web apps or nightly batch processing jobs will more than likely run just fine under these conditions. If you run into memory issues (the dreaded OOM killer!), you can tune down your webserver / database of choice and PHP to handle fewer connections at a time or stretch the interval between parallel tasks. The default tuning parameters should be mostly acceptable for this size unless you start hammering away at Apache or if your DB grows significantly. -- Tony Pulickal Lead Systems Engineer // Unix Systems Group Columbia University Libraries IT tony.pulic...@columbia.edu // (212) 851-2937 On Thu, Aug 2, 2018 at 3:35 PM, Pennington, Buddy D. wrote: > I'm not a pro at this either. :-) > > Our campus base virtual machine includes the following: > 2 Central Processing Units (CPU) > 2 GB of Memory > 1 Gigabit Ethernet Card > 1 Operating System (OS), either Windows or Linux > 200 GB of Disk space > > Adding to memory, disk space, etc. costs extra and we have had to beef up > servers for Drupal, Jira, and Confluence, though not by much. We don't run > Sierra ourselves. That's managed at another campus so I don't have specs > for that. > > Buddy Pennington > Head of Library Systems & Technology (Librarian III) > Miller Nichols Library > University of Missouri--Kansas City > > -Original Message- > From: Code for Libraries On Behalf Of Julie Cole > Sent: Thursday, August 02, 2018 2:27 PM > To: CODE4LIB@LISTS.CLIR.ORG > Subject: [CODE4LIB] Web Server Specs? > > I'm well aware that I'm asking an impossible question. If someone were to > ask me this question I would definitely answer with "it depends" on a nice > day and potentially something snarkier on a bad one. > > Still, here goes... > > I've been working in my position as Library Systems Administrator for > nearly a year at a small one campus college (our Sierra ILS shows roughly > 100,000 patrons in a 14GB database). My colleague who is a Systems > Librarian and I have been trying to get IT to give us a Web Server to do > sandbox testing and internal processing tasks. Well be careful what you > wish for... Now we have been told that we have been granted this wish and > we need to spec it. Well usually in order to spec a server you need to > know what applications you want to run on it. Guidance from our Director > is to neither be too greedy nor too skimpy. We anticipate this environment > lasting us 5 years. > > We plan to install a LAPP or LAMP stack on it. (maybe both databases?) I > don't see this as being a very heavy transaction oriented environment. > Some thoughts of things we would like to do with it: > run some customized ebook linkchecking scripts schedule queries to get > data from our ILS to a local database to keep more history and merge with > other datasets from various sources. > Install websites for callbacks for SUSHI and REST APIs ??? the future ??? > > A bit of note on our background. My colleague has a Computer Science > Degree and I have a Computer Systems Diploma, but the reality is that we > are not full fledged developers. When I got my diploma, we programmed in > C, (not C++ - that wasn't a thing yet). I've got some recent experience > with JavaScript and have been teaching myself Python just using my Windows > PC and I'm having a lot of fun with it, but I still have lots to learn. > > This will likely be a virtual server, so it won't be written in stone, but > I'm still unclear on how much to ask for in terms of CPU, Memory and RAM. > Any tips, pointers, gotchas? Things you wish you'd ask for when setting > up your own test development server? > > Thanks so much for any feedback! > Julie. >
Re: [CODE4LIB] Web Server Specs?
Thanks so much for the info. Sierra is on a different machine for us too and the IT takes care of JIRA, Confluence and WordPress, so this is just for our internal development. Julie. -Original Message- From: Code for Libraries On Behalf Of Pennington, Buddy D. Sent: Thursday, August 2, 2018 12:36 PM To: CODE4LIB@LISTS.CLIR.ORG Subject: Re: [CODE4LIB] Web Server Specs? I'm not a pro at this either. :-) Our campus base virtual machine includes the following: 2 Central Processing Units (CPU) 2 GB of Memory 1 Gigabit Ethernet Card 1 Operating System (OS), either Windows or Linux 200 GB of Disk space Adding to memory, disk space, etc. costs extra and we have had to beef up servers for Drupal, Jira, and Confluence, though not by much. We don't run Sierra ourselves. That's managed at another campus so I don't have specs for that. Buddy Pennington Head of Library Systems & Technology (Librarian III) Miller Nichols Library University of Missouri--Kansas City -Original Message- From: Code for Libraries On Behalf Of Julie Cole Sent: Thursday, August 02, 2018 2:27 PM To: CODE4LIB@LISTS.CLIR.ORG Subject: [CODE4LIB] Web Server Specs? I'm well aware that I'm asking an impossible question. If someone were to ask me this question I would definitely answer with "it depends" on a nice day and potentially something snarkier on a bad one. Still, here goes... I've been working in my position as Library Systems Administrator for nearly a year at a small one campus college (our Sierra ILS shows roughly 100,000 patrons in a 14GB database). My colleague who is a Systems Librarian and I have been trying to get IT to give us a Web Server to do sandbox testing and internal processing tasks. Well be careful what you wish for... Now we have been told that we have been granted this wish and we need to spec it. Well usually in order to spec a server you need to know what applications you want to run on it. Guidance from our Director is to neither be too greedy nor too skimpy. We anticipate this environment lasting us 5 years. We plan to install a LAPP or LAMP stack on it. (maybe both databases?) I don't see this as being a very heavy transaction oriented environment. Some thoughts of things we would like to do with it: run some customized ebook linkchecking scripts schedule queries to get data from our ILS to a local database to keep more history and merge with other datasets from various sources. Install websites for callbacks for SUSHI and REST APIs ??? the future ??? A bit of note on our background. My colleague has a Computer Science Degree and I have a Computer Systems Diploma, but the reality is that we are not full fledged developers. When I got my diploma, we programmed in C, (not C++ - that wasn't a thing yet). I've got some recent experience with JavaScript and have been teaching myself Python just using my Windows PC and I'm having a lot of fun with it, but I still have lots to learn. This will likely be a virtual server, so it won't be written in stone, but I'm still unclear on how much to ask for in terms of CPU, Memory and RAM. Any tips, pointers, gotchas? Things you wish you'd ask for when setting up your own test development server? Thanks so much for any feedback! Julie.
Re: [CODE4LIB] Web Server Specs?
I'm not a pro at this either. :-) Our campus base virtual machine includes the following: 2 Central Processing Units (CPU) 2 GB of Memory 1 Gigabit Ethernet Card 1 Operating System (OS), either Windows or Linux 200 GB of Disk space Adding to memory, disk space, etc. costs extra and we have had to beef up servers for Drupal, Jira, and Confluence, though not by much. We don't run Sierra ourselves. That's managed at another campus so I don't have specs for that. Buddy Pennington Head of Library Systems & Technology (Librarian III) Miller Nichols Library University of Missouri--Kansas City -Original Message- From: Code for Libraries On Behalf Of Julie Cole Sent: Thursday, August 02, 2018 2:27 PM To: CODE4LIB@LISTS.CLIR.ORG Subject: [CODE4LIB] Web Server Specs? I'm well aware that I'm asking an impossible question. If someone were to ask me this question I would definitely answer with "it depends" on a nice day and potentially something snarkier on a bad one. Still, here goes... I've been working in my position as Library Systems Administrator for nearly a year at a small one campus college (our Sierra ILS shows roughly 100,000 patrons in a 14GB database). My colleague who is a Systems Librarian and I have been trying to get IT to give us a Web Server to do sandbox testing and internal processing tasks. Well be careful what you wish for... Now we have been told that we have been granted this wish and we need to spec it. Well usually in order to spec a server you need to know what applications you want to run on it. Guidance from our Director is to neither be too greedy nor too skimpy. We anticipate this environment lasting us 5 years. We plan to install a LAPP or LAMP stack on it. (maybe both databases?) I don't see this as being a very heavy transaction oriented environment. Some thoughts of things we would like to do with it: run some customized ebook linkchecking scripts schedule queries to get data from our ILS to a local database to keep more history and merge with other datasets from various sources. Install websites for callbacks for SUSHI and REST APIs ??? the future ??? A bit of note on our background. My colleague has a Computer Science Degree and I have a Computer Systems Diploma, but the reality is that we are not full fledged developers. When I got my diploma, we programmed in C, (not C++ - that wasn't a thing yet). I've got some recent experience with JavaScript and have been teaching myself Python just using my Windows PC and I'm having a lot of fun with it, but I still have lots to learn. This will likely be a virtual server, so it won't be written in stone, but I'm still unclear on how much to ask for in terms of CPU, Memory and RAM. Any tips, pointers, gotchas? Things you wish you'd ask for when setting up your own test development server? Thanks so much for any feedback! Julie.
