Re: 500 instances of tomcat on the same server
Eric, On 6/28/21 13:08, Eric Robinson wrote: -Original Message- From: Christopher Schultz Sent: Monday, June 28, 2021 8:54 AM To: users@tomcat.apache.org Subject: Re: 500 instances of tomcat on the same server Eric, On 6/25/21 22:58, Eric Robinson wrote: We can run 75 to 125 instances of tomcat on a single Linux server with 12 cores and 128GB RAM. It works great. CPU is around 25%, our JVMs are not throwing OOMEs, iowait is minimal, and network traffic is about 30Mbps. We're happy with the results. Now we're upping the ante. We have a 48-core server with 1TB RAM, and we're planning to run 600+ tomcat instances on it simultaneously. What caveats or pitfalls should we watch out for? Are there any hard limits that would prevent this from working as expected? If you have the resources, I see no reason why this would present any problems. On the other hand, what happens when you need to upgrade the OS on this beast? You are now talking about disturbing not 72-125 clients, but 600 of them. There are two load-balanced servers, each with adequate power to support the whole load. When we want to maintain Server A, we drain it at the load balancer and wait for the last active connection to complete. Then we reboot/maintain the server and add it back into the rotation gracefully. Sounds good. I'm curious, are you using the LoadBalancerDrainingValve for that purpose? What are you using for your load-balancer and/or reverse-proxy? If I had a beast like this, I'd run VMWare (or similar) on it, carve it up into virtual machines, and run fewer clients on each just for the sheer flexibility of it. We considered doing it that way. Performance is top priority, so we ultimately decided to run the instances on metal rather than introducing a few trillion lines of OS code into the mix. We might consider containerizing. If this is already a virtualized/cloud environment, then I think you're doing it wrong: don't provision one huge instance and use it for multiple clients. Instead, provision lots of small instances and use them for fewer (or even 1) at a time. That makes sense until you know the environment better. It's a canned application and we're not the publisher. Breaking it out this way gives us the ability to present each customer and a unique entity to the publisher for support purposes. When their techs connect, the sandbox allows them to troubleshoot and support our mutual customer independently, which puts them in an environment their techs are comfortable with, and removed the risk of them doing something that impacts everybody on the server (or in the VM, if we used those). Okay. I'm sure I don't understand, but if you have heterogeneous support getting involved, to me it would be even more important to isolate all those applications from each other. Maybe you mean in-application support for mutual customers and not in-OS, etc. support. All I can tell you is we've been running it this way for 15 years and we've never looked back and wished we were doing it differently. That's a good position to be in. :) -chris - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: 500 instances of tomcat on the same server
All, On 6/29/21 11:33, Eric Robinson wrote: -Original Message- From: Berneburg, Cris J. - US Sent: Tuesday, June 29, 2021 7:16 AM To: users@tomcat.apache.org Subject: RE: 500 instances of tomcat on the same server Eric and Mark Just curious... Eric> We can run 75 to 125 instances of tomcat on a single Linux server Eric, Do you have or need a centralized way of managing all those instances? Hi Cris and Mark, We have about 1500 instances of tomcat (750 load-balanced virtual services across 20 physical servers). We currently manage the environment with scripts. Due to the simplicity and consistency of our internal deployment standards, it works well and is pretty easy to manage on an instance-by-instance basis, but a web console where we can see the environment as a whole, or filter on portions of it, would be amazing! A few years ago, a team from Cerner health came to ApacheCon and introduced their product called Jwala that was intended to do this kind of thing. You can find their presentation slides and video linked from the Tomcat presentations page: http://tomcat.apache.org/presentations.html The product appears to have undergone very little development in the past 4 years: it looks like they dropped it on GitHub and mostly forgot about it. -chris - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: 500 instances of tomcat on the same server
> -Original Message- > From: Berneburg, Cris J. - US > Sent: Tuesday, June 29, 2021 7:16 AM > To: users@tomcat.apache.org > Subject: RE: 500 instances of tomcat on the same server > > Eric and Mark > > Just curious... > > Eric> We can run 75 to 125 instances of tomcat on a single Linux server > > Eric, Do you have or need a centralized way of managing all those instances? Hi Cris and Mark, We have about 1500 instances of tomcat (750 load-balanced virtual services across 20 physical servers). We currently manage the environment with scripts. Due to the simplicity and consistency of our internal deployment standards, it works well and is pretty easy to manage on an instance-by-instance basis, but a web console where we can see the environment as a whole, or filter on portions of it, would be amazing! > It sounds like different support groups connect to their own instances, if I > understand correctly. > Same software vendor, different technicians. When they call, they are seeking to support an individual customer. We give them an interface through which they see a sandboxed representation of that customer's deployment. The servers themselves are multi-tenanted, but when the support techs connect, it looks to them like a single instance. Each customer is running the same canned webapp, but possibly different versions of it, with different memory configurations, and requiring different versions of tomcat and the JVM. > Mark> if there are changes we could make to Tomcat that would it easier > Mark> to run and manage that many instances do let us know. > Mark> We'd be happy to consider them. > > Mark, did you already have something in mind? Like a TC Manager-manager? > Some sort of dashboard that is able to perform TC Manager ops against all > the instances? > > -- > Cris Berneburg > CACI Senior Software Engineer > > > > > This electronic message contains information from CACI International Inc or > subsidiary companies, which may be company sensitive, proprietary, > privileged or otherwise protected from disclosure. The information is > intended to be used solely by the recipient(s) named above. If you are not > an intended recipient, be aware that any review, disclosure, copying, > distribution or use of this transmission or its contents is prohibited. If you > have received this transmission in error, please notify the sender > immediately. > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org Disclaimer : This email and any files transmitted with it are confidential and intended solely for intended recipients. If you are not the named addressee you should not disseminate, distribute, copy or alter this email. Any views or opinions presented in this email are solely those of the author and might not represent those of Physician Select Management. Warning: Although Physician Select Management has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: 500 instances of tomcat on the same server
Eric and Mark Just curious... Eric> We can run 75 to 125 instances of tomcat on a single Linux server Eric, Do you have or need a centralized way of managing all those instances? It sounds like different support groups connect to their own instances, if I understand correctly. Mark> if there are changes we could make to Tomcat that would it Mark> easier to run and manage that many instances do let us know. Mark> We'd be happy to consider them. Mark, did you already have something in mind? Like a TC Manager-manager? Some sort of dashboard that is able to perform TC Manager ops against all the instances? -- Cris Berneburg CACI Senior Software Engineer This electronic message contains information from CACI International Inc or subsidiary companies, which may be company sensitive, proprietary, privileged or otherwise protected from disclosure. The information is intended to be used solely by the recipient(s) named above. If you are not an intended recipient, be aware that any review, disclosure, copying, distribution or use of this transmission or its contents is prohibited. If you have received this transmission in error, please notify the sender immediately. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: 500 instances of tomcat on the same server
No need to be discouraged. Docker is just a set of tools. You can still use docker to create images, but you dont need docker to use those images in a container. K8s is using industry standard containerd. https://kubernetes.io/blog/2020/12/02/dont-panic-kubernetes-and-docker/ John Larsen On Mon, Jun 28, 2021 at 2:22 PM Eric Robinson wrote: > Guido, > > I think you intended that message for me, not Brian. Thanks much for the > feedback. I have been reading about Kubernetes, but I got discouraged when > I saw that they dropped Docker support, since Docker seems to be the most > popular containeriziation technology. Also, most of the Kubernetes > tutorials I saw on YouTube seem to approach it as a dev platform, and we're > not developers. > > -Eric > > > > -Original Message- > > From: Guido Jäkel > > Sent: Monday, June 28, 2021 2:43 PM > > To: Brian Wolfe > > Cc: Tomcat Users List > > Subject: Re: 500 instances of tomcat on the same server > > > > Dear Brian, > > > > please take the time to read about Linux Kernel namespaces as the > technical > > base of containers. It's like two viewpoints to one thing. Take the > network > > namespace as an example: From the conceptual point of view it looks like > > you have N indipended, functional identical "IP Stacks". But from the > > technical point of view, it's just the "well known" single instance just > with an > > additional field at all items that need this (packets, routing tables, > ...) to take > > a tag value that identify the namespace instance. > > > > You may use namespaces with the raw tools like enterns or with LXC or > > Dockers. During runtime of a started container, there's nothing more you > > have to trust but the kernel because for the basics, there's no need of > > additional userland processes to keep a container running. > > > > To run an application in a "container", you start it with a bunch of > instances of > > this namespaces, at least the process namespace. You'll probably take the > > same name for the technical namespace instances - from the conceptual > > point of view this is the name of the container. > > > > Most will start something like the init binary located in a directory > tree of a > > small Linux distribution userland. This may "boot" common services and > the > > result may act like an "indipended platfrom". But you may also launch > just > > single high-level applications like a JVM running a Tomcat. > > > > That's very close to your architecture, but much more easy to handle. > For the > > network stack e.g. you may use the same ports for listeners and have the > full > > range of ports available for connections in each namespace. There are > > different ways available to route the traffic, but in any case you may > use > > individual IPs in each namespace. > > > > greetings > > > > Guido > > > > On 2021-06-28 19:22, Brian Wolfe wrote: > > > Generally, I'd agree too. We are considering using containers, but I'm > > > not yet sure what that buys us in terms of stability. > > > > - > > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > > For additional commands, e-mail: users-h...@tomcat.apache.org > > Disclaimer : This email and any files transmitted with it are confidential > and intended solely for intended recipients. If you are not the named > addressee you should not disseminate, distribute, copy or alter this email. > Any views or opinions presented in this email are solely those of the > author and might not represent those of Physician Select Management. > Warning: Although Physician Select Management has taken reasonable > precautions to ensure no viruses are present in this email, the company > cannot accept responsibility for any loss or damage arising from the use of > this email or attachments. >
RE: 500 instances of tomcat on the same server
Guido, I think you intended that message for me, not Brian. Thanks much for the feedback. I have been reading about Kubernetes, but I got discouraged when I saw that they dropped Docker support, since Docker seems to be the most popular containeriziation technology. Also, most of the Kubernetes tutorials I saw on YouTube seem to approach it as a dev platform, and we're not developers. -Eric > -Original Message- > From: Guido Jäkel > Sent: Monday, June 28, 2021 2:43 PM > To: Brian Wolfe > Cc: Tomcat Users List > Subject: Re: 500 instances of tomcat on the same server > > Dear Brian, > > please take the time to read about Linux Kernel namespaces as the technical > base of containers. It's like two viewpoints to one thing. Take the network > namespace as an example: From the conceptual point of view it looks like > you have N indipended, functional identical "IP Stacks". But from the > technical point of view, it's just the "well known" single instance just with > an > additional field at all items that need this (packets, routing tables, ...) > to take > a tag value that identify the namespace instance. > > You may use namespaces with the raw tools like enterns or with LXC or > Dockers. During runtime of a started container, there's nothing more you > have to trust but the kernel because for the basics, there's no need of > additional userland processes to keep a container running. > > To run an application in a "container", you start it with a bunch of > instances of > this namespaces, at least the process namespace. You'll probably take the > same name for the technical namespace instances - from the conceptual > point of view this is the name of the container. > > Most will start something like the init binary located in a directory tree of > a > small Linux distribution userland. This may "boot" common services and the > result may act like an "indipended platfrom". But you may also launch just > single high-level applications like a JVM running a Tomcat. > > That's very close to your architecture, but much more easy to handle. For the > network stack e.g. you may use the same ports for listeners and have the full > range of ports available for connections in each namespace. There are > different ways available to route the traffic, but in any case you may use > individual IPs in each namespace. > > greetings > > Guido > > On 2021-06-28 19:22, Brian Wolfe wrote: > > Generally, I'd agree too. We are considering using containers, but I'm > > not yet sure what that buys us in terms of stability. > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org Disclaimer : This email and any files transmitted with it are confidential and intended solely for intended recipients. If you are not the named addressee you should not disseminate, distribute, copy or alter this email. Any views or opinions presented in this email are solely those of the author and might not represent those of Physician Select Management. Warning: Although Physician Select Management has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments.
RE: 500 instances of tomcat on the same server
> -Original Message- > From: Brian Wolfe > Sent: Monday, June 28, 2021 12:23 PM > To: Tomcat Users List > Subject: Re: 500 instances of tomcat on the same server > > I tend to agree with the initial assessment from Mark, your only issue would > be on the OS level. # of file descriptors for connections. That many tomcat > servers and your gonna start using a lot of ports and push the OS limits on > file > read/write capabilities. > Those are some of my concerns as well, which is why I asked the question. I can work around the limitation on ephemeral client ports by adding additional IPs to the box and using the localSocketAddress property of Connector/J. I can set the max file descriptors to something like half a million. Are there other potential limitations you can think off? > From an architecture perspective you should probably work on moving to a > more modern deployment model of containerization of these apps. You > would be better served by containerizing each customer deployment and > running them on a kubernetes cluster. you can avoid the need for having > large machines and scale more appropriately. and moving between hardware > would be as simple as adding/removing nodes to your cluster. It is 2 cents well spent. I have also considered Kubernetes and containerization, but I don't yet understand it well enough to know exactly how it benefits me. >It sounds like > the apps must be simple to be able to scale it to different clients like that. > just my 2 cents. > Not simple, but predictable. We've been hosting it for over decade, and we have a good feel for its resource utilization. -Eric > On Mon, Jun 28, 2021 at 1:12 PM Eric Robinson > wrote: > > > > > > > > > > > > -Original Message- > > > From: Mark Thomas > > > Sent: Monday, June 28, 2021 9:04 AM > > > To: users@tomcat.apache.org > > > Subject: Re: 500 instances of tomcat on the same server > > > > > > On 28/06/2021 14:53, Christopher Schultz wrote: > > > > Eric, > > > > > > > > On 6/25/21 22:58, Eric Robinson wrote: > > > >> We can run 75 to 125 instances of tomcat on a single Linux server > > > >> with > > > >> 12 cores and 128GB RAM. It works great. CPU is around 25%, our > > > >> JVMs are not throwing OOMEs, iowait is minimal, and network > > > >> traffic is about 30Mbps. We're happy with the results. > > > >> > > > >> Now we're upping the ante. We have a 48-core server with 1TB RAM, > > > >> and we're planning to run 600+ tomcat instances on it simultaneously. > > > >> What caveats or pitfalls should we watch out for? Are there any > > > >> hard limits that would prevent this from working as expected? > > > > If you have the resources, I see no reason why this would present > > > > any problems. > > > > > > > > On the other hand, what happens when you need to upgrade the OS > on > > > > this beast? You are now talking about disturbing not 72-125 > > > > clients, but 600 of them. > > > > > > > > If I had a beast like this, I'd run VMWare (or similar) on it, > > > > carve it up into virtual machines, and run fewer clients on > > > > each just for the sheer flexibility of it. > > > That just moves the goal posts. You'll have the same issue when the > > > hypervisor needs updating (which admittedly may need a reboot less > > > often than the OS). > > > > > > > If this is already a virtualized/cloud environment, then I think > > > > you're doing it wrong: don't provision one huge instance and use > > > > it for multiple clients. Instead, provision lots of small > > > > instances and use them for fewer (or even 1) at a time. > > > > > > But it adds the overhead of an OS for each instance. And costs if > > > you > > have to > > > pay for that OS instance. > > > > > > > The overhead issue is an important factor. The other is the fact that > > it's a canned app, supported by the publisher, and doing it our way > > pays big dividends in terms of that workflow. > > > > > As always there are trade-offs to be made and the "right" answer > > > will > > vary > > > based on circumstances and what you are trying to optimize for. I do > > agree > > > that, generally, more smaller instances will be a closer fit to more > > > use > > cases > > > but that is only a general answer. > > > > > >
Re: 500 instances of tomcat on the same server
Dear Brian, please take the time to read about Linux Kernel namespaces as the technical base of containers. It's like two viewpoints to one thing. Take the network namespace as an example: From the conceptual point of view it looks like you have N indipended, functional identical "IP Stacks". But from the technical point of view, it's just the "well known" single instance just with an additional field at all items that need this (packets, routing tables, ...) to take a tag value that identify the namespace instance. You may use namespaces with the raw tools like enterns or with LXC or Dockers. During runtime of a started container, there's nothing more you have to trust but the kernel because for the basics, there's no need of additional userland processes to keep a container running. To run an application in a "container", you start it with a bunch of instances of this namespaces, at least the process namespace. You'll probably take the same name for the technical namespace instances - from the conceptual point of view this is the name of the container. Most will start something like the init binary located in a directory tree of a small Linux distribution userland. This may "boot" common services and the result may act like an "indipended platfrom". But you may also launch just single high-level applications like a JVM running a Tomcat. That's very close to your architecture, but much more easy to handle. For the network stack e.g. you may use the same ports for listeners and have the full range of ports available for connections in each namespace. There are different ways available to route the traffic, but in any case you may use individual IPs in each namespace. greetings Guido On 2021-06-28 19:22, Brian Wolfe wrote: > Generally, I'd agree too. We are considering using containers, but I'm not > yet sure what that buys us in terms of stability. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: 500 instances of tomcat on the same server
I tend to agree with the initial assessment from Mark, your only issue would be on the OS level. # of file descriptors for connections. That many tomcat servers and your gonna start using a lot of ports and push the OS limits on file read/write capabilities. >From an architecture perspective you should probably work on moving to a more modern deployment model of containerization of these apps. You would be better served by containerizing each customer deployment and running them on a kubernetes cluster. you can avoid the need for having large machines and scale more appropriately. and moving between hardware would be as simple as adding/removing nodes to your cluster. It sounds like the apps must be simple to be able to scale it to different clients like that. just my 2 cents. On Mon, Jun 28, 2021 at 1:12 PM Eric Robinson wrote: > > > > > > -Original Message- > > From: Mark Thomas > > Sent: Monday, June 28, 2021 9:04 AM > > To: users@tomcat.apache.org > > Subject: Re: 500 instances of tomcat on the same server > > > > On 28/06/2021 14:53, Christopher Schultz wrote: > > > Eric, > > > > > > On 6/25/21 22:58, Eric Robinson wrote: > > >> We can run 75 to 125 instances of tomcat on a single Linux server > > >> with > > >> 12 cores and 128GB RAM. It works great. CPU is around 25%, our JVMs > > >> are not throwing OOMEs, iowait is minimal, and network traffic is > > >> about 30Mbps. We're happy with the results. > > >> > > >> Now we're upping the ante. We have a 48-core server with 1TB RAM, and > > >> we're planning to run 600+ tomcat instances on it simultaneously. > > >> What caveats or pitfalls should we watch out for? Are there any hard > > >> limits that would prevent this from working as expected? > > > If you have the resources, I see no reason why this would present any > > > problems. > > > > > > On the other hand, what happens when you need to upgrade the OS on > > > this beast? You are now talking about disturbing not 72-125 clients, > > > but 600 of them. > > > > > > If I had a beast like this, I'd run VMWare (or similar) on it, carve > > > it up into virtual machines, and run fewer clients on each just > > > for the sheer flexibility of it. > > That just moves the goal posts. You'll have the same issue when the > > hypervisor needs updating (which admittedly may need a reboot less often > > than the OS). > > > > > If this is already a virtualized/cloud environment, then I think > > > you're doing it wrong: don't provision one huge instance and use it > > > for multiple clients. Instead, provision lots of small instances and > > > use them for fewer (or even 1) at a time. > > > > But it adds the overhead of an OS for each instance. And costs if you > have to > > pay for that OS instance. > > > > The overhead issue is an important factor. The other is the fact that it's > a canned app, supported by the publisher, and doing it our way pays big > dividends in terms of that workflow. > > > As always there are trade-offs to be made and the "right" answer will > vary > > based on circumstances and what you are trying to optimize for. I do > agree > > that, generally, more smaller instances will be a closer fit to more use > cases > > but that is only a general answer. > > > > Generally, I'd agree too. We are considering using containers, but I'm not > yet sure what that buys us in terms of stability. > > > Mark > > > > - > > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > > For additional commands, e-mail: users-h...@tomcat.apache.org > > Disclaimer : This email and any files transmitted with it are confidential > and intended solely for intended recipients. If you are not the named > addressee you should not disseminate, distribute, copy or alter this email. > Any views or opinions presented in this email are solely those of the > author and might not represent those of Physician Select Management. > Warning: Although Physician Select Management has taken reasonable > precautions to ensure no viruses are present in this email, the company > cannot accept responsibility for any loss or damage arising from the use of > this email or attachments. > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > > -- Thanks, Brian Wolfe https://www.linkedin.com/in/brian-wolfe-3136425a/
RE: 500 instances of tomcat on the same server
> -Original Message- > From: Mark Thomas > Sent: Monday, June 28, 2021 9:04 AM > To: users@tomcat.apache.org > Subject: Re: 500 instances of tomcat on the same server > > On 28/06/2021 14:53, Christopher Schultz wrote: > > Eric, > > > > On 6/25/21 22:58, Eric Robinson wrote: > >> We can run 75 to 125 instances of tomcat on a single Linux server > >> with > >> 12 cores and 128GB RAM. It works great. CPU is around 25%, our JVMs > >> are not throwing OOMEs, iowait is minimal, and network traffic is > >> about 30Mbps. We're happy with the results. > >> > >> Now we're upping the ante. We have a 48-core server with 1TB RAM, and > >> we're planning to run 600+ tomcat instances on it simultaneously. > >> What caveats or pitfalls should we watch out for? Are there any hard > >> limits that would prevent this from working as expected? > > If you have the resources, I see no reason why this would present any > > problems. > > > > On the other hand, what happens when you need to upgrade the OS on > > this beast? You are now talking about disturbing not 72-125 clients, > > but 600 of them. > > > > If I had a beast like this, I'd run VMWare (or similar) on it, carve > > it up into virtual machines, and run fewer clients on each just > > for the sheer flexibility of it. > That just moves the goal posts. You'll have the same issue when the > hypervisor needs updating (which admittedly may need a reboot less often > than the OS). > > > If this is already a virtualized/cloud environment, then I think > > you're doing it wrong: don't provision one huge instance and use it > > for multiple clients. Instead, provision lots of small instances and > > use them for fewer (or even 1) at a time. > > But it adds the overhead of an OS for each instance. And costs if you have to > pay for that OS instance. > The overhead issue is an important factor. The other is the fact that it's a canned app, supported by the publisher, and doing it our way pays big dividends in terms of that workflow. > As always there are trade-offs to be made and the "right" answer will vary > based on circumstances and what you are trying to optimize for. I do agree > that, generally, more smaller instances will be a closer fit to more use cases > but that is only a general answer. > Generally, I'd agree too. We are considering using containers, but I'm not yet sure what that buys us in terms of stability. > Mark > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org Disclaimer : This email and any files transmitted with it are confidential and intended solely for intended recipients. If you are not the named addressee you should not disseminate, distribute, copy or alter this email. Any views or opinions presented in this email are solely those of the author and might not represent those of Physician Select Management. Warning: Although Physician Select Management has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: 500 instances of tomcat on the same server
> -Original Message- > From: Christopher Schultz > Sent: Monday, June 28, 2021 8:54 AM > To: users@tomcat.apache.org > Subject: Re: 500 instances of tomcat on the same server > > Eric, > > On 6/25/21 22:58, Eric Robinson wrote: > > We can run 75 to 125 instances of tomcat on a single Linux server with > > 12 cores and 128GB RAM. It works great. CPU is around 25%, our JVMs > > are not throwing OOMEs, iowait is minimal, and network traffic is > > about 30Mbps. We're happy with the results. > > > > Now we're upping the ante. We have a 48-core server with 1TB RAM, and > > we're planning to run 600+ tomcat instances on it simultaneously. > > What caveats or pitfalls should we watch out for? Are there any hard > > limits that would prevent this from working as expected? > If you have the resources, I see no reason why this would present any > problems. > > On the other hand, what happens when you need to upgrade the OS on this > beast? You are now talking about disturbing not 72-125 clients, but 600 of > them. > There are two load-balanced servers, each with adequate power to support the whole load. When we want to maintain Server A, we drain it at the load balancer and wait for the last active connection to complete. Then we reboot/maintain the server and add it back into the rotation gracefully. > If I had a beast like this, I'd run VMWare (or similar) on it, carve it up > into > virtual machines, and run fewer clients on each just for the sheer > flexibility > of it. > We considered doing it that way. Performance is top priority, so we ultimately decided to run the instances on metal rather than introducing a few trillion lines of OS code into the mix. We might consider containerizing. > If this is already a virtualized/cloud environment, then I think you're doing > it > wrong: don't provision one huge instance and use it for multiple clients. > Instead, provision lots of small instances and use them for fewer (or even 1) > at a time. > That makes sense until you know the environment better. It's a canned application and we're not the publisher. Breaking it out this way gives us the ability to present each customer and a unique entity to the publisher for support purposes. When their techs connect, the sandbox allows them to troubleshoot and support our mutual customer independently, which puts them in an environment their techs are comfortable with, and removed the risk of them doing something that impacts everybody on the server (or in the VM, if we used those). All I can tell you is we've been running it this way for 15 years and we've never looked back and wished we were doing it differently. > -chris > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org Disclaimer : This email and any files transmitted with it are confidential and intended solely for intended recipients. If you are not the named addressee you should not disseminate, distribute, copy or alter this email. Any views or opinions presented in this email are solely those of the author and might not represent those of Physician Select Management. Warning: Although Physician Select Management has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: 500 instances of tomcat on the same server
Mark, On 6/28/21 10:04, Mark Thomas wrote: On 28/06/2021 14:53, Christopher Schultz wrote: Eric, On 6/25/21 22:58, Eric Robinson wrote: We can run 75 to 125 instances of tomcat on a single Linux server with 12 cores and 128GB RAM. It works great. CPU is around 25%, our JVMs are not throwing OOMEs, iowait is minimal, and network traffic is about 30Mbps. We're happy with the results. Now we're upping the ante. We have a 48-core server with 1TB RAM, and we're planning to run 600+ tomcat instances on it simultaneously. What caveats or pitfalls should we watch out for? Are there any hard limits that would prevent this from working as expected? If you have the resources, I see no reason why this would present any problems. On the other hand, what happens when you need to upgrade the OS on this beast? You are now talking about disturbing not 72-125 clients, but 600 of them. If I had a beast like this, I'd run VMWare (or similar) on it, carve it up into virtual machines, and run fewer clients on each just for the sheer flexibility of it. > That just moves the goal posts. You'll have the same issue when the hypervisor needs updating (which admittedly may need a reboot less often than the OS). Agreed. My assertion is that the hypervisor should require updates far less frequently. My AWS EC2 instances can stay up for months without being forcibly rebooted, for example. (I tend to restart them, anyway, for e.g. OS updates, but the point is the same.) If this is already a virtualized/cloud environment, then I think you're doing it wrong: don't provision one huge instance and use it for multiple clients. Instead, provision lots of small instances and use them for fewer (or even 1) at a time. But it adds the overhead of an OS for each instance. And costs if you have to pay for that OS instance. As always there are trade-offs to be made and the "right" answer will vary based on circumstances and what you are trying to optimize for. I do agree that, generally, more smaller instances will be a closer fit to more use cases but that is only a general answer. -chris - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: 500 instances of tomcat on the same server
On 28/06/2021 14:53, Christopher Schultz wrote: Eric, On 6/25/21 22:58, Eric Robinson wrote: We can run 75 to 125 instances of tomcat on a single Linux server with 12 cores and 128GB RAM. It works great. CPU is around 25%, our JVMs are not throwing OOMEs, iowait is minimal, and network traffic is about 30Mbps. We're happy with the results. Now we're upping the ante. We have a 48-core server with 1TB RAM, and we're planning to run 600+ tomcat instances on it simultaneously. What caveats or pitfalls should we watch out for? Are there any hard limits that would prevent this from working as expected? If you have the resources, I see no reason why this would present any problems. On the other hand, what happens when you need to upgrade the OS on this beast? You are now talking about disturbing not 72-125 clients, but 600 of them. If I had a beast like this, I'd run VMWare (or similar) on it, carve it up into virtual machines, and run fewer clients on each just for the sheer flexibility of it. That just moves the goal posts. You'll have the same issue when the hypervisor needs updating (which admittedly may need a reboot less often than the OS). If this is already a virtualized/cloud environment, then I think you're doing it wrong: don't provision one huge instance and use it for multiple clients. Instead, provision lots of small instances and use them for fewer (or even 1) at a time. But it adds the overhead of an OS for each instance. And costs if you have to pay for that OS instance. As always there are trade-offs to be made and the "right" answer will vary based on circumstances and what you are trying to optimize for. I do agree that, generally, more smaller instances will be a closer fit to more use cases but that is only a general answer. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: 500 instances of tomcat on the same server
Eric, On 6/25/21 22:58, Eric Robinson wrote: We can run 75 to 125 instances of tomcat on a single Linux server with 12 cores and 128GB RAM. It works great. CPU is around 25%, our JVMs are not throwing OOMEs, iowait is minimal, and network traffic is about 30Mbps. We're happy with the results. Now we're upping the ante. We have a 48-core server with 1TB RAM, and we're planning to run 600+ tomcat instances on it simultaneously. What caveats or pitfalls should we watch out for? Are there any hard limits that would prevent this from working as expected? If you have the resources, I see no reason why this would present any problems. On the other hand, what happens when you need to upgrade the OS on this beast? You are now talking about disturbing not 72-125 clients, but 600 of them. If I had a beast like this, I'd run VMWare (or similar) on it, carve it up into virtual machines, and run fewer clients on each just for the sheer flexibility of it. If this is already a virtualized/cloud environment, then I think you're doing it wrong: don't provision one huge instance and use it for multiple clients. Instead, provision lots of small instances and use them for fewer (or even 1) at a time. -chris - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: 500 instances of tomcat on the same server
On 26/06/2021 03:58, Eric Robinson wrote: We can run 75 to 125 instances of tomcat on a single Linux server with 12 cores and 128GB RAM. It works great. CPU is around 25%, our JVMs are not throwing OOMEs, iowait is minimal, and network traffic is about 30Mbps. We're happy with the results. Now we're upping the ante. We have a 48-core server with 1TB RAM, and we're planning to run 600+ tomcat instances on it simultaneously. What caveats or pitfalls should we watch out for? Are there any hard limits that would prevent this from working as expected? Nothing comes to mind from a Tomcat perspective. If there was going to be an issue, it would be with the OS or the hardware and you look to have thought all of that through. File limits are the only thing you didn't mention that I'd want to keep an eye on. Changing topic slightly, if there are changes we could make to Tomcat that would it easier to run and manage that many instances do let us know. We'd be happy to consider them. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: 500 instances of tomcat on the same server
> -Original Message- > From: Shawn Heisey > Sent: Saturday, June 26, 2021 8:09 PM > To: users@tomcat.apache.org > Subject: Re: 500 instances of tomcat on the same server > > On 6/25/2021 8:58 PM, Eric Robinson wrote: > > We can run 75 to 125 instances of tomcat on a single Linux server with 12 > cores and 128GB RAM. It works great. CPU is around 25%, our JVMs are not > throwing OOMEs, iowait is minimal, and network traffic is about 30Mbps. > We're happy with the results. > > > > Now we're upping the ante. We have a 48-core server with 1TB RAM, and > we're planning to run 600+ tomcat instances on it simultaneously. What > caveats or pitfalls should we watch out for? Are there any hard limits that > would prevent this from working as expected? > > I'm a lurker here. I have some experience with Tomcat, but most of my > experience is with other Apache projects. > > I'm hoping that my question mirrors what the experienced folks around here > are thinking: > > For something like this ... why are you running so many multiple instances? > Why not run one instance, or a few of them, and have each one handle many > many webapps? I bet you'll find that the overall memory requirements go > way down, because there will be far fewer instances of Java running. > > Maybe you've got good reasons for the architecture you have chosen ... > but it seems like a complete waste of resources to me. > > Thanks, > Shawn Hi Shawn -- We architected it that way many years ago and have always been happy with our early decisions. By running separate tomcats, each from a separate folder and listening on a unique port, we are able to perform maintenance on individual customer's instances, stop and start services, etc., without impacting other customers. We can customize tomcat settings and configurations to customer needs, fine-tune per-customer memory usage, run different versions of tomcat and JDK, create customer-specific security sandboxes, per-customer filesystem permissions, samba shares, etc., and move entire instances around the farm to distribute load as needed. Errors and problems are easier to track down because the tomcat and webapps logs are in separate folders unique to each instance. If Customer A is having an issue with their application, we know that everything associated with their application, including all settings and logs, are in a folder dedicated to them. -Eric Disclaimer : This email and any files transmitted with it are confidential and intended solely for intended recipients. If you are not the named addressee you should not disseminate, distribute, copy or alter this email. Any views or opinions presented in this email are solely those of the author and might not represent those of Physician Select Management. Warning: Although Physician Select Management has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: 500 instances of tomcat on the same server
On 6/25/2021 8:58 PM, Eric Robinson wrote: We can run 75 to 125 instances of tomcat on a single Linux server with 12 cores and 128GB RAM. It works great. CPU is around 25%, our JVMs are not throwing OOMEs, iowait is minimal, and network traffic is about 30Mbps. We're happy with the results. Now we're upping the ante. We have a 48-core server with 1TB RAM, and we're planning to run 600+ tomcat instances on it simultaneously. What caveats or pitfalls should we watch out for? Are there any hard limits that would prevent this from working as expected? I'm a lurker here. I have some experience with Tomcat, but most of my experience is with other Apache projects. I'm hoping that my question mirrors what the experienced folks around here are thinking: For something like this ... why are you running so many multiple instances? Why not run one instance, or a few of them, and have each one handle many many webapps? I bet you'll find that the overall memory requirements go way down, because there will be far fewer instances of Java running. Maybe you've got good reasons for the architecture you have chosen ... but it seems like a complete waste of resources to me. Thanks, Shawn - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
