Re: Classloading Behavior in Embedded Tomcat
Hi,
I guess I found the issue.
WebAppClassLoader delegates to parent classloader where my
environment doesn't have the javax.ws.rs.MessageBodyReader class. But this
was present in my build/compile time environment.
So WebAppClassLoader only ignores the ClassNotFoundException:
if (delegateLoad) {
if (log.isDebugEnabled())
log.debug(" Delegating to parent classloader1 " + parent);
try {
clazz = Class.forName(name, false, parent);
if (clazz != null) {
if (log.isDebugEnabled())
log.debug(" Loading class from parent");
if (resolve)
resolveClass(clazz);
return clazz;
}
} catch (ClassNotFoundException e) {
// Ignore
}
But in this case I get a NoClassDefFoundError and hence the context
fails to start.
I guess I will need to implement a custom class loader. Is there any
other way to make this work?
Thanks,
Chirag
On Thu, Jul 23, 2020 at 11:32 PM Chirag Dewan
wrote:
> Hey Mark,
>
> Any clues, how to proceed with this for embedded?
>
> Thanks
> Chirag
>
>
> On Wed, 22 Jul, 2020, 7:31 pm Chirag Dewan,
> wrote:
>
>> I tried that with standalone Tomcat and it works. I can deploy Jersey-1
>> and Jersey-2 apps together.
>>
>> I was wondering whether Loader with delegate=false work in context for
>> embedded?
>>
>> Thanks
>> Chirag
>>
>> On Wed, 22 Jul, 2020, 5:03 pm Mark Thomas, wrote:
>>
>>> On 22/07/2020 11:35, Chirag Dewan wrote:
>>> > Thanks for a very quick reply Mark.
>>> >
>>> > The Tomcat version is 8.5.46.
>>> >
>>> > Though I do plan to upgrade to 9.0.36 in the coming weeks.
>>>
>>> OK. Both behave the same way which makes things easier.
>>>
>>> See the source code for full details but the summary is:
>>>
>>> First Tomcat checks if the requested class is available from the same
>>> class loader that loaded java.lang.String. If it is, it loads it from
>>> that class loader. This prevents web applications over-ridding Java SE
>>> classes.
>>>
>>> If the class has not been loaded yet, Tomcat then checks to see if it is
>>> a class that Tomcat is expected to provide. If it is, it uses the common
>>> class loader to load it.
>>>
>>> If the class isn't loaded the following search order is used:
>>> - WEB-INF/classes
>>> - WEB-INF/lib
>>> - $CATALINA_BASE/lib (for classes)
>>> - $CATALINA_BASE/lib (for JARs)
>>> - bootstrap / classpath
>>>
>>> Embedded scenarios tend not to set up the same class loader structure as
>>> stand alone Tomcat. Often, they use a single class loader for
>>> everything. I'd check that your scenario works in a stand alone Tomcat
>>> first and once you have that working, transfer it to embedded with
>>> particular attention to class loader configuration.
>>>
>>> Mark
>>>
>>>
>>> >
>>> > Chirag
>>> >
>>> > On Wed, 22 Jul, 2020, 4:03 pm Mark Thomas, wrote:
>>> >
>>> >> On 22/07/2020 11:18, Chirag Dewan wrote:
>>> >>> Hi,
>>> >>>
>>> >>> Due to some backward compatibility concerns, I need to support both
>>> >>> Jersey-1 and Jersey-2 on the same Tomcat instance. This is an
>>> embedded
>>> >>> tomcat which runs inside a JVM application.
>>> >>>
>>> >>> Since, Jersey-1 and Jersey-2 have different JAXRS versions, I tried
>>> to
>>> >>> remove both jsr311 and javax.ws.rs-2 from my JVMs classpath. And
>>> instead
>>> >>> packaged these in the WAR/WEB-INF\lib along with jersey version
>>> specific
>>> >>> jars like jersey-core-1.x, jersey-common-2.x etc.
>>> >>>
>>> >>> Now when I start my Jersey-1 application, it couldn't find
>>> >>> javax.ws.rs.ext.MessageBodyReader.
>>> >>>
>>> >>> I read Classloader HOW-TO and although it says that the order of
>>> loading
>>> >>> JavaEE classes is bootstrap first, it never says anything about
>>> WEB-INF
>>> >> as
>>> >>> a source for these jars.
>>> >>>
>>> >>> So if there any way I can load javax.* classes from WEB-INF\lib?
>>> >>
>>> >> Tomcat version?
>>> >>
>>> >> Different Tomcat versions have taken different approaches to
>>> >> implementing this requirement. A recent(ish) implementation should be
>>> >> fine but with the exact version number we can give a better answer.
>>> >>
>>> >> Mark
>>> >>
>>> >> -
>>> >> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>>> >> For additional commands, e-mail: users-h...@tomcat.apache.org
>>> >>
>>> >>
>>> >
>>>
>>>
>>> -
>>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>>> For additional commands, e-mail: users-h...@tomcat.apache.org
>>>
>>>
Re: Classloading Behavior in Embedded Tomcat
Hey Mark, Any clues, how to proceed with this for embedded? Thanks Chirag On Wed, 22 Jul, 2020, 7:31 pm Chirag Dewan, wrote: > I tried that with standalone Tomcat and it works. I can deploy Jersey-1 > and Jersey-2 apps together. > > I was wondering whether Loader with delegate=false work in context for > embedded? > > Thanks > Chirag > > On Wed, 22 Jul, 2020, 5:03 pm Mark Thomas, wrote: > >> On 22/07/2020 11:35, Chirag Dewan wrote: >> > Thanks for a very quick reply Mark. >> > >> > The Tomcat version is 8.5.46. >> > >> > Though I do plan to upgrade to 9.0.36 in the coming weeks. >> >> OK. Both behave the same way which makes things easier. >> >> See the source code for full details but the summary is: >> >> First Tomcat checks if the requested class is available from the same >> class loader that loaded java.lang.String. If it is, it loads it from >> that class loader. This prevents web applications over-ridding Java SE >> classes. >> >> If the class has not been loaded yet, Tomcat then checks to see if it is >> a class that Tomcat is expected to provide. If it is, it uses the common >> class loader to load it. >> >> If the class isn't loaded the following search order is used: >> - WEB-INF/classes >> - WEB-INF/lib >> - $CATALINA_BASE/lib (for classes) >> - $CATALINA_BASE/lib (for JARs) >> - bootstrap / classpath >> >> Embedded scenarios tend not to set up the same class loader structure as >> stand alone Tomcat. Often, they use a single class loader for >> everything. I'd check that your scenario works in a stand alone Tomcat >> first and once you have that working, transfer it to embedded with >> particular attention to class loader configuration. >> >> Mark >> >> >> > >> > Chirag >> > >> > On Wed, 22 Jul, 2020, 4:03 pm Mark Thomas, wrote: >> > >> >> On 22/07/2020 11:18, Chirag Dewan wrote: >> >>> Hi, >> >>> >> >>> Due to some backward compatibility concerns, I need to support both >> >>> Jersey-1 and Jersey-2 on the same Tomcat instance. This is an embedded >> >>> tomcat which runs inside a JVM application. >> >>> >> >>> Since, Jersey-1 and Jersey-2 have different JAXRS versions, I tried >> to >> >>> remove both jsr311 and javax.ws.rs-2 from my JVMs classpath. And >> instead >> >>> packaged these in the WAR/WEB-INF\lib along with jersey version >> specific >> >>> jars like jersey-core-1.x, jersey-common-2.x etc. >> >>> >> >>> Now when I start my Jersey-1 application, it couldn't find >> >>> javax.ws.rs.ext.MessageBodyReader. >> >>> >> >>> I read Classloader HOW-TO and although it says that the order of >> loading >> >>> JavaEE classes is bootstrap first, it never says anything about >> WEB-INF >> >> as >> >>> a source for these jars. >> >>> >> >>> So if there any way I can load javax.* classes from WEB-INF\lib? >> >> >> >> Tomcat version? >> >> >> >> Different Tomcat versions have taken different approaches to >> >> implementing this requirement. A recent(ish) implementation should be >> >> fine but with the exact version number we can give a better answer. >> >> >> >> Mark >> >> >> >> - >> >> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >> >> For additional commands, e-mail: users-h...@tomcat.apache.org >> >> >> >> >> > >> >> >> - >> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >> For additional commands, e-mail: users-h...@tomcat.apache.org >> >>
Re: Classloading Behavior in Embedded Tomcat
I tried that with standalone Tomcat and it works. I can deploy Jersey-1 and Jersey-2 apps together. I was wondering whether Loader with delegate=false work in context for embedded? Thanks Chirag On Wed, 22 Jul, 2020, 5:03 pm Mark Thomas, wrote: > On 22/07/2020 11:35, Chirag Dewan wrote: > > Thanks for a very quick reply Mark. > > > > The Tomcat version is 8.5.46. > > > > Though I do plan to upgrade to 9.0.36 in the coming weeks. > > OK. Both behave the same way which makes things easier. > > See the source code for full details but the summary is: > > First Tomcat checks if the requested class is available from the same > class loader that loaded java.lang.String. If it is, it loads it from > that class loader. This prevents web applications over-ridding Java SE > classes. > > If the class has not been loaded yet, Tomcat then checks to see if it is > a class that Tomcat is expected to provide. If it is, it uses the common > class loader to load it. > > If the class isn't loaded the following search order is used: > - WEB-INF/classes > - WEB-INF/lib > - $CATALINA_BASE/lib (for classes) > - $CATALINA_BASE/lib (for JARs) > - bootstrap / classpath > > Embedded scenarios tend not to set up the same class loader structure as > stand alone Tomcat. Often, they use a single class loader for > everything. I'd check that your scenario works in a stand alone Tomcat > first and once you have that working, transfer it to embedded with > particular attention to class loader configuration. > > Mark > > > > > > Chirag > > > > On Wed, 22 Jul, 2020, 4:03 pm Mark Thomas, wrote: > > > >> On 22/07/2020 11:18, Chirag Dewan wrote: > >>> Hi, > >>> > >>> Due to some backward compatibility concerns, I need to support both > >>> Jersey-1 and Jersey-2 on the same Tomcat instance. This is an embedded > >>> tomcat which runs inside a JVM application. > >>> > >>> Since, Jersey-1 and Jersey-2 have different JAXRS versions, I tried to > >>> remove both jsr311 and javax.ws.rs-2 from my JVMs classpath. And > instead > >>> packaged these in the WAR/WEB-INF\lib along with jersey version > specific > >>> jars like jersey-core-1.x, jersey-common-2.x etc. > >>> > >>> Now when I start my Jersey-1 application, it couldn't find > >>> javax.ws.rs.ext.MessageBodyReader. > >>> > >>> I read Classloader HOW-TO and although it says that the order of > loading > >>> JavaEE classes is bootstrap first, it never says anything about WEB-INF > >> as > >>> a source for these jars. > >>> > >>> So if there any way I can load javax.* classes from WEB-INF\lib? > >> > >> Tomcat version? > >> > >> Different Tomcat versions have taken different approaches to > >> implementing this requirement. A recent(ish) implementation should be > >> fine but with the exact version number we can give a better answer. > >> > >> Mark > >> > >> - > >> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > >> For additional commands, e-mail: users-h...@tomcat.apache.org > >> > >> > > > > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > >
Re: Classloading Behavior in Embedded Tomcat
Thanks for a very quick reply Mark. The Tomcat version is 8.5.46. Though I do plan to upgrade to 9.0.36 in the coming weeks. Chirag On Wed, 22 Jul, 2020, 4:03 pm Mark Thomas, wrote: > On 22/07/2020 11:18, Chirag Dewan wrote: > > Hi, > > > > Due to some backward compatibility concerns, I need to support both > > Jersey-1 and Jersey-2 on the same Tomcat instance. This is an embedded > > tomcat which runs inside a JVM application. > > > > Since, Jersey-1 and Jersey-2 have different JAXRS versions, I tried to > > remove both jsr311 and javax.ws.rs-2 from my JVMs classpath. And instead > > packaged these in the WAR/WEB-INF\lib along with jersey version specific > > jars like jersey-core-1.x, jersey-common-2.x etc. > > > > Now when I start my Jersey-1 application, it couldn't find > > javax.ws.rs.ext.MessageBodyReader. > > > > I read Classloader HOW-TO and although it says that the order of loading > > JavaEE classes is bootstrap first, it never says anything about WEB-INF > as > > a source for these jars. > > > > So if there any way I can load javax.* classes from WEB-INF\lib? > > Tomcat version? > > Different Tomcat versions have taken different approaches to > implementing this requirement. A recent(ish) implementation should be > fine but with the exact version number we can give a better answer. > > Mark > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > >
Classloading Behavior in Embedded Tomcat
Hi, Due to some backward compatibility concerns, I need to support both Jersey-1 and Jersey-2 on the same Tomcat instance. This is an embedded tomcat which runs inside a JVM application. Since, Jersey-1 and Jersey-2 have different JAXRS versions, I tried to remove both jsr311 and javax.ws.rs-2 from my JVMs classpath. And instead packaged these in the WAR/WEB-INF\lib along with jersey version specific jars like jersey-core-1.x, jersey-common-2.x etc. Now when I start my Jersey-1 application, it couldn't find javax.ws.rs.ext.MessageBodyReader. I read Classloader HOW-TO and although it says that the order of loading JavaEE classes is bootstrap first, it never says anything about WEB-INF as a source for these jars. So if there any way I can load javax.* classes from WEB-INF\lib? Thanks in advance Chirag
Re: Connection Closure due to Fatal Stream with HTTP2
Thanks a lot, Mark. https://ibb.co/LgzFh6t - Memory snapshot after 15minutes of the test. It's certainly better than the graph with 9.0.36, but I will wait for this test to run for another few hours. I will update later. Cheers, Chirag On Fri, Jun 26, 2020 at 6:20 PM Mark Thomas wrote: > On 26/06/2020 12:48, Mark Thomas wrote: > > On 26/06/2020 12:45, Chirag Dewan wrote: > >> Absolutely Mark. Shouldn't take long. > > > > Great. I think I have found a potential root cause. If I am right, NIO > > will show the same issues NIO2 did. > > > > I should have a test build for you shortly. > > Try this: > https://people.apache.org/~markt/dev/v9.0.37-dev/ > > Please note: > > This is NOT an official Apache Release. > > This build is only intended to be used to test this issue. > > Using the test build is it your own risk. > > Thanks, > > Mark > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > >
Re: Connection Closure due to Fatal Stream with HTTP2
Absolutely Mark. Shouldn't take long. On Fri, 26 Jun, 2020, 4:16 pm Mark Thomas, wrote: > Aha! > > h2c could be the significant factor here. Let me take a look. > > Are you in a position to test against a dev build if the need arises? > > Mark > > > On 26/06/2020 11:30, Chirag Dewan wrote: > > Hey Mark, > > > > *Are you using h2c or h2 in your test?* > > We are using h2c > > > > > > *Do you see the same issue if you switch the the NIO connector? Note > > performance differences between NIO and NIO2 are very small.* > > > > I havent tried with NIO honestly. Can quickly execute and check. Will > > update the results. > > > > *How long does a single request take to process?* > > In normal scenarios, less than 3ms. > > > > Thanks, > > Chirag > > > > On Fri, Jun 26, 2020 at 3:26 PM Mark Thomas wrote: > > > >> Hi, > >> > >> Thanks for the additional information. The GC roots were particularly > >> informative. > >> > >> Those RequestInfo objects are associated with HTTP/1.1 requests, not > >> HTTP/2 requests. > >> > >> Some further questions to try and track down what is going on: > >> > >> - Are you using h2c or h2 in your test? > >> > >> - Do you see the same issue if you switch the the NIO connector? Note > >> performance differences between NIO and NIO2 are very small. > >> > >> - How long does a single request take to process? > >> > >> Thanks, > >> > >> Mark > >> > >> On 26/06/2020 09:24, Chirag Dewan wrote: > >>> Thanks Mark. > >>> > >>> *What is the typical response size for one of these requests? * > >>> It' basically a dummy response JSON of ~300bytes. I expect 2300bytes of > >>> response in my production use case, but the purpose here was to isolate > >> as > >>> many things as possible. Hence a dummy response. > >>> > >>> *How long does a typical test take to process? * > >>> I see Tomcat's memory reaching 28G in about 2hours at 19K TPS. The > number > >>> of streams on my client was 500 - which means about 40 connections per > >>> second. > >>> > >>> * What are the GC roots for those RequestInfo objects?* > >>> https://ibb.co/fMRmCXZ > >>> > >>> I hope I was able to answer everything as expected. Thanks. > >>> > >>> On Thu, Jun 25, 2020 at 8:30 PM Mark Thomas wrote: > >>> > >>>> Thanks. > >>>> > >>>> I've looked at the code and I have tried various tests but I am unable > >>>> to re-create a memory leak. > >>>> > >>>> The code used to (before I made a few changes this afternoon) retain a > >>>> lot more memory per Stream and it is possible that what you are seeing > >>>> is a system that doesn't have enough memory to achieve steady state. > >>>> > >>>> If you are able to build the latest 9.0.x and test that, that could be > >>>> helpful. Alternatively, I could provide a test build for you to > >>>> experiment with. > >>>> > >>>> Some additional questions that might aid understanding: > >>>> > >>>> - What is the typical response size for one of these requests? > >>>> - How long does a typical test take to process? > >>>> - What are the GC roots for those RequestInfo objects? > >>>> > >>>> Thanks again, > >>>> > >>>> Mark > >>>> > >>>> > >>>> > >>>> > >>>> On 25/06/2020 15:10, Chirag Dewan wrote: > >>>>> Hi Mark, > >>>>> > >>>>> Its the default APR connector with 150 Threads. > >>>>> > >>>>> Chirag > >>>>> > >>>>> On Thu, 25 Jun, 2020, 7:30 pm Mark Thomas, wrote: > >>>>> > >>>>>> On 25/06/2020 11:00, Chirag Dewan wrote: > >>>>>>> Thanks for the quick check Mark. > >>>>>>> > >>>>>>> These are the images I tried referring to: > >>>>>>> > >>>>>>> https://ibb.co/LzKtRgh > >>>>>>> > >>>>>>> https://ibb.co/2s7hqRL > >>>>>>> > >>>>>
Re: Connection Closure due to Fatal Stream with HTTP2
Hey Mark, *Are you using h2c or h2 in your test?* We are using h2c *Do you see the same issue if you switch the the NIO connector? Note performance differences between NIO and NIO2 are very small.* I havent tried with NIO honestly. Can quickly execute and check. Will update the results. *How long does a single request take to process?* In normal scenarios, less than 3ms. Thanks, Chirag On Fri, Jun 26, 2020 at 3:26 PM Mark Thomas wrote: > Hi, > > Thanks for the additional information. The GC roots were particularly > informative. > > Those RequestInfo objects are associated with HTTP/1.1 requests, not > HTTP/2 requests. > > Some further questions to try and track down what is going on: > > - Are you using h2c or h2 in your test? > > - Do you see the same issue if you switch the the NIO connector? Note > performance differences between NIO and NIO2 are very small. > > - How long does a single request take to process? > > Thanks, > > Mark > > On 26/06/2020 09:24, Chirag Dewan wrote: > > Thanks Mark. > > > > *What is the typical response size for one of these requests? * > > It' basically a dummy response JSON of ~300bytes. I expect 2300bytes of > > response in my production use case, but the purpose here was to isolate > as > > many things as possible. Hence a dummy response. > > > > *How long does a typical test take to process? * > > I see Tomcat's memory reaching 28G in about 2hours at 19K TPS. The number > > of streams on my client was 500 - which means about 40 connections per > > second. > > > > * What are the GC roots for those RequestInfo objects?* > > https://ibb.co/fMRmCXZ > > > > I hope I was able to answer everything as expected. Thanks. > > > > On Thu, Jun 25, 2020 at 8:30 PM Mark Thomas wrote: > > > >> Thanks. > >> > >> I've looked at the code and I have tried various tests but I am unable > >> to re-create a memory leak. > >> > >> The code used to (before I made a few changes this afternoon) retain a > >> lot more memory per Stream and it is possible that what you are seeing > >> is a system that doesn't have enough memory to achieve steady state. > >> > >> If you are able to build the latest 9.0.x and test that, that could be > >> helpful. Alternatively, I could provide a test build for you to > >> experiment with. > >> > >> Some additional questions that might aid understanding: > >> > >> - What is the typical response size for one of these requests? > >> - How long does a typical test take to process? > >> - What are the GC roots for those RequestInfo objects? > >> > >> Thanks again, > >> > >> Mark > >> > >> > >> > >> > >> On 25/06/2020 15:10, Chirag Dewan wrote: > >>> Hi Mark, > >>> > >>> Its the default APR connector with 150 Threads. > >>> > >>> Chirag > >>> > >>> On Thu, 25 Jun, 2020, 7:30 pm Mark Thomas, wrote: > >>> > >>>> On 25/06/2020 11:00, Chirag Dewan wrote: > >>>>> Thanks for the quick check Mark. > >>>>> > >>>>> These are the images I tried referring to: > >>>>> > >>>>> https://ibb.co/LzKtRgh > >>>>> > >>>>> https://ibb.co/2s7hqRL > >>>>> > >>>>> https://ibb.co/KmKj590 > >>>>> > >>>>> > >>>>> The last one is the MAT screenshot showing many RequestInfo objects. > >>>> > >>>> Thanks. That certainly looks like a memory leak. I'll take a closer > >>>> look. Out of interest, how many threads is the Connector configured to > >> use? > >>>> > >>>> Mark > >>>> > >>>> > >>>>> > >>>>> > >>>>> Thanks, > >>>>> > >>>>> Chirag > >>>>> > >>>>> On Wed, Jun 24, 2020 at 8:30 PM Mark Thomas > wrote: > >>>>> > >>>>>> On 24/06/2020 12:17, Mark Thomas wrote: > >>>>>>> On 22/06/2020 11:06, Chirag Dewan wrote: > >>>>>>>> Hi, > >>>>>>>> > >>>>>>>> Update: We found that Tomcat goes OOM when a client closes and > opens > >>>> new > >>>>>>>> connections every second. In the memory dump, we see a l
Re: Connection Closure due to Fatal Stream with HTTP2
Thanks Mark. *What is the typical response size for one of these requests? * It' basically a dummy response JSON of ~300bytes. I expect 2300bytes of response in my production use case, but the purpose here was to isolate as many things as possible. Hence a dummy response. *How long does a typical test take to process? * I see Tomcat's memory reaching 28G in about 2hours at 19K TPS. The number of streams on my client was 500 - which means about 40 connections per second. * What are the GC roots for those RequestInfo objects?* https://ibb.co/fMRmCXZ I hope I was able to answer everything as expected. Thanks. On Thu, Jun 25, 2020 at 8:30 PM Mark Thomas wrote: > Thanks. > > I've looked at the code and I have tried various tests but I am unable > to re-create a memory leak. > > The code used to (before I made a few changes this afternoon) retain a > lot more memory per Stream and it is possible that what you are seeing > is a system that doesn't have enough memory to achieve steady state. > > If you are able to build the latest 9.0.x and test that, that could be > helpful. Alternatively, I could provide a test build for you to > experiment with. > > Some additional questions that might aid understanding: > > - What is the typical response size for one of these requests? > - How long does a typical test take to process? > - What are the GC roots for those RequestInfo objects? > > Thanks again, > > Mark > > > > > On 25/06/2020 15:10, Chirag Dewan wrote: > > Hi Mark, > > > > Its the default APR connector with 150 Threads. > > > > Chirag > > > > On Thu, 25 Jun, 2020, 7:30 pm Mark Thomas, wrote: > > > >> On 25/06/2020 11:00, Chirag Dewan wrote: > >>> Thanks for the quick check Mark. > >>> > >>> These are the images I tried referring to: > >>> > >>> https://ibb.co/LzKtRgh > >>> > >>> https://ibb.co/2s7hqRL > >>> > >>> https://ibb.co/KmKj590 > >>> > >>> > >>> The last one is the MAT screenshot showing many RequestInfo objects. > >> > >> Thanks. That certainly looks like a memory leak. I'll take a closer > >> look. Out of interest, how many threads is the Connector configured to > use? > >> > >> Mark > >> > >> > >>> > >>> > >>> Thanks, > >>> > >>> Chirag > >>> > >>> On Wed, Jun 24, 2020 at 8:30 PM Mark Thomas wrote: > >>> > >>>> On 24/06/2020 12:17, Mark Thomas wrote: > >>>>> On 22/06/2020 11:06, Chirag Dewan wrote: > >>>>>> Hi, > >>>>>> > >>>>>> Update: We found that Tomcat goes OOM when a client closes and opens > >> new > >>>>>> connections every second. In the memory dump, we see a lot of > >>>>>> RequestInfo objects that are causing the memory spike. > >>>>>> > >>>>>> After a while, Tomcat goes OOM and start rejecting request(I get a > >>>>>> request timed out on my client). This seems like a bug to me. > >>>>>> > >>>>>> For better understanding, let me explain my use case again: > >>>>>> > >>>>>> I have a jetty client that sends HTTP2 requests to Tomcat. My > >>>>>> requirement is to close a connection after a configurable(say 5000) > >>>>>> number of requests/streams and open a new connection that continues > to > >>>>>> send requests. I close a connection by sending a GoAway frame. > >>>>>> > >>>>>> When I execute this use case under load, I see that after ~2hours my > >>>>>> requests fail and I get a series of errors like request > >>>>>> timeouts(5seconds), invalid window update frame, and connection > close > >>>>>> exception on my client. > >>>>>> On further debugging, I found that it's a Tomcat memory problem and > it > >>>>>> goes OOM after sometime under heavy load with multiple connections > >> being > >>>>>> re-established by the clients. > >>>>>> > >>>>>> image.png > >>>>>> > >>>>>> image.png > >>>>>> > >>>>>> Is this a known issue? Or a known behavior with Tomcat? > >>>>> > >>>>> Embedded images get dropped by the list software. Po
Re: Connection Closure due to Fatal Stream with HTTP2
Hi Mark, Its the default APR connector with 150 Threads. Chirag On Thu, 25 Jun, 2020, 7:30 pm Mark Thomas, wrote: > On 25/06/2020 11:00, Chirag Dewan wrote: > > Thanks for the quick check Mark. > > > > These are the images I tried referring to: > > > > https://ibb.co/LzKtRgh > > > > https://ibb.co/2s7hqRL > > > > https://ibb.co/KmKj590 > > > > > > The last one is the MAT screenshot showing many RequestInfo objects. > > Thanks. That certainly looks like a memory leak. I'll take a closer > look. Out of interest, how many threads is the Connector configured to use? > > Mark > > > > > > > > Thanks, > > > > Chirag > > > > On Wed, Jun 24, 2020 at 8:30 PM Mark Thomas wrote: > > > >> On 24/06/2020 12:17, Mark Thomas wrote: > >>> On 22/06/2020 11:06, Chirag Dewan wrote: > >>>> Hi, > >>>> > >>>> Update: We found that Tomcat goes OOM when a client closes and opens > new > >>>> connections every second. In the memory dump, we see a lot of > >>>> RequestInfo objects that are causing the memory spike. > >>>> > >>>> After a while, Tomcat goes OOM and start rejecting request(I get a > >>>> request timed out on my client). This seems like a bug to me. > >>>> > >>>> For better understanding, let me explain my use case again: > >>>> > >>>> I have a jetty client that sends HTTP2 requests to Tomcat. My > >>>> requirement is to close a connection after a configurable(say 5000) > >>>> number of requests/streams and open a new connection that continues to > >>>> send requests. I close a connection by sending a GoAway frame. > >>>> > >>>> When I execute this use case under load, I see that after ~2hours my > >>>> requests fail and I get a series of errors like request > >>>> timeouts(5seconds), invalid window update frame, and connection close > >>>> exception on my client. > >>>> On further debugging, I found that it's a Tomcat memory problem and it > >>>> goes OOM after sometime under heavy load with multiple connections > being > >>>> re-established by the clients. > >>>> > >>>> image.png > >>>> > >>>> image.png > >>>> > >>>> Is this a known issue? Or a known behavior with Tomcat? > >>> > >>> Embedded images get dropped by the list software. Post those images > >>> somewhere we can see them. > >>> > >>>> Please let me know if you any experience with such a situation. Thanks > >>>> in advance. > >>> > >>> Nothing comes to mind. > >>> > >>> I'll try some simple tests with HTTP/2. > >> > >> I don't see a memory leak (the memory is reclaimed eventually) but I do > >> see possibilities to release memory associated with request processing > >> sooner. > >> > >> Right now you need to allocate more memory to the Java process to enable > >> Tomcat to handle the HTTP/2 load it is presented with. > >> > >> It looks like a reasonable chunk of memory is released when the > >> Connection closes that could be released earlier when the associated > >> Stream closes. I'll take a look at what can be done in that area. In the > >> meantime, reducing the number of Streams you allow on a Connection > >> before it is closed should reduce overall memory usage. > >> > >> Mark > >> > >> - > >> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > >> For additional commands, e-mail: users-h...@tomcat.apache.org > >> > >> > > > > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > >
Re: Connection Closure due to Fatal Stream with HTTP2
Thanks for the quick check Mark. These are the images I tried referring to: https://ibb.co/LzKtRgh https://ibb.co/2s7hqRL https://ibb.co/KmKj590 The last one is the MAT screenshot showing many RequestInfo objects. Thanks, Chirag On Wed, Jun 24, 2020 at 8:30 PM Mark Thomas wrote: > On 24/06/2020 12:17, Mark Thomas wrote: > > On 22/06/2020 11:06, Chirag Dewan wrote: > >> Hi, > >> > >> Update: We found that Tomcat goes OOM when a client closes and opens new > >> connections every second. In the memory dump, we see a lot of > >> RequestInfo objects that are causing the memory spike. > >> > >> After a while, Tomcat goes OOM and start rejecting request(I get a > >> request timed out on my client). This seems like a bug to me. > >> > >> For better understanding, let me explain my use case again: > >> > >> I have a jetty client that sends HTTP2 requests to Tomcat. My > >> requirement is to close a connection after a configurable(say 5000) > >> number of requests/streams and open a new connection that continues to > >> send requests. I close a connection by sending a GoAway frame. > >> > >> When I execute this use case under load, I see that after ~2hours my > >> requests fail and I get a series of errors like request > >> timeouts(5seconds), invalid window update frame, and connection close > >> exception on my client. > >> On further debugging, I found that it's a Tomcat memory problem and it > >> goes OOM after sometime under heavy load with multiple connections being > >> re-established by the clients. > >> > >> image.png > >> > >> image.png > >> > >> Is this a known issue? Or a known behavior with Tomcat? > > > > Embedded images get dropped by the list software. Post those images > > somewhere we can see them. > > > >> Please let me know if you any experience with such a situation. Thanks > >> in advance. > > > > Nothing comes to mind. > > > > I'll try some simple tests with HTTP/2. > > I don't see a memory leak (the memory is reclaimed eventually) but I do > see possibilities to release memory associated with request processing > sooner. > > Right now you need to allocate more memory to the Java process to enable > Tomcat to handle the HTTP/2 load it is presented with. > > It looks like a reasonable chunk of memory is released when the > Connection closes that could be released earlier when the associated > Stream closes. I'll take a look at what can be done in that area. In the > meantime, reducing the number of Streams you allow on a Connection > before it is closed should reduce overall memory usage. > > Mark > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > >
Re: Connection Closure due to Fatal Stream with HTTP2
Hi,
Update: We found that Tomcat goes OOM when a client closes and opens new
connections every second. In the memory dump, we see a lot of RequestInfo
objects that are causing the memory spike.
After a while, Tomcat goes OOM and start rejecting request(I get a
request timed out on my client). This seems like a bug to me.
For better understanding, let me explain my use case again:
I have a jetty client that sends HTTP2 requests to Tomcat. My requirement
is to close a connection after a configurable(say 5000) number of
requests/streams and open a new connection that continues to send requests.
I close a connection by sending a GoAway frame.
When I execute this use case under load, I see that after ~2hours my
requests fail and I get a series of errors like request timeouts(5seconds),
invalid window update frame, and connection close exception on my client.
On further debugging, I found that it's a Tomcat memory problem and it goes
OOM after sometime under heavy load with multiple connections being
re-established by the clients.
[image: image.png]
[image: image.png]
Is this a known issue? Or a known behavior with Tomcat?
Please let me know if you any experience with such a situation. Thanks in
advance.
On Sun, Jun 14, 2020 at 11:30 AM Chirag Dewan
wrote:
> Hi,
>
> This is without load balancer actually. I am directly sending to Tomcat.
>
> Update:
>
> A part issue I found was to be 9.0.29. I observed that when request were
> timed out on client (2seconds), the client would send a RST frame. And the
> GoAway from Tomcat was perhaps a bug. In 9.0.36, RST frame is replied with
> a RST from Tomcat.
>
> Now the next part to troubleshoot is why after about an hour or so,
> requests are timed out at Tomcat.
>
> Could close to 100 HTTP2 connections per second cause this on Tomcat?
>
> Thanks
>
> On Sun, 14 Jun, 2020, 12:27 AM Michael Osipov,
> wrote:
>
>> Am 2020-06-13 um 08:42 schrieb Chirag Dewan:
>> > Hi,
>> >
>> > We are observing that under high load, my clients start receiving a
>> GoAway
>> > frame with error:
>> >
>> > *Connection[{id}], Stream[{id}] an error occurred during processing that
>> > was fatal to the connection.*
>> >
>> > Background : We have implemented our clients to close connections after
>> > every 500-1000 requests (streams). This is a load balancer requirement
>> that
>> > we are working on and hence such a behavior. So with a throughput of
>> around
>> > 19k, almost 40 connections are closed and recreated every second.
>> >
>> > After we receive this frame, my clients start behaving erroneously.
>> Before
>> > this as well, my clients start sending RST_STREAM with canceled for each
>> > request. Could this be due to the number of connections we open? Is it
>> > related to the version of Tomcat? Or maybe my clients are misbehaving?
>> >
>> > Now since I only receive this under heavy load, I can't quite picture
>> > enough reasons for this to happen.
>> >
>> > Any possible clues on where I should start looking?
>> >
>> > My Stack:
>> > Server - Tomcat 9.0.29
>> > Client - Jetty 9.x
>>
>> Does the same happen w/o the load balancer?
>>
>> -
>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>> For additional commands, e-mail: users-h...@tomcat.apache.org
>>
>>
Re: Connection Closure due to Fatal Stream with HTTP2
Hi,
This is without load balancer actually. I am directly sending to Tomcat.
Update:
A part issue I found was to be 9.0.29. I observed that when request were
timed out on client (2seconds), the client would send a RST frame. And the
GoAway from Tomcat was perhaps a bug. In 9.0.36, RST frame is replied with
a RST from Tomcat.
Now the next part to troubleshoot is why after about an hour or so,
requests are timed out at Tomcat.
Could close to 100 HTTP2 connections per second cause this on Tomcat?
Thanks
On Sun, 14 Jun, 2020, 12:27 AM Michael Osipov, wrote:
> Am 2020-06-13 um 08:42 schrieb Chirag Dewan:
> > Hi,
> >
> > We are observing that under high load, my clients start receiving a
> GoAway
> > frame with error:
> >
> > *Connection[{id}], Stream[{id}] an error occurred during processing that
> > was fatal to the connection.*
> >
> > Background : We have implemented our clients to close connections after
> > every 500-1000 requests (streams). This is a load balancer requirement
> that
> > we are working on and hence such a behavior. So with a throughput of
> around
> > 19k, almost 40 connections are closed and recreated every second.
> >
> > After we receive this frame, my clients start behaving erroneously.
> Before
> > this as well, my clients start sending RST_STREAM with canceled for each
> > request. Could this be due to the number of connections we open? Is it
> > related to the version of Tomcat? Or maybe my clients are misbehaving?
> >
> > Now since I only receive this under heavy load, I can't quite picture
> > enough reasons for this to happen.
> >
> > Any possible clues on where I should start looking?
> >
> > My Stack:
> > Server - Tomcat 9.0.29
> > Client - Jetty 9.x
>
> Does the same happen w/o the load balancer?
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>
Connection Closure due to Fatal Stream with HTTP2
Hi,
We are observing that under high load, my clients start receiving a GoAway
frame with error:
*Connection[{id}], Stream[{id}] an error occurred during processing that
was fatal to the connection.*
Background : We have implemented our clients to close connections after
every 500-1000 requests (streams). This is a load balancer requirement that
we are working on and hence such a behavior. So with a throughput of around
19k, almost 40 connections are closed and recreated every second.
After we receive this frame, my clients start behaving erroneously. Before
this as well, my clients start sending RST_STREAM with canceled for each
request. Could this be due to the number of connections we open? Is it
related to the version of Tomcat? Or maybe my clients are misbehaving?
Now since I only receive this under heavy load, I can't quite picture
enough reasons for this to happen.
Any possible clues on where I should start looking?
My Stack:
Server - Tomcat 9.0.29
Client - Jetty 9.x
Thanks in advance
Re: Performance Comparison HTTP1 vs HTTP2 in Tomcat 9.0.29
Thanks for the quick response Mark. I agree 1024 concurrent streams are a bit far fetched and may cause an overhead. But at the same time, I have tried the same test with the Jetty Multiplexed connection pool with 100 concurrent streams(that is actually updated from the initial Settings frame). And even in such kind of a connection strategy, we see ~6K around throughput. And to my surprise, even with 20 established connections, we could not reach the throughput in an HTTP1.1 connector. Are there any benchmarking results for HTTP2, in comparison to HTTP1.1 I can refer? Chirag On Fri, May 22, 2020 at 4:29 PM Mark Thomas wrote: > On 22/05/2020 11:23, Chirag Dewan wrote: > > Hi, > > > > I am trying to move to HTTP2 based APR connector from my HTTP1 based > > connector because of some customer requirements. > > > > I am trying to form some sort of throughput benchmark for HTTP2 in > > comparison to HTTP1. I have a simple Jersey service that accepts a JSON > > request and sends 200 with some headers. > > > > I have observed that HTTP2 is somehow stuck at 6K as compared to 15K in > > HTTP1.1 with the same amount of CPU and memory consumed. > > My client with HTTP1.1 is based on HTTP components and opens up to 100 > > connections with Tomcat. On HTTP2, I have a Jetty client that opens 2 > > connections with multiplexing of 1024. I tried increasing the > > connections to 20 as well, but that only has adverse affects. > > > > I am running Tomcat on a K8 pod with 3Gi CPU and 2Gi memory. With both > > HTTP2 and HTTP1.1 Tomcat consumes all 3 cores and approximately 800m > memory. > > > > In the thread dumps with HTTP2, I see a lot of BLOCKED threads: > > image.png > > Most of the threads are blocked in /writeHeaders. / > > / > > / > > Am I missing something here? Any help is much appreciated. > > With such a simple response and high concurrency I suspect you are > hitting a bottleneck with 1024 (or 100 if you haven't changed the > defaults) threads all trying to write to a single network connection at > once. That is never going to perform well. > > HTTP/2 is not a magic "make things faster" protocol. It reduces overhead > in some areas and increases overhead in others. Whether you see a > benefit is going to depend on where the bottleneck is in your system. > > If you are testing on a single machine or on a local network I'd expect > the additional complexity of HTTP/2 multiplexiing to quickly dominate > the results. > > If you want an idea of what is going on, I recommend using a profiler > although be aware that - unless there is an obvious performance issue - > you can quickly get to the point where getting the level of detail > required to track down the next bottleneck causes the profiler to create > more overhead than the issue you are trying to measure thereby > distorting the results. > > Mark > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > >
Performance Comparison HTTP1 vs HTTP2 in Tomcat 9.0.29
Hi, I am trying to move to HTTP2 based APR connector from my HTTP1 based connector because of some customer requirements. I am trying to form some sort of throughput benchmark for HTTP2 in comparison to HTTP1. I have a simple Jersey service that accepts a JSON request and sends 200 with some headers. I have observed that HTTP2 is somehow stuck at 6K as compared to 15K in HTTP1.1 with the same amount of CPU and memory consumed. My client with HTTP1.1 is based on HTTP components and opens up to 100 connections with Tomcat. On HTTP2, I have a Jetty client that opens 2 connections with multiplexing of 1024. I tried increasing the connections to 20 as well, but that only has adverse affects. I am running Tomcat on a K8 pod with 3Gi CPU and 2Gi memory. With both HTTP2 and HTTP1.1 Tomcat consumes all 3 cores and approximately 800m memory. In the thread dumps with HTTP2, I see a lot of BLOCKED threads: [image: image.png] Most of the threads are blocked in *writeHeaders. * Am I missing something here? Any help is much appreciated. Thank you Chirag
Re: HTTP2 Pseudo Headers in HttpServletRequest
Hi Mark, Thanks for the quick update. That would work. Thank you, Chirag On Wed, 15 Apr, 2020, 1:00 PM Mark Thomas, wrote: > On 15/04/2020 06:32, Chirag Dewan wrote: > > Hi, > > > > I have a Jersey application deployed on an Embedded Tomcat 9.0.29. I have > > enabled the APR connector with HTTP2 Upgrade Protocol to support HTTP2 > > requests. > > > > Now when I try to get :authority and :path header from the > > HttpServletRequest, I get null. > > > > I debugged this and saw that the coyoteRequest - > org.apache.coyote.Request > > has only HTTP1.1 headers and skips HTTP2 headers. > > > > Although I know that I can probably use coyoteRequest to get the URL, > path > > and other pseudo header details, but is there any other way to get the > > pseudo headers like other HTTP1.1 headers? > > :method > No HTTP/1.1 equivalent header (it is part of the request line) > Use HttpServletRequest.getMethod() for HTTP/1.1 and HTTP/2. > > :scheme > No HTTP/1.1 equivalent header (it is part of the request line) > Use ServletRequest.getScheme() for HTTP/1.1 and HTTP/2. > > :path > No HTTP/1.1 equivalent header (it is part of the request line) > Use HttpServletRequest.getRequestURI() for HTTP/1.1 and HTTP/2. > > :authority > Equivalent to the HTTP/1.1 Host header > Use ServletRequest.getServerName() and ServletRequest.getServerPort() > for HTTP/1.1 and HTTP/2. > > Mark > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > >
HTTP2 Pseudo Headers in HttpServletRequest
Hi, I have a Jersey application deployed on an Embedded Tomcat 9.0.29. I have enabled the APR connector with HTTP2 Upgrade Protocol to support HTTP2 requests. Now when I try to get :authority and :path header from the HttpServletRequest, I get null. I debugged this and saw that the coyoteRequest - org.apache.coyote.Request has only HTTP1.1 headers and skips HTTP2 headers. Although I know that I can probably use coyoteRequest to get the URL, path and other pseudo header details, but is there any other way to get the pseudo headers like other HTTP1.1 headers? Thanks, Chirag
Re: Issue while using SSL with Embedded Tomcat 6.0.37
Hi All, Thanks a lot for your assistance. I enabled these protocols but that dint help. Finally after 24 hours of digging in I figured out the problem. We found out that the clients connection timeout was close to around 80mins(which is inexplicable) and there is a firewall in between the client and the server. After 60sec(server timeout) server sent a FIN to the client, but there is no ACK from the other side and server connection closes. So if the next request comes within 80mins it tries to use the connection which is in TIME_WAIT state,and thus fails since SSL handshaking is closed by the server. And second request onwards handshaking happens again. Appreciate all of you for your help. Chirag. Sent from Yahoo! Mail on Android
Re: Issue while using SSL with Embedded Tomcat 6.0.37
Hi , A small update. The customers client is C++ client,which uses OpenSSL. And I found that client hello message is SSLv2 protocol. And the server response(server hello) is a TLSv1 protocol. Is there something I am missing? Chirag On Wednesday, 9 October 2013 9:25 PM, Chirag Dewan wrote: Chris, This is a legacy code and do need some tweaks for sure. Regarding the issue,for some other Cipher as well the handshaking is failing. I get a TCP_ZERO_WINDOW in my snoops. And thus resulting in Server sending a RST to client. Chirag Sent from Yahoo! Mail on Android
Re: Issue while using SSL with Embedded Tomcat 6.0.37
Chris, This is a legacy code and do need some tweaks for sure. Regarding the issue,for some other Cipher as well the handshaking is failing. I get a TCP_ZERO_WINDOW in my snoops. And thus resulting in Server sending a RST to client. Chirag Sent from Yahoo! Mail on Android
Re: Issue while using SSL with Embedded Tomcat 6.0.37
Hi,
I have a wrapper connector class :
public HTTPConnector(int port, String keystoreFile, String password, int
maxKeepAliveRequests, int maxThreads, int connectionTimeout, Logger logger)
throws Exception
{
myLogger = logger;
this.keyStoreFile = keystoreFile;
this.keyStorePassword = password;
if( maxKeepAliveRequests != 0 && maxKeepAliveRequests >= -1 )
{
IntrospectionUtils.setProperty( this, "maxKeepAliveRequests",
String.valueOf( maxKeepAliveRequests ) );
}
if( maxThreads > 0 )
{
IntrospectionUtils.setProperty( this, "maxThreads", String.valueOf(
maxThreads ) );
}
InetAddress address = null;
try
{
if( keystoreFile != null )
{
String cipherSet = System.getProperty("https.cipher.set");
setSecure( true );
if( myLogger.isLoggable( Level.FINER ) ) myLogger.finer(
"EmbeddedTomcat using HTTPS and cipher sets " + cipherSet );
setScheme( "https" );
try
{
// Added TLS since there is a bug in Tomcat 5.5.9. No
default protocol is set.
IntrospectionUtils.setProperty( this, "sslProtocol", "TLS"
);
IntrospectionUtils.setProperty( this, "keystore",
keyStoreFile );
IntrospectionUtils.setProperty( this, "keypass",
keyStorePassword );
IntrospectionUtils.setProperty( this, "SSLEnabled", "true"
);
if(cipherSet != null && !cipherSet.equalsIgnoreCase("")){
IntrospectionUtils.setProperty( this, "ciphers",
cipherSet );
}
}
catch( Exception exception )
{
myLogger.severe( "Could not load SSL server socket
factory." );
throw new Exception( "Could not load SSL server socket
factory." );
}
}
else
{
setSecure( false );
}
address = InetAddress.getLocalHost();
if( address != null )
{
IntrospectionUtils.setProperty( this, "address", "" + address );
}
IntrospectionUtils.setProperty( this, "port", "" + port );
IntrospectionUtils.setProperty( this, "connectionTimeout",
String.valueOf((connectionTimeout * 1000)) );
}
catch( Exception exception )
{
myLogger.severe( "Exception occurred while making HTTP Connector. "
);
throw new Exception( "Exception occurred while making HTTP
Connector. " );
}
try
{
setEnableLookups( false );
}
catch( Exception exception )
{
myLogger.severe( "Exception occurred while enabling lookups. " );
throw new Exception( "Exception occurred while enabling lookups. "
);
}
}
and I attach it to the container by :
Embedded embedded = new Embedded();
embedded.addConnector( connector );
connector.start();
and I call embedded.start(); during intialization,so I have the Tomcat running.
Chris,"cipherSet" is a configurable parameter. I am usually using
TLS_DHE_RSA_WITH_AES_128_CBC_SHA for testing purpose.
Thanks!
Chirag
On Wednesday, 9 October 2013 7:17 PM, Christopher Schultz
wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
Chirag,
On 10/9/13 8:39 AM, Chirag Dewan wrote:
> The first request after the Server is started gets rejected.
Interesting.
> I am setting my connector as follows:
>
> IntrospectionUtils.setProperty( this, "sslProtocol", "TLS" );
> IntrospectionUtils.setProperty( this, "keystore", keyStoreFile );
> IntrospectionUtils.setProperty( this, "keypass", keyStorePassword
> ); IntrospectionUtils.setProperty( this, "SSLEnabled", "true" );
> IntrospectionUtils.setProperty( this, "ciphers", cipherSet );
>
> This is my connector configuration. I am now setting cipher,as you
> can see. And it is selecting the specified cipher,so that way I
> can limit the cipher sets to be selected by Server.
What is the value of "cipherSet"?
Perhaps you could share some /more/ code... the above for instance
doesn't show how you initialize the connector, attach it to the
container, etc.
- -chris
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.14 (Darwin)
Comment: GPGTools - http://gpgtools.o
Re: Issue while using SSL with Embedded Tomcat 6.0.37
Hi, The first request after the Server is started gets rejected. I am setting my connector as follows: IntrospectionUtils.setProperty( this, "sslProtocol", "TLS" ); IntrospectionUtils.setProperty( this, "keystore", keyStoreFile ); IntrospectionUtils.setProperty( this, "keypass", keyStorePassword ); IntrospectionUtils.setProperty( this, "SSLEnabled", "true" ); IntrospectionUtils.setProperty( this, "ciphers", cipherSet ); This is my connector configuration. I am now setting cipher,as you can see. And it is selecting the specified cipher,so that way I can limit the cipher sets to be selected by Server. On Wednesday, 9 October 2013 5:45 PM, Christopher Schultz wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Chirag, On 10/8/13 9:48 PM, Chirag Dewan wrote: > For this particular cipher, the server sends a RST to the client > after the certificate exchange is done. And the handshaking > fails,for the first time only. Second request onwards handshaking > happens and the traffic flows as usual. > > What I understand is,I can provide a set of ciphers to the > connector and the client will select from that particular set and > can thus avoid the particular cipher. Technically speaking, the server selects the cipher given the list sent by the client and any restrictions the server has (e.g. using the "ciphers" list from your connector). It's very strange that the server would be selecting that (evidently missing) cipher in the first place (then failing) and then choosing it /again/ and succeeding. I must be missing something. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.14 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBCAAGBQJSVUi4AAoJEBzwKT+lPKRYvn4P/AlheFMNQ/6b+tydwX6/YzlQ gbI20pnDMZV2+03lQB/3ACLNmWWkS4NbjW/6FeYfaN8yEZFMP0EwvnTkVye9VMaG RlGpgNjOsANruNrtmDT/frAIGjLQxQM+f+4Bjc+TGBSNAnjeoDnintDhEAcXHtpp NaL2H2IJMtXJrGBhfZ3z0xu1gYLY6+SCuSeilahl6uvMS1PU0s5KT0Nm3D8xIkeC L48yU+y0kVV9Ok2gojNPxCxKt6EJP+/WrR37q+H7LFCsfESKG8yYK72tBU0ex6Z2 mUinqnYcbLXTGrtWzuFJUGMUyJWpLeuZJii5McBtwqudOnbFWo8hbXPKj2IfEMMT FuSBJgLyVXAoZ51SrRYCOjYMy3tH2kB4FgHc43GWlxwznrHVgTeOUe5OQRiFpfa5 BDKCtEQF/xHUp9zE2BKpvieR1lCxUc+8zgJvThvFElGkvdenK0kgE1RgsgEf9cTv cgyr+PM6T8Yp99ngj69kfqvTt8tNfg8UUZVzHmUPj4zMEII9uKRElk26g31yiRf7 2FXguR1ANn2cbn6mDQvrE48LjfQ6Zupr6XWUXSBffenI3yyJdeueRc0BvrY4LUYH Yn2OVjd0NnITmqfDje03cDxtosGJTESUc+LtBK4eJUYftkSgGMxZmKE0tM+QucVM k0EQUBpo4cU/QqFZ6lyU =3uiN -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Issue while using SSL with Embedded Tomcat 6.0.37
Hi Chris, Thanks for the code,it helped a lot. Now,using that code on my server machine I found out that TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA is not even in the defaults ciphers for jdk1.6.0_39. Isn't this a strange behaviour? Server can only select available ciphers,I suppose. Thanks Chirag On Tuesday, 8 October 2013 9:10 PM, Christopher Schultz wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Chirag, On 10/8/13 6:01 AM, Chirag Dewan wrote: > I am using Embedded Tomcat 6.0.37. I have a servlet which is > running over HTTPS using SSL Connector. I have a Java Keystore with > Customer Certificate imported in it. > > Now,there is a HTTP Client on the customers end which connects > with the servlet over HTTPS(I have very little information about > the customers client configuration) > > The problem we are facing is: > > For the first request from Client,the SSL handshaking fails. How, specifically? What do you observe on the server? What do you observe on the client? > From second request, handshaking is completed successfully and the > requests are processed. I have observed when Server selects > TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA as the cipher suite, only then > the Server sends a reset to the client and the handshaking fails. > On second request, with same cipher suite,it works fine. So the first request and second request seem to both negotiate the same cipher suite (TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA) but the first one fails? > Can anyone assist me in understanding why it is failing for the > first time? And is there any way I can force the Tomcat not to > select this cipher suite? Or any other way that I can resolve this > issue. See the Connector documentation, specifically the "cipherSuites" attribute. Unfortunately, Tomcat's cipherSuites configuration is only explicit... you can't say something like "defaults without TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA". See this thread in the archives for a short program that will dump-out the available cipher suites and indicates which ones are available by default in your environment (note that the results will change for every different version of Java you use): http://markmail.org/message/zn4namfhypyxum23 - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.14 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBCAAGBQJSVCdnAAoJEBzwKT+lPKRYM9cP/3GztDeXBYguwJ/Q+YBnNSea NzEQuJXFmaSJhhhCP4NMrHz0Fq4zZlKu9khxicK4gwcfGuCZ0i2BkNx1jZh9wgOF aedSeZinpXtF5L+EiWUCm9Xz2yPHuX40+VIaW9X4/TfG+DMcDVvFFAH1onjHQ5im KECrmK5ratXmVfm9o37SrXItoqNFLqk70mxcZlVec40fp7nu3Bn2ReMIKcSCSXcb Sr97cHlRD8yMBqTn42RNTSzfFfJ/5TFNzmwXzlrSJcWO+6mpKYmXXdbJc3voNd3W e+ZWmJQheJEVm6n86z2PMqwJyBtaiNFRxOxbeXHtU1BwemhSAP1EVPtZSUKQ5k+4 vHbZ4CfhuSgM6IaoTZjqqZkvch4POTLUWPArFJeEyOS8p9vayNoVhFectMtutR4O zHxanjckpCgJYp5w82jRaZ4Xs9SojTedHn6gSElxZK94fg9H4dL6g43h+zSpnuJC 0KF4U47FMklZJBikjDXbkcH3YY8Bd+e+5JMl2Uu+TyjG12Cj6wxyOKM4ubAF7pMO IZbs9WEgHx2Oj515RgFNQGF8uXLysLo4uBiCbTEvFQ3T/eGrSzvYi6kLKi/izPuc TbSYcS1UEAiRKABPMRbUKDqmD6IOTOjbR66lamwTzNFvsyH+BhoaB1RVHy9TUC2U YicDQSfyb9kfCnANiGwR =pYDx -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Issue while using SSL with Embedded Tomcat 6.0.37
Hi, For this particular cipher, the server sends a RST to the client after the certificate exchange is done. And the handshaking fails,for the first time only. Second request onwards handshaking happens and the traffic flows as usual. What I understand is,I can provide a set of ciphers to the connector and the client will select from that particular set and can thus avoid the particular cipher. Can I do this in embedded tomcat? And what set of ciphers should I allow with that connector? Thanks! Chirag Sent from Yahoo! Mail on Android
Issue while using SSL with Embedded Tomcat 6.0.37
Hi All, I am using Embedded Tomcat 6.0.37. I have a servlet which is running over HTTPS using SSL Connector. I have a Java Keystore with Customer Certificate imported in it. Now,there is a HTTP Client on the customers end which connects with the servlet over HTTPS(I have very little information about the customers client configuration) The problem we are facing is: For the first request from Client,the SSL handshaking fails. From second request,handshaking is completed successfully and the requests are processed. I have observed when Server selects TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA as the cypher suite,only then the Server sends a reset to the client and the handshaking fails. On second request,with same cypher suite,it works fine. Can anyone assist me in understanding why it is failing for the first time? And is there any way I can force the Tomcat not to select this cypher suite? Or any other way that I can resolve this issue. Thanks a lot. Chirag Dewan
Re: OOME issue in Tomcat 6.0.18(with SSL)
Hi Chris,
A little more digging in and I found out that only with SSL,tomcat is creating
a large number of sessions. I can see in the logs for HTTP:
INFO: SessionListener: sessionDestroyed('2E8DE01EE3F0D166FEFC8A45353CD9ED')
Now,in case of HTTPS I see a large number of such logs. So I believe for HTTPS
requests it is creating a session for every request as compared to HTTP?
This is my SSL setting in client:
public HttpTLSProtocolSocketFactory()
{
try
{
this.sslcontext = SSLContext.getInstance("TLS");
TrustManager[] arrayOfTrustManager = { new DummyX509TrustManager() };
this.sslcontext.init(null, arrayOfTrustManager, null);
}
catch (Exception localException)
{
this.logger.log(Level.SEVERE, "Failed to created SSLContext: " +
localException.getMessage());
}
}
public Socket createSocket(String paramString, int paramInt1, InetAddress
paramInetAddress, int paramInt2, HttpConnectionParams paramHttpConnectionParams)
throws IOException, UnknownHostException, ConnectTimeoutException
{
if (paramHttpConnectionParams == null)
throw new IllegalArgumentException("Parameters may not be null");
int i = paramHttpConnectionParams.getConnectionTimeout();
SSLSocketFactory localSSLSocketFactory = this.sslcontext.getSocketFactory();
if (i == 0)
return localSSLSocketFactory.createSocket(paramString, paramInt1,
paramInetAddress, paramInt2);
Socket localSocket = localSSLSocketFactory.createSocket();
InetSocketAddress localInetSocketAddress1 = new
InetSocketAddress(paramInetAddress, paramInt2);
InetSocketAddress localInetSocketAddress2 = new
InetSocketAddress(paramString, paramInt1);
localSocket.bind(localInetSocketAddress1);
localSocket.connect(localInetSocketAddress2, i);
return localSocket;
}
Can different sockets create different sessions?
So for HTTP I am not creating any new sessions,same should be the case for
HTTPS. Right?
The only thing which differs in my is the SSLSocketFactory Implementation,which
creates sockets.
Chirag.
From: Christopher Schultz
To: Tomcat Users List
Sent: Wednesday, 12 June 2013 8:14 PM
Subject: Re: OOME issue in Tomcat 6.0.18(with SSL)
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
Chirag,
On 6/12/13 1:01 AM, Chirag Dewan wrote:
> I am facing an Out of Memory Issue with my application. I am using
> Embedded Tomcat 6.0.18. I have a simple servlet deployed which
> does nothing but set the HTTPResponse and return it.
Are you sure you don't trigger session creation?
> Now I have 2 testing clients
>
> Client 1:
>
> HTTPClient 3.1 with MultiThreadedHttpConnectionManager with SSL
> maxConnections = 200 maxConnectionsPerHost=200
>
> Client 2:
>
> HTTPClient 4.1 in which I configured multiple threads
>
> for (int i = 0; i < noofConnections; i++) { Thread t = new
> Thread(my); t.start(); }
What is noofConnections set to?
> With every thread executing HTTPGet on the servlet. This too with
> SSL.
>
>
> Now with client 1,my JVM crashes after just a few minutes. Running
> with 8gb heap space.
>
>
> I took a heap dump for 2gb heap space using Jmap and analyzed it
> with MAT. It seems that there were many instances of
> org.apache.catalina.session.StandardManager consuming almost 95% of
> heap space.
There should only be one StandardManager present per web application
deployed. You only have a single web application? Are you sure that
you have many StandardManager instances, or do you have a single
StandardManager with a huge number of sessions?
> CPU utilization is only 13 -14% and I can see only 65 threads
> active with netstat.
>
>
> With Client 2,everything looks fine. Though JVM crashes at 2gb,but
> at 8gb it works fine. Though CPU utilization is almost 90% and 100
> simultaneous threads created for 100 simultaneous threads.
>
> Both my server and clients are on Linux 64 bit machines.
>
> I believe that this is something related to the client 1 i.e.
> either the HTTPClient 3.1 or the MultiThreadedConnectionManager,but
> posting here if someone can assist me in what might be the root
> cause.
Try a single thread and a single connection. Take a packet-capture and
see what the response headers look like. I wouldn't be surprised if
you are generating a new session with each and every request.
- -chris
-BEGIN PGP SIGNATURE-
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJRuIlpAAoJEBzwKT+lPKRYjo4P/2S+89ycBzPTnxonur/aTUxm
0ZIHWHsDbKlg7YRlMlvpcPGuyXniW9mDOw5wWajt4WSCmcWNDEfI3bIIAI139lkw
EeEFIVplkK/ZY2uLViJ0B4V9A/kBB7qkGDNPPubeNnhpi6gVGl+j3jB1S9aHOT9t
dmLGavG1j1pRtAUTItr1kVKAegyaYCSlwbDe8YQefRF8+N/DyAi3wutSdoOpGYMA
WytWWW
Re: OOME issue in Tomcat 6.0.18(with SSL)
Hi Chuck,
Tried the same on 6.0.37 its the same issue.
I am using JVM 1.6.0.39 and both my client and server are on separate linux
x86 machines.
The issue is with one of my client as it seems. HttpClient 3.1.
A code snippet from both my clients:
3.1
MultiThreadedHttpConnectionManager myConnectionManager = new
MultiThreadedHttpConnectionManager();
HttpClient myClient = new HttpClient(myConnectionManager);
myConnectionManager.getParams().setMaxConnectionsPerHost(200);
myConnectionManager.getParams().setMaxTotalConnections(200);
For SSL :
this.sslcontext = SSLContext.getInstance("TLS");
TrustManager[] arrayOfTrustManager = { new DummyX509TrustManager() };
this.sslcontext.init(null, arrayOfTrustManager, null);
SSLSocketFactory localSSLSocketFactory = this.sslcontext.getSocketFactory();
And for 4.1:
PoolingClientConnectionManager pm = new
PoolingClientConnectionManager();
pm.setDefaultMaxPerRoute(200);
pm.setMaxTotal(200);
HttpClient httpclient = new DefaultHttpClient(pm);
try {
SSLSocketFactory socketFactory = new
SSLSocketFactory(trustStore);
Scheme sch = new Scheme("https", port, socketFactory);
pm.getSchemeRegistry().register(sch);
Chirag
From: "Caldarale, Charles R"
To: Tomcat Users List
Sent: Wednesday, 12 June 2013 11:18 AM
Subject: RE: OOME issue in Tomcat 6.0.18(with SSL)
> From: Chirag Dewan [mailto:chirag.dewa...@yahoo.in]
> Subject: OOME issue in Tomcat 6.0.18(with SSL)
> I am using Embedded Tomcat 6.0.18.
Which is nearly five years old. Many, many fixes (including serious
security-related ones) have gone in since that version was released; you should
see if the problem still exists on the current version of Tomcat 6 or 7.
What JVM version are you using, and what platform are you running on? (Be
precise.)
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
MATERIAL and is thus for use only by the intended recipient. If you received
this in error, please contact the sender and delete the e-mail and its
attachments from all computers.
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
OOME issue in Tomcat 6.0.18(with SSL)
Hi All,
I am facing an Out of Memory Issue with my application. I am using Embedded
Tomcat 6.0.18. I have a simple servlet deployed which does nothing but set the
HTTPResponse and return it.
Now I have 2 testing clients
Client 1:
HTTPClient 3.1 with MultiThreadedHttpConnectionManager with SSL
maxConnections = 200
maxConnectionsPerHost=200
Client 2:
HTTPClient 4.1 in which I configured multiple threads
for (int i = 0; i < noofConnections; i++) {
Thread t = new Thread(my);
t.start();
}
With every thread executing HTTPGet on the servlet. This too with SSL.
Now with client 1,my JVM crashes after just a few minutes. Running with 8gb
heap space.
I took a heap dump for 2gb heap space using Jmap and analyzed it with MAT. It
seems that there were many instances of
org.apache.catalina.session.StandardManager consuming almost 95% of heap space.
CPU utilization is only 13 -14% and I can see only 65 threads active with
netstat.
With Client 2,everything looks fine. Though JVM crashes at 2gb,but at 8gb it
works fine. Though CPU utilization is almost 90% and 100 simultaneous threads
created for 100 simultaneous threads.
Both my server and clients are on Linux 64 bit machines.
I believe that this is something related to the client 1 i.e. either the
HTTPClient 3.1 or the MultiThreadedConnectionManager,but posting here if
someone can assist me in what might be the root cause.
Chirag
Re: Performance Issue while upgrading from Embedded Tomcat 6 to Tomcat 7
Chris, Sorry I should have posted the data first. I probably missed the most important part of a load test. I will do it shortly. And I am not using Jmeter now,I am using an http client for load test. I am testing it on Solaris x86 server 64bit JVM. And i have collected the samples for Tomcat 6.0.18,0.37,7.0.30 and 6.0.40. I do not want to use ab because of factor that my benchmark in tomcat 6 were established on a custom http client. I do not have the exact data with me right now. Will surely post it asap. Thanks. Sent from Yahoo! Mail on Android
Re: Performance Issue while upgrading from Embedded Tomcat 6 to Tomcat 7
Chris, The profiler shows very high CPU utilization in Tomcat threads. As I said before a lot of time was spent during the ResponseFacade.setContentType() method call. That doesn't tell the whole story but more or less the high utilization is mainly in the tomcat threads. The thing which puzzles me the most is I am able to scale the req/sec with multiple clients on Tomcat 6 and the same doesn't happen on Tomcat 7? Can it be the JVM? Would it help trying with a latter version of JVM? Thanks. From: Christopher Schultz To: Tomcat Users List Sent: Friday, 24 May 2013 3:24 AM Subject: Re: Performance Issue while upgrading from Embedded Tomcat 6 to Tomcat 7 -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Chirag, On 5/23/13 1:21 PM, Chirag Dewan wrote: > With c=1 it is 70k req/sec and with c=2 it is 35k in both i.e the > total req/sec cannot be scaled beyond 70k. With Tomcat 6 it is 60k > in both clients i.e total of 120k. > > I do not expect more than 75k req/sec being served by Tomcat 7,but > its the benchmark set by Tomcat 6 which I can't overlook. As I said > I wouldn't have mind 80-90 % utilization on peak load,but its the > already running server performance which I am compelled to follow > or even carry out more. What do you see when you attach a profiler to Tomcat while you run your load tests against it? (Now that you have removed your own webapp from the mix and are just fetching trivial resources). - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBCAAGBQJRnpAFAAoJEBzwKT+lPKRY91AQAJicVT7nLNBzr4m7eqOs6sRn R+hCNT37A0HWS8+A8Ns52FyCUUmOljrFVP8LcwA1J0IXFxxI1GvCGNdL7y8248dF cWKyyMYUpmiGbwRBcz0XTvxJoGLwo55Wv7pnYsdecU7d93epXcPSZTQHM3zI82ZB TEShV/AHigcnA9mEe8NV1XgLzCZclh9mjfP2uIhhBHvNkoK3gX0oMey8xK9lezlJ ArGKy1RUNsvnWaDvmFC5o5FQyh8CPDnxXalGgDKnjxFHQfUOtCXaFdWdgeiPL6xn TPiuWIEdqNXERLAjY9IjXzoXpBkxRcYPEwN4V3/tHZapw6C7M9OmcmuXMfhF5UEP G4e+nhcy2CtdUum38j4g1LLEB06QRmc1DIt6czb3kqKQR3FTBpRSkBdWEjJhZnxU JVRIYiK8rOqxXygJcga0VlI5xSIzHh2JK7qlT6rc+zqWQTHXOFkmxyGMfjb+hx+i Mofp0lrO9aczFopb/C3uF7gOAb5c0hhhf1Smxvy9nCl2YscA+BarhWdwiOeEGd+6 mNTK/unQIZLJI3Sqb8xgaaGoa5FEvfoaRptweiqON76hNZGrFDxCnQeG9h6W12uW CGQfEWnbgmWkao/ZrIGYkxmRQNd7n4w2jEfzxdD40yrDX+jyQ7VbWrKkX/m/0K+b 7h8osL0Vaqk+GtaWfuPA =cRWt -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Performance Issue while upgrading from Embedded Tomcat 6 to Tomcat 7
Chris, With c=1 it is 70k req/sec and with c=2 it is 35k in both i.e the total req/sec cannot be scaled beyond 70k. With Tomcat 6 it is 60k in both clients i.e total of 120k. I do not expect more than 75k req/sec being served by Tomcat 7,but its the benchmark set by Tomcat 6 which I can't overlook. As I said I wouldn't have mind 80-90 % utilization on peak load,but its the already running server performance which I am compelled to follow or even carry out more. Thanks. Sent from Yahoo! Mail on Android
Re: Performance Issue while upgrading from Embedded Tomcat 6 to Tomcat 7
Hi All, I tested Tomcat 7.0.40 with Solaris x86. It gave better CPU performance than Tomcat 7.0.30 . It was able to handle 70K requests at 45% CPU utilization which is still behind my bench mark of 70k requests at 25-30% utilization with Tomcat 6. I also tested with Tomcat 6.0.37 and the results were more or less same as Tomcat 6.0.18 One thing I observed,I ran 3 clients pointing to 3 different static HTML pages. With Tomcat 6(both 6.0.18 and 6.0.37) each client gave me a concurrency of 48k req/sec at 50-55% CPU utilization. However when I ran the same test case for Tomcat 7(7.0.40 and 7.0.30) the req/sec were divided b/w the clients and the total concurrency was consistent around 70k with 45% utilization. Now no matter how many clients I run,the req/sec is constant. This is a bit strange. What could possibly be the reason? I am not sure I would be able to upgrade to Tomcat 7 under these circumstances. Thanks. From: Christopher Schultz To: Tomcat Users List Sent: Wednesday, 22 May 2013 7:30 PM Subject: Re: Performance Issue while upgrading from Embedded Tomcat 6 to Tomcat 7 -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Chirag, On 5/21/13 11:03 PM, Chirag Dewan wrote: > I was monitoring the CPU utilization specifically. I can > compromise on 1 less transactions/sec,but 80-90% utilization is > not good. While it's nice to reduce resource utilization as much as possible, why would you want your server to run at 25% CPU all the time? That's a waste of 75% of the CPU available. The test you are performing sounds like a peak-load test (100 simultaneous connections is pretty decent load). At peak load, you have 10-20% breathing room before your CPU can't keep up. Sounds like you have an appropriately-sized server for the type of load you expect. Do you have some kind of performance benchmarks that you have to hit? > I have configured 1000 max threads in the connector,and I am > testing with 100 concurrent threads. So you have 10 times the number of threads available than necessary. Did you do that simply to make sure that the thread-pool wouldn't be the problem? What kind of connector are you using? Re-reading the thread it looks like you are using the BIO (default) connector. I'm not sure switching to NIO or APR would help raw throughput unless you are using SSL (switch to APR) or have lots of dropped keepalive connections (switch to NIO - or APR?). > Well I tested a sample html page with Tomcat 6.0.18 and 7.0.30 on > Solaris x86 server. The req/sec were almost the same for both,but > CPU utilization for Tomcat 6 was 23-27% and for Tomcat 7 it was > 80-90% consistently. What about Tomcat 6.0.37? > On the other hand on a Linux system,the req/sec were a little > higher in Tomcat 7 say 65k to 55k on tomcat 6 and the CPU > utilization was the same for both 6 and 7. ... and what was that CPU utilization? > My Jprofiler stack trace on Solaris is a lot different. As far as > I have observed,for Tomcat 7 the stack Trace leads me to > ResponseFacade.setContentType,which was not the behaviour in > Tomcat 6. Can that be a bottleneck? See Mark's message: there have been some changes to how Tomcat performs a setContentType -- because the string actually needs to be parsed to see if there is a character encoding hiding in there, and the output writer may need it's encoding changed. You can see in the changelog (http://tomcat.apache.org/tomcat-7.0-doc/changelog.html) this entry under 7.0.33, which I think is the relevant part: "The HTTP header parser added to address 52811 has been removed..." Take a look at the references provided if you are interested in the details. Otherwise, just upgrade to 7.0.40 and try again. > Or is there something platform dependent that I should take care > of? There could be a bug in one of the JVMs, but this is fairly well-exercised code that you are talking about. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBCAAGBQJRnM+NAAoJEBzwKT+lPKRYDo4P/2LGsApJTBj39lPKdshAYXba DwDWIllFM+fn/XRLsx2fTinBLPupWoGbGrPoVOA7xTTqrJVddJCfwbbmiS2RvcB5 kPMNn+baAWMN3f4z/5fVkXwVhuyfRNKMFNzL64N3hjuqhzMqQGBvoaUbKMq27bzt VzQiLiEC5mUJ4VsnUj+h4yWCFPv2JT7Q4tifneAX3F3ouOI6KqJ7y30LX2qz4xPp 4f/KDgQwMTxrsdDe2R3+dmBFZgo2sHuZZUggGCvTADmjAaPgikAWAO5Yov2tPVhf ZRLrn6s6VAQpRDPN6jNQbogWWw/zakDf8zPrTSjr/BQRrhF7+eRyr3/jWlA9UOkd rs/WIXm6URZjhbzdO7h0qkPOziSw9cekUt/cVUigd9N+Mgyp0YcDw8wtg6Hx0PJC Ig/nvuOB3ePxP01EPc0hXg3fw6GlwTSS0H5GRox0n5cvR83lDtGrzCvqZe+py6z4 JaKtA7FhcYcZrqZiiGMNVYsMcBL5CEr4gQztCbQscfltnTjCIQoM7BclQ+VMHDQI SfLEsMfNBix6ap88H335RAyGUyMbz1kWMtBv7+GvzEoBQ6GDfk39iDu2rLfFLiMi calZtW9asdFFQwpe87hdadPToOgFuyyAIm7EZpSWF69kh5nM6DXSEBGOhjCrc9OB QGR+cvRCWYnodpC4YZhm =NxRr -END PGP SIGNATURE- -
Re: Performance Issue while upgrading from Embedded Tomcat 6 to Tomcat 7
Chris, I am setting previously running Tomcat 6 performance data as a benchmark. And I am straight forward comparing the results with Tomcat 7 to the previous results. I expect a similar performance,if not better. Actually,the max no. of connections in my case are configurable from a GUI with a maximum of 1000 that a user can set. Hence 1000 max threads would be my peak load case. Well for now I have completely dropped my application out of the scope. I am testing and comparing Tomcat 6 and 7 on the same servers with sample HTML pages. Next in line would be Tomcat 6.0.37 and 7.0.40. I would have to upgrade to minor version of Tomcat 6 otherwise. Thanks. From: Christopher Schultz To: Tomcat Users List Sent: Wednesday, 22 May 2013 7:30 PM Subject: Re: Performance Issue while upgrading from Embedded Tomcat 6 to Tomcat 7 -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Chirag, On 5/21/13 11:03 PM, Chirag Dewan wrote: > I was monitoring the CPU utilization specifically. I can > compromise on 1 less transactions/sec,but 80-90% utilization is > not good. While it's nice to reduce resource utilization as much as possible, why would you want your server to run at 25% CPU all the time? That's a waste of 75% of the CPU available. The test you are performing sounds like a peak-load test (100 simultaneous connections is pretty decent load). At peak load, you have 10-20% breathing room before your CPU can't keep up. Sounds like you have an appropriately-sized server for the type of load you expect. Do you have some kind of performance benchmarks that you have to hit? > I have configured 1000 max threads in the connector,and I am > testing with 100 concurrent threads. So you have 10 times the number of threads available than necessary. Did you do that simply to make sure that the thread-pool wouldn't be the problem? What kind of connector are you using? Re-reading the thread it looks like you are using the BIO (default) connector. I'm not sure switching to NIO or APR would help raw throughput unless you are using SSL (switch to APR) or have lots of dropped keepalive connections (switch to NIO - or APR?). > Well I tested a sample html page with Tomcat 6.0.18 and 7.0.30 on > Solaris x86 server. The req/sec were almost the same for both,but > CPU utilization for Tomcat 6 was 23-27% and for Tomcat 7 it was > 80-90% consistently. What about Tomcat 6.0.37? > On the other hand on a Linux system,the req/sec were a little > higher in Tomcat 7 say 65k to 55k on tomcat 6 and the CPU > utilization was the same for both 6 and 7. ... and what was that CPU utilization? > My Jprofiler stack trace on Solaris is a lot different. As far as > I have observed,for Tomcat 7 the stack Trace leads me to > ResponseFacade.setContentType,which was not the behaviour in > Tomcat 6. Can that be a bottleneck? See Mark's message: there have been some changes to how Tomcat performs a setContentType -- because the string actually needs to be parsed to see if there is a character encoding hiding in there, and the output writer may need it's encoding changed. You can see in the changelog (http://tomcat.apache.org/tomcat-7.0-doc/changelog.html) this entry under 7.0.33, which I think is the relevant part: "The HTTP header parser added to address 52811 has been removed..." Take a look at the references provided if you are interested in the details. Otherwise, just upgrade to 7.0.40 and try again. > Or is there something platform dependent that I should take care > of? There could be a bug in one of the JVMs, but this is fairly well-exercised code that you are talking about. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBCAAGBQJRnM+NAAoJEBzwKT+lPKRYDo4P/2LGsApJTBj39lPKdshAYXba DwDWIllFM+fn/XRLsx2fTinBLPupWoGbGrPoVOA7xTTqrJVddJCfwbbmiS2RvcB5 kPMNn+baAWMN3f4z/5fVkXwVhuyfRNKMFNzL64N3hjuqhzMqQGBvoaUbKMq27bzt VzQiLiEC5mUJ4VsnUj+h4yWCFPv2JT7Q4tifneAX3F3ouOI6KqJ7y30LX2qz4xPp 4f/KDgQwMTxrsdDe2R3+dmBFZgo2sHuZZUggGCvTADmjAaPgikAWAO5Yov2tPVhf ZRLrn6s6VAQpRDPN6jNQbogWWw/zakDf8zPrTSjr/BQRrhF7+eRyr3/jWlA9UOkd rs/WIXm6URZjhbzdO7h0qkPOziSw9cekUt/cVUigd9N+Mgyp0YcDw8wtg6Hx0PJC Ig/nvuOB3ePxP01EPc0hXg3fw6GlwTSS0H5GRox0n5cvR83lDtGrzCvqZe+py6z4 JaKtA7FhcYcZrqZiiGMNVYsMcBL5CEr4gQztCbQscfltnTjCIQoM7BclQ+VMHDQI SfLEsMfNBix6ap88H335RAyGUyMbz1kWMtBv7+GvzEoBQ6GDfk39iDu2rLfFLiMi calZtW9asdFFQwpe87hdadPToOgFuyyAIm7EZpSWF69kh5nM6DXSEBGOhjCrc9OB QGR+cvRCWYnodpC4YZhm =NxRr -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: RE: Performance Issue while upgrading from Embedded Tomcat 6 to Tomcat 7
Yes. With same JVM on both Solaris and Linux i.e Java 1.6.39. It is 64 bit version. Thanks. Sent from Yahoo! Mail on Android
Re: Performance Issue while upgrading from Embedded Tomcat 6 to Tomcat 7
Hi Chris, I was monitoring the CPU utilization specifically. I can compromise on 1 less transactions/sec,but 80-90% utilization is not good. I tested the Async Servlet and the Hello World Servlet with the same environment and the client. Hello world servlet gave 20k req/sec and on the other hand Async Servlet was only 100 req/sec even less during the course. Both the results with 100 threads. I have configured 1000 max threads in the connector,and I am testing with 100 concurrent threads. Well I tested a sample html page with Tomcat 6.0.18 and 7.0.30 on Solaris x86 server. The req/sec were almost the same for both,but CPU utilization for Tomcat 6 was 23-27% and for Tomcat 7 it was 80-90% consistently. On the other hand on a Linux system,the req/sec were a little higher in Tomcat 7 say 65k to 55k on tomcat 6 and the CPU utilization was the same for both 6 and 7. My Jprofiler stack trace on Solaris is a lot different. As far as I have observed,for Tomcat 7 the stack Trace leads me to ResponseFacade.setContentType,which was not the behaviour in Tomcat 6. Can that be a bottleneck? Or is there something platform dependent that I should take care of? Thanks. From: Christopher Schultz To: Tomcat Users List Sent: Tuesday, 21 May 2013 9:08 PM Subject: Re: Performance Issue while upgrading from Embedded Tomcat 6 to Tomcat 7 -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Chirag, On 5/20/13 10:38 AM, Chirag Dewan wrote: > I ran my test client on Hello World example servlet on Tomcat > 7.0.30. It was 12K req/sec with 80% CPU utilization. > > The same test case on tomcat 6.0.18 gave me similar req/sec but > CPU utilization of 70% . The results on my linux server,with client > and server running on the same machine. But all I am doing is > comparing the 2 instances in the same environement. Running both client and server on the same machine is not a great test if you want to determine the server load by looking at CPU utilization. Were you looking at overall CPU utilization, or just utilization of the Tomcat JVM process? > I am pretty sure the utilization is due to the server. > > The Jprofiler shows a lot of logging threads in tomcat consuming a > lot of memory. Something I should be careful about? Can you be more specific? AFAIK, there are no "logging threads" in Tomcat at all. > Also I ran an asynchronous servlet with the same client. The > trans/sec reduced drastically,but the CPU utilization was only 4% I would expect a small reduction in trans/sec when using async, but nothing I would describe as "drastic". Can you give us some numbers? > Are there too may threads waiting for connection? Would thread > pooling help, if we can? I think you mean "too many connections waiting for a thread" instead of the other way around. Threads are already pooled. What are your configuration details, and what are the parameters for your load test? If you only have 10 connections configured, launching 10,000 clients isn't exactly a fair test. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBCAAGBQJRm5UNAAoJEBzwKT+lPKRYKoEQAJeWY07c4jr1prAqvdrOyR4X X9CaWrZ4DwvsKuSn2NtZgXqf5lcLhsDzgP3Ej6cNhBx7WcK6zy8zpTZ2RdVo+YOB 1fGfdpZ6rMODT3i2UI8mAQmbG54NwA4C40ShsTpYVtdXb4GfAX+81j8HqtClr5Tf NEBq2xu4LJDc0ajE4mh+Vyr3M3wUG5CEsQAKhj3Owddtus6sltbrBs8116ZzcpQt 10COnt/e08ikyUVuAYxGZPvQ+rwhlFWuvTf0NlG9/oZSDxSb19KcdZ+vMi2HTNQr McQpVxd2RtqpTgM2NJ3uTS6rQ7R4I1moP96DOgNrU6jS2u5sJVoH98weq4AU4uxa 5sXJG4N4eaVsaFE9m2Nio3zgXZDObmF2jh1df79d7TzHkcIJHjtbZ9VHf1gLVqoW v2pWVU+7di6MiDg7P1Y8y2pXwYYF13ZVx6uozyfv6hGuGXDZkMmmg0QatpKguVXl sO3idan5klic3RJJg2zduq6PUlJZyqJJulqJas1DoUHARo6pSDkZWy1hYZQC9BGT dHgmxrA4Jo5Npq1VpiNMCXl/wCT2vKdsseqsCEQN4EIvgJWuZ5l8NkZtceQTPXtE 4CEcsSd83hIDmichkhVw3PoNHKSSFGwdRRI5sEXqW1Vv88Vj7ldtD8518vJUoQLu s2xIFlQjkJKW/W46Y+on =2P5W -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Performance Issue while upgrading from Embedded Tomcat 6 to Tomcat 7
It might be weird but its confusing me a lot. I used HttpClient 4.2.1 for Tomcat 6.0.18 and I used the same client with Tomcat 7.0.30,causing the CPU utilization and reduced trans/sec. Now,the same thing I have tested with Apache Jmeter 2.6,and its working fine. Any possible reasons? Thanks. From: Chirag Dewan To: Tomcat Users List Sent: Monday, 20 May 2013 8:08 PM Subject: Re: Performance Issue while upgrading from Embedded Tomcat 6 to Tomcat 7 I ran my test client on Hello World example servlet on Tomcat 7.0.30. It was 12K req/sec with 80% CPU utilization. The same test case on tomcat 6.0.18 gave me similar req/sec but CPU utilization of 70% . The results on my linux server,with client and server running on the same machine. But all I am doing is comparing the 2 instances in the same environement. I am pretty sure the utilization is due to the server. The Jprofiler shows a lot of logging threads in tomcat consuming a lot of memory. Something I should be careful about? Also I ran an asynchronous servlet with the same client. The trans/sec reduced drastically,but the CPU utilization was only 4% Are there too may threads waiting for connection? Would thread pooling help,if we can? Thanks. From: Mark Thomas To: Tomcat Users List Sent: Monday, 20 May 2013 1:48 PM Subject: Re: Performance Issue while upgrading from Embedded Tomcat 6 to Tomcat 7 On 20/05/2013 06:59, Chirag Dewan wrote: > Hi, > > I have profiled the application using JProfiler,and it seems to me > that its my servlet which is taking the majority of time. > > > Though the time is in mili seconds,but I guess since the servlet is > code is same as with Tomcat 6.0.18 is it the Servlet 3.0 API which is > causing the reduced throughput? If the time is being spent in your servlet then it can't possibly be the Servlet 3.0 implementation causing the delay since that would be time spent inside Tomcat code, not your code. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Performance Issue while upgrading from Embedded Tomcat 6 to Tomcat 7
I ran my test client on Hello World example servlet on Tomcat 7.0.30. It was 12K req/sec with 80% CPU utilization. The same test case on tomcat 6.0.18 gave me similar req/sec but CPU utilization of 70% . The results on my linux server,with client and server running on the same machine. But all I am doing is comparing the 2 instances in the same environement. I am pretty sure the utilization is due to the server. The Jprofiler shows a lot of logging threads in tomcat consuming a lot of memory. Something I should be careful about? Also I ran an asynchronous servlet with the same client. The trans/sec reduced drastically,but the CPU utilization was only 4% Are there too may threads waiting for connection? Would thread pooling help,if we can? Thanks. From: Mark Thomas To: Tomcat Users List Sent: Monday, 20 May 2013 1:48 PM Subject: Re: Performance Issue while upgrading from Embedded Tomcat 6 to Tomcat 7 On 20/05/2013 06:59, Chirag Dewan wrote: > Hi, > > I have profiled the application using JProfiler,and it seems to me > that its my servlet which is taking the majority of time. > > > Though the time is in mili seconds,but I guess since the servlet is > code is same as with Tomcat 6.0.18 is it the Servlet 3.0 API which is > causing the reduced throughput? If the time is being spent in your servlet then it can't possibly be the Servlet 3.0 implementation causing the delay since that would be time spent inside Tomcat code, not your code. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Performance Issue while upgrading from Embedded Tomcat 6 to Tomcat 7
Hi, I have profiled the application using JProfiler,and it seems to me that its my servlet which is taking the majority of time. Though the time is in mili seconds,but I guess since the servlet is code is same as with Tomcat 6.0.18 is it the Servlet 3.0 API which is causing the reduced throughput? Thanks. From: Chirag Dewan To: "users@tomcat.apache.org" ; "ma...@apache.org" Sent: Friday, 17 May 2013 6:30 PM Subject: Re: Performance Issue while upgrading from Embedded Tomcat 6 to Tomcat 7 I am running it on Solaris x86 server. I haven't even changed my code much. Let me add a profiler,will post the results. Thanks. Sent from Yahoo! Mail on Android
Re: Performance Issue while upgrading from Embedded Tomcat 6 to Tomcat 7
I am running it on Solaris x86 server. I haven't even changed my code much. Let me add a profiler,will post the results. Thanks. Sent from Yahoo! Mail on Android
Re: Performance Issue while upgrading from Embedded Tomcat 6 to Tomcat 7
Another important factor is the CPU utilization. Earlier for same trans/sec it was 40% now its close to 80%. And yes,the OS,JVM and memory setting are unchanged. Thanks. Sent from Yahoo! Mail on Android
Re: Performance Issue while upgrading from Embedded Tomcat 6 to Tomcat 7
I am comparing tomcat 6.0.18 with tomcat 7.0.30. Would adding NIO or APR connector help? Currently I am using default(BIO) connector. Thanks. Sent from Yahoo! Mail on Android
Performance Issue while upgrading from Embedded Tomcat 6 to Tomcat 7
Hi All, I have upgraded my application from Embedded Tomcat 6 to Embedded Tomcat 7. With Tomcat 6, I was getting around 7 Transactions per sec with a simple HTTP service,which sets the HTTPResponse and returns it. Now with Tomcat 7,I am getting 54000 transactions per sec. Going through the web I figured out that it may be due to Servlet 3.0 API. I have updated my web.xml so that Servlet API doesn't scan the classes for annotations. This is how my web.xml looks like: http://java.sun.com/xml/ns/javaee"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"; version="3.0"> Can anyone provide some other measures I can implement? I am using an HTTP Client to test my performnace. Thanks
Unable to deploy web service on Embedded Tomcat 7.0.30
Hi, Another issue I am facing with Embedding tomcat 7. I am deploying a webservice with the war structure like: > classes > wsdl > One.wsdl >web.xml My code looks like this : Context context = Tomcat.addContext( "/service", "somepath/one.war" ); ( (StandardContext) context ).setUnpackWAR( false ); context.setReloadable( false ); There is no exception,but I cannot access my service after deployment. Is there something wrong I am doing? Also I need to disable the httponly flag. How can I do that in Embedded Tomcat? Thanks. From: Mark Thomas To: Tomcat Users List Sent: Thursday, 16 May 2013 3:04 PM Subject: Re: Embedded Tomcat 7 SSL Properties On 16/05/2013 04:09, Chirag Dewan wrote: > > > Sorry for the previous mail. Adding Subject. > > ----- Forwarded Message - > From: Chirag Dewan > To: Tomcat Users List > Sent: Thursday, 16 May 2013 8:37 AM > Subject: > > > > Hi All, > > I am upgrading my Embedded Tomcat 6 to Tomcat 7. I am adding an HTTPS > connector to my service. > > Code for Tomcat 6 > > IntrospectionUtils.setProperty( this, "sslProtocol", >"TLS" ); > IntrospectionUtils.setProperty( this, "keystore", >keyStoreFile ); > IntrospectionUtils.setProperty( this, "keypass", >keyStorePassword ); > IntrospectionUtils.setProperty( this, "SSLEnabled", >"true" ); > > > > Embedded.addConnector( this ); > > > Code for Tomcat 7 > > > IntrospectionUtils.setProperty( this, "sslProtocol", >"TLS" ); > IntrospectionUtils.setProperty( this, "keystore", >keyStoreFile ); > IntrospectionUtils.setProperty( this, "keypass", >keyStorePassword ); > IntrospectionUtils.setProperty( this, "SSLEnabled", >"true" ); > > Tomcat.getService().addConnector( this ); > > > This is just a sample code. It worked fine in Tomcat 6,but when I start the > connector in Tomcat 7 I get an /home/.keystore(my keystore file is not in the > default location) not found exception. Now I have it corrected by changing > the attributes "keystore" to "keystoreFile" and "keypass" to > "keystorePass",the way it should be. > > > But I am puzzled,how was it still working in Tomcat 6? keystore and keypass were internal names used for those attributes. Tomcat 6 had code that mapped from the documented names to the internal names but you could use the internal names directly if you knew what they were. One of the many code clean-ups in Tomcat 7 aligned the internal names with the documented names and removed the mapping. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Embedded Tomcat 7 SSL Properties
Sorry for the previous mail. Adding Subject. - Forwarded Message - From: Chirag Dewan To: Tomcat Users List Sent: Thursday, 16 May 2013 8:37 AM Subject: Hi All, I am upgrading my Embedded Tomcat 6 to Tomcat 7. I am adding an HTTPS connector to my service. Code for Tomcat 6 IntrospectionUtils.setProperty( this, "sslProtocol", "TLS" ); IntrospectionUtils.setProperty( this, "keystore", keyStoreFile ); IntrospectionUtils.setProperty( this, "keypass", keyStorePassword ); IntrospectionUtils.setProperty( this, "SSLEnabled", "true" ); Embedded.addConnector( this ); Code for Tomcat 7 IntrospectionUtils.setProperty( this, "sslProtocol", "TLS" ); IntrospectionUtils.setProperty( this, "keystore", keyStoreFile ); IntrospectionUtils.setProperty( this, "keypass", keyStorePassword ); IntrospectionUtils.setProperty( this, "SSLEnabled", "true" ); Tomcat.getService().addConnector( this ); This is just a sample code. It worked fine in Tomcat 6,but when I start the connector in Tomcat 7 I get an /home/.keystore(my keystore file is not in the default location) not found exception. Now I have it corrected by changing the attributes "keystore" to "keystoreFile" and "keypass" to "keystorePass",the way it should be. But I am puzzled,how was it still working in Tomcat 6? Thanks.
Re: Port still busy after removing connector in Embedded Tomcat 7.0.30
> You need to destroy the connector to close the port.
Mark,
That seems to work. :)
Now the port is free. But is it the right approach? Is this something we need
to do in Tomcat 7 specifically?
Thanks.
From: Mark Thomas
To: Tomcat Users List
Sent: Tuesday, 14 May 2013 2:41 AM
Subject: Re: Port still busy after removing connector in Embedded Tomcat 7.0.30
On 13/05/2013 16:34, Chirag Dewan wrote:
> Hi,
>
> I am embedding Apache Tomcat 7.0.30 in my application. I am using the Tomcat
> class,and my application requires dynamic addition and removal of
> connectors(HTTP).
>
> Now while removing the connectors,the application gets undeployed but the
> port remains occupied and the connector continue to listen on the port. I am
> using customized HTTP connector.
>
>
> Here is a snippet from my source code:
>
> for removing connector:
>
> if( connector != null )
> {
> connector.decUsage();
> if( connector.getUsage() == 0 )
> {
> connector.stop();
> this.tomcat.getService().removeConnector( connector );
> }
> }
>
> for adding connector:
>
> this.tomcat.getService().addConnector( connector );
>
>
> I know the port is not getting free. Because when I try to deploy to the same
> port again,it gives me an address already in use exception.
> I have surfed for this issue a lot,but to no avail. Any help would be greatly
> appreciated.
You need to destroy the connector to close the port.
Mark
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Port still busy after removing connector in Embedded Tomcat 7.0.30
> If the context was removed, but the Connector was still there, should you not
> get a 404
Not Found, rather than a timeout ?
If I am calling the removeConnector,shouldn't it cleanup the context and the
free the port?
I mean is my context blocking the connector even after I removed it? i should
get a 404. Yes.
Sorry I forget to mention,it worked fine with Tomcat 6 Embedded version. I am
trying to upgrade it to Tomcat 7.0.30 and I am following the same process as in
Tomcat 6.
Thanks
From: André Warnier
To: Tomcat Users List
Sent: Monday, 13 May 2013 10:13 PM
Subject: Re: Port still busy after removing connector in Embedded Tomcat 7.0.30
Chirag Dewan wrote:
>> How do you observe that the connector is still bound to the port?
>
> Yes. I used netstat to observe that. Plus when I try to add another context
> to the same port,I get "Address already in use" exception.
>
>> What does it show?
>
> I can see my java process running on that port.
>
>> What happens if you make a request?
>
> As I said,I am sure that the deployed context is removed. I get a timeout on
>a request.
If the context was removed, but the Connector was still there, should you not
get a 404
Not Found, rather than a timeout ?
>
> When I start my Tomcat Server(tomcat.start()),I understand it adds a default
> connector to the service on port 8080,so after adding my connector on port
> 1090,i can see my java process listening on port 1090 and 8080.
>
> I really dont understand what wrong I might be doing.
>
>
> Thanks
>
>
>
>
>
>
>
>
> From: Christopher Schultz
> To: Tomcat Users List
> Sent: Monday, 13 May 2013 9:47 PM
> Subject: Re: Port still busy after removing connector in Embedded Tomcat
> 7.0.30
>
>
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> Chirag,
>
> On 5/13/13 11:34 AM, Chirag Dewan wrote:
>> I am embedding Apache Tomcat 7.0.30 in my application. I am using
>> the Tomcat class,and my application requires dynamic addition and
>> removal of connectors(HTTP).
>>
>> Now while removing the connectors,the application gets undeployed
>> but the port remains occupied and the connector continue to listen
>> on the port. I am using customized HTTP connector.
>>
>>
>> Here is a snippet from my source code:
>>
>> for removing connector:
>>
>> if( connector != null ) { connector.decUsage(); if(
>> connector.getUsage() == 0 ) { connector.stop();
>> this.tomcat.getService().removeConnector( connector ); } }
>>
>> for adding connector:
>>
>> this.tomcat.getService().addConnector( connector );
>>
>>
>> I know the port is not getting free. Because when I try to deploy
>> to the same port again,it gives me an address already in use
>> exception. I have surfed for this issue a lot,but to no avail. Any
>> help would be greatly appreciated.
>
> Can you take a thread dump and find which code look like it's doing
> connection-related things?
>
> How do you observe that the connector is still bound to the port?
> netstat? What does it show? What happens if you make a request?
>
> - -chris
> -BEGIN PGP SIGNATURE-
> Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQIcBAEBCAAGBQJRkRIbAAoJEBzwKT+lPKRYH4kP/2WiZK75ifzqSXojAw1ckw6k
> iH5NvnWNNFeibFvqtj0tN2Nun+mb3atqs4ySwQahN/FAtz8klhdfGLTlccMxzh4u
> x1yvMKLEh1Xr3qTxeot/y98zER/gubdgNsM4lr5MkkFVtaprK7/emz1OCwcuZe0L
> XApJRuSNF+GxtH9lSKEw4Tn4kzc5Asv2gMQOQmLAmBi3fKI9MsoRS9V5TPU88dhL
> XBk+rkcUdvL2KWKbP9yrt2Geu9NqBzqADCA4FJUESS9sm8A/VECzyUZ+4uW/jqiP
> qpn4GSAtpaH9CyHzynZF5qyA8Fx+BllhyIFwZfUl5DMqk+t4gw3LdPay8Rkpfg7G
> fdLHXrLez/Pcj2nvOp60OLeC4DD13AJON9KyYXmnUuk29aV/lrOes9nmUbIVDh8k
> 2LXh9cYErtD55bCgmTLPT0bLBpC9v1aUGEjFzeLjfJX9hEkvpUrN5OWvKHYg5Vaw
> 2qc+bV31qitjrUfZYU58S5UX1txATx6Ig76y/pt49XfiQFc6/GG25Ju5UCJOpat0
> mOV4qTck+Exy+YuGvME7V/IcLAbymm9vGIB4bQD9dLq36Kx0cL5lE/DOYANd2PJD
> 0nBO/co4uRvKW54NDnHVzzpAU7IUXBoQ+SmU4IQMv1+TP6auHO8ekwDW7+59ZgCc
> Ev3xtRom84ly50SQcxKf
> =zDpF
> -END PGP SIGNATURE-
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Port still busy after removing connector in Embedded Tomcat 7.0.30
>How do you observe that the connector is still bound to the port?
Yes. I used netstat to observe that. Plus when I try to add another context to
the same port,I get "Address already in use" exception.
> What does it show?
I can see my java process running on that port.
>What happens if you make a request?
As I said,I am sure that the deployed context is removed. I get a timeout on a
request.
When I start my Tomcat Server(tomcat.start()),I understand it adds a default
connector to the service on port 8080,so after adding my connector on port
1090,i can see my java process listening on port 1090 and 8080.
I really dont understand what wrong I might be doing.
Thanks
From: Christopher Schultz
To: Tomcat Users List
Sent: Monday, 13 May 2013 9:47 PM
Subject: Re: Port still busy after removing connector in Embedded Tomcat 7.0.30
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
Chirag,
On 5/13/13 11:34 AM, Chirag Dewan wrote:
> I am embedding Apache Tomcat 7.0.30 in my application. I am using
> the Tomcat class,and my application requires dynamic addition and
> removal of connectors(HTTP).
>
> Now while removing the connectors,the application gets undeployed
> but the port remains occupied and the connector continue to listen
> on the port. I am using customized HTTP connector.
>
>
> Here is a snippet from my source code:
>
> for removing connector:
>
> if( connector != null ) { connector.decUsage(); if(
> connector.getUsage() == 0 ) { connector.stop();
> this.tomcat.getService().removeConnector( connector ); } }
>
> for adding connector:
>
> this.tomcat.getService().addConnector( connector );
>
>
> I know the port is not getting free. Because when I try to deploy
> to the same port again,it gives me an address already in use
> exception. I have surfed for this issue a lot,but to no avail. Any
> help would be greatly appreciated.
Can you take a thread dump and find which code look like it's doing
connection-related things?
How do you observe that the connector is still bound to the port?
netstat? What does it show? What happens if you make a request?
- -chris
-BEGIN PGP SIGNATURE-
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJRkRIbAAoJEBzwKT+lPKRYH4kP/2WiZK75ifzqSXojAw1ckw6k
iH5NvnWNNFeibFvqtj0tN2Nun+mb3atqs4ySwQahN/FAtz8klhdfGLTlccMxzh4u
x1yvMKLEh1Xr3qTxeot/y98zER/gubdgNsM4lr5MkkFVtaprK7/emz1OCwcuZe0L
XApJRuSNF+GxtH9lSKEw4Tn4kzc5Asv2gMQOQmLAmBi3fKI9MsoRS9V5TPU88dhL
XBk+rkcUdvL2KWKbP9yrt2Geu9NqBzqADCA4FJUESS9sm8A/VECzyUZ+4uW/jqiP
qpn4GSAtpaH9CyHzynZF5qyA8Fx+BllhyIFwZfUl5DMqk+t4gw3LdPay8Rkpfg7G
fdLHXrLez/Pcj2nvOp60OLeC4DD13AJON9KyYXmnUuk29aV/lrOes9nmUbIVDh8k
2LXh9cYErtD55bCgmTLPT0bLBpC9v1aUGEjFzeLjfJX9hEkvpUrN5OWvKHYg5Vaw
2qc+bV31qitjrUfZYU58S5UX1txATx6Ig76y/pt49XfiQFc6/GG25Ju5UCJOpat0
mOV4qTck+Exy+YuGvME7V/IcLAbymm9vGIB4bQD9dLq36Kx0cL5lE/DOYANd2PJD
0nBO/co4uRvKW54NDnHVzzpAU7IUXBoQ+SmU4IQMv1+TP6auHO8ekwDW7+59ZgCc
Ev3xtRom84ly50SQcxKf
=zDpF
-END PGP SIGNATURE-
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Port still busy after removing connector in Embedded Tomcat 7.0.30
Hi,
I am embedding Apache Tomcat 7.0.30 in my application. I am using the Tomcat
class,and my application requires dynamic addition and removal of
connectors(HTTP).
Now while removing the connectors,the application gets undeployed but the port
remains occupied and the connector continue to listen on the port. I am using
customized HTTP connector.
Here is a snippet from my source code:
for removing connector:
if( connector != null )
{
connector.decUsage();
if( connector.getUsage() == 0 )
{
connector.stop();
this.tomcat.getService().removeConnector( connector );
}
}
for adding connector:
this.tomcat.getService().addConnector( connector );
I know the port is not getting free. Because when I try to deploy to the same
port again,it gives me an address already in use exception.
I have surfed for this issue a lot,but to no avail. Any help would be greatly
appreciated.
Thanks
