Re: TomcatCon @ ApacheCon

[Coty Sutherland]

Hi Emmanuel,

> If you already have a draft of your presentation you can send it to me
> and I'll insert a few slides about Debian.

I haven't even started yet :( I need to block out some time to submit
the abstract and start the presentation soon. If you can just give me
a few slides on the basic layout of the debian distro I can add them
to the presentation I create and then send it to you to review. If
you'd rather add to my presentation yourself, I'll just send it over
when I start working on it and we can go from there.

On Fri, Feb 3, 2017 at 9:44 AM, Emmanuel Bourg  wrote:
> Le 1/02/2017 à 20:20, Coty Sutherland a écrit :
>> I'm still planning to submit for the linux packaging talk (though I
>> haven't heard anything else form the other distro maintainers), just
>> haven't done it yet. I suppose I could volunteer for one of the
>> others, I'll check the list.
>
> Hi Coty,
>
> If you already have a draft of your presentation you can send it to me
> and I'll insert a few slides about Debian.
>
> Emmanuel Bourg
>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Increasing maxThreads results in more "Connection refused" errors?

[Christopher Schultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

John,

On 2/3/17 11:17 AM, john.e.gr...@wellsfargo.com.INVALID wrote:
> I'm doing some scalability testing of an app in Tomcat 7.0.73.
> 
> Some relevant connector config details: maxThreads=80 
> maxKeepAliveRequests=100 keepAliveTimeout=1 maxConnections
> unspecified (defaults to maxThreads according to the docs) 
> acceptCount unspecified (100 according to the docs) 
> clientAuth=true

Which connector?

> FWIW, I'm testing two Tomcat instances on the same server.  They
> are behind a load balancer.

Which load-balancer using which protocol?

> It appears that when the load generator tries to exceed
> maxThreads, the new connection rate goes up quickly and CPU usage
> shoots up with it.

When you say the "new connection rate" goes up, what exactly do you
mean? When using a load-generator, I would certainly expect the "new
connection rate" to go up.

> I assume this is because Tomcat is proactively closing idle keep
> alive connections to service new connections.

No. Tomcat behaves the same under load as when not under load. If you
are using the BIO connector (it sounds like you are), then each
connection will have to wait for the KeepAlive timeout to expire
before servicing another request from another client.

This is complicated by the reverse-proxy and so those details are
important to provide.

> In an effort to keep the CPU in check, I tried increasing
> maxThreads from 80 to 120.

If the CPU is very busy, allowing more threads to run is
counter-productive, isn't it?

> This seemed to work well in a lot of ways. New connection rate
> didn't increase as much, CPU didn't increase as much, there was
> more connection reuse (more requests per connection,) and response
> times didn't deteriorate as much.
> 
> Great, right?
> 
> Then I noticed a large increase in "Connection refused" errors on 
> the load generator. In other words, a higher maxThreads also
> results in a high error rate. The total hits per second from the
> client's perspective is about 60 in both cases. With maxThreads=80,
> there are about 3 connection refused errors per second at that
> volume. With maxThreads=120, there are about 10 connection refused
> errors per second.

Are you hitting the load-balancer or Tomcat directly during your
load-test?

If you are hitting the load-balancer, then the details of that
configuration are more important for the client: if the lb will only
accept 10 connections, it doesn't matter what Tomcat is willing to accep
t.

> I have no idea why this is.  Can someone explain this or what I can
> do about it?

Post more data. We'll get to the bottom of it.

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBCAAGBQJYlLjdAAoJEBzwKT+lPKRYRyYQAKs8F2HQ60cmlInZuubIQaq5
MuJxMZV/rCSgFMwoBzR+KDMS2PlTUvpoUMlp3/tTz8oON4x6j3jGzQdA9EGcjje4
W7D/G6TmoMptjeI8Zz0UbYnIRM5JXEvxOzuuGURbER7iCaa9VVsoWnhcv+Ja02Ut
Xl9dzEwFpfXasr8ioHL5Yn4W7F0QUDz9vboBQ/9TwYAib9UAKEP8uzWWjh6ylLaw
Z9ShyJtCU5rQ47ZVYhCcd+ZgCnP0sbcjD9kloD29dDFHgALTWPuAQA50ZDPiAUBR
I0WuDMGITpsl0ni4jbwMjrnv3voaVD0HtSCZon6fynux2BciaHVGx8s0lGtRMz6v
isG5L6+io8JR3asNdb8Ug3G+5zX8FMZx5VwlsQNVqo10xsEXYosACtqTmASyLPx6
EwgLDnCeMwKK3A6hmKqw0aEl1R9TqlpLKK5h9g7+RPpm6imUdsZvHAjSL3OWJa8e
Laqujf8MxCN/DsZhKOocwv5F98zcsn/mD9VZ/2IG+78JqfB3qU4wa5brD11n71xe
uYMgGIH0zIGN9ZGRtWdAbPbY2JIu2qIkFzz06hgYr/W5QLrZzy2dGYkyGBPMHrDV
TPR1+EbNNOtGVq5IF6t8ZChOE6toNnIdWdy5bVNiXEhIpVXquWsfoysNVlBmyA4V
VICVGii1G2WK9kfP7Q82
=UyRB
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Increasing maxThreads results in more "Connection refused" errors?

[John.E.Gregg]

All,

I'm doing some scalability testing of an app in Tomcat 7.0.73.

Some relevant connector config details:
maxThreads=80
maxKeepAliveRequests=100
keepAliveTimeout=1
maxConnections unspecified (defaults to maxThreads according to the docs)
acceptCount unspecified (100 according to the docs)
clientAuth=true

FWIW, I'm testing two Tomcat instances on the same server.  They are behind a 
load balancer.

It appears that when the load generator tries to exceed maxThreads, the new 
connection rate goes up quickly and CPU usage shoots up with it.  I assume this 
is because Tomcat is proactively closing idle keep alive connections to service 
new connections.  In an effort to keep the CPU in check, I tried increasing 
maxThreads from 80 to 120.  This seemed to work well in a lot of ways.  New 
connection rate didn't increase as much, CPU didn't increase as much, there was 
more connection reuse (more requests per connection,) and response times didn't 
deteriorate as much.

Great, right?

Then I noticed a large increase in "Connection refused" errors on the load 
generator.  In other words, a higher maxThreads also results in a high error 
rate.  The total hits per second from the client's perspective is about 60 in 
both cases.  With maxThreads=80, there are about 3 connection refused errors 
per second at that volume.  With maxThreads=120, there are about 10 connection 
refused errors per second.

I have no idea why this is.  Can someone explain this or what I can do about it?

Thanks

John

Triggering chunked encoding from PHP client

[Christopher Schultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

All,

I've got a service that has been working quite well for a long time,
and I've been asked to provide some sample PHP code to connect to it.
I've written the code and it connects, etc. just fine, except that
it's waiting for the keep-alive timeout (5s) before completing.

When I connect using a Java client, I can see that Tomcat (through
httpd) is using chunked encoding:

DEBUG: Request properties:
DEBUG:   Accept-Charset: UTF-8, ISO-8859-1, *
DEBUG:   Cache-Control: private, no-cache, no-store, no-transform
DEBUG:   Accept: application/xml, text/xml, text/plain
DEBUG:   User-Agent: My Awesome Java Client
DEBUG:   Pragma: no-cache
DEBUG:   Accept-Encoding: gzip, deflate
DEBUG:   Content-Type: application/xml
DEBUG: Received first response packet after 219ms
DEBUG: Dumping HTTP response headers
DEBUG:   Transfer-Encoding: chunked
DEBUG:   Keep-Alive: timeout=5, max=100
DEBUG:   null: HTTP/1.1 200 OK
DEBUG:   Server: Apache/2.2.22 (Debian)
DEBUG:   Connection: Keep-Alive
DEBUG:   Date: Fri, 03 Feb 2017 15:33:13 GMT
DEBUG:   Content-Type: application/xml;charset=UTF-8
DEBUG: Read response in 7ms

When I use the PHP client, here's what I get in the response headers:

array(6) {
  [0]=>
  string(15) "HTTP/1.1 200 OK"
  [1]=>
  string(35) "Date: Fri, 03 Feb 2017 15:37:01 GMT"
  [2]=>
  string(30) "Server: Apache/2.2.22 (Debian)"
  [3]=>
  string(30) "Keep-Alive: timeout=5, max=100"
  [4]=>
  string(22) "Connection: Keep-Alive"
  [5]=>
  string(43) "Content-Type: application/xml;charset=UTF-8"
}

Note that there isn't any "Transfer-Encoding: chunked" response header.

I can confirm that the PHP client is making an HTTP/1.1 request (and
the response is using HTTP/1.1 as you can see) but chunked encoding
seems not to be in use, here.

I believe I am sending the same request headers from the PHP client.
Here's what the server is seeing:

2017-02-03 10:44:59,287 [catalina-exec-2] TRACE MyServlet- [1d]
== Request from x.y.113.203
2017-02-03 10:44:59,287 [catalina-exec-2] TRACE MyServlet- [1d] host:
example.com
2017-02-03 10:44:59,287 [catalina-exec-2] TRACE MyServlet- [1d]
content-length: 135
2017-02-03 10:44:59,287 [catalina-exec-2] TRACE MyServlet- [1d]
content-type: application/xml
2017-02-03 10:44:59,287 [catalina-exec-2] TRACE MyServlet- [1d]
user-agent: My Awesome PHP Client
2017-02-03 10:44:59,287 [catalina-exec-2] TRACE MyServlet- [1d]
accept: application/xml, text/xml, text/plain
2017-02-03 10:44:59,287 [catalina-exec-2] TRACE MyServlet- [1d]
Accept-Charset: UTF-8, ISO-8859-1, *
2017-02-03 10:44:59,288 [catalina-exec-2] TRACE MyServlet- [1d]
Accept-Encoding: gzip, deflate
2017-02-03 10:44:59,288 [catalina-exec-2] TRACE MyServlet- [1d]
Cache-Control: private, no-cache, no-store, no-transform
2017-02-03 10:44:59,288 [catalina-exec-2] TRACE MyServlet- [1d]
pragma: no-cache
2017-02-03 10:44:59,288 [catalina-exec-2] TRACE MyServlet- [1d]
connection: keep-alive

And the same from the Java client:

2017-02-03 10:46:33,684 [catalina-exec-3] TRACE MyServlet- [1e]
== Request from x.y.113.203
2017-02-03 10:46:33,684 [catalina-exec-3] TRACE MyServlet- [1e]
user-agent: CHADIS-API Example Client (Java)
2017-02-03 10:46:33,684 [catalina-exec-3] TRACE MyServlet- [1e]
content-type: application/xml
2017-02-03 10:46:33,684 [catalina-exec-3] TRACE MyServlet- [1e]
accept: application/xml, text/xml, text/plain
2017-02-03 10:46:33,684 [catalina-exec-3] TRACE MyServlet- [1e]
Accept-Encoding: gzip, deflate
2017-02-03 10:46:33,684 [catalina-exec-3] TRACE MyServlet- [1e]
Accept-Charset: UTF-8, ISO-8859-1, *
2017-02-03 10:46:33,684 [catalina-exec-3] TRACE MyServlet- [1e]
Cache-Control: private, no-cache, no-store, no-transform
2017-02-03 10:46:33,684 [catalina-exec-3] TRACE MyServlet- [1e]
pragma: no-cache
2017-02-03 10:46:33,684 [catalina-exec-3] TRACE MyServlet- [1e] host:
example.com
2017-02-03 10:46:33,684 [catalina-exec-3] TRACE MyServlet- [1e]
connection: keep-alive
2017-02-03 10:46:33,685 [catalina-exec-3] TRACE MyServlet- [1e]
content-length: 135


I'm using the same "call" payload and the response should be identical
for the two clients.

If I add "Connection: close" to the headers from the PHP client then
the call completes immediately without the 5-second delay.

PHP is documented to support chunked encoding, but the server doesn't
look like it's even trying to use chunked encoding, since the response
headers don't say anything about it.

Any ideas as to what might be happening?

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBCAAGBQJYlKbWAAoJEBzwKT+lPKRYRlIQAKnMWIv+hTKTQYxLVrG/CX3A
7abMOCmgOeSuWc2ejNmC4GnIbrObAoDgB874+uNJFJudhUScNMRUtyaVnhtbrvp7
TpBLhpb+y6qNjRjSGGmCmZ5L1S4Z7eoYdZAjXKR2zBKaRSR7mjHGq+yu7HsfjEue
/cum3MDa9fy/5hCi3zfoy7oiRh1yRLLbUq3mXmfP5xblu+BPvJ35Z5bQWrmRU68B
wWwDF6aV6eGaqq/jODnpTJ1N+sw0ifY8UpTeLEmtZnolONfLxG07piUKair2u3Wz

Re: TomcatCon @ ApacheCon

[Huxing Zhang]

Hi Chris,

I've submitted the proposal to:
http://events.linuxfoundation.org/cfp/proposals/14951/13401

Any thoughts, sugestions are appreacated!

I am planning to add my proposal to the wiki page [1], but not sure which 
section it should fit to. Support? Performance? Community?

BTW, your two propsals seems mis-linked in [1], the links should be swapped...

[1] https://cwiki.apache.org/confluence/display/TOMCAT/TomcatCon+NA+2017

Thanks,
Huxing


--
From:Christopher Schultz 
Time:2017 Feb 3 (Fri) 22:16
To:Tomcat Users List 
Subject:Re: TomcatCon @ ApacheCon


-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Huxing,

On 2/2/17 9:59 PM, Huxing Zhang wrote:
> Happy Chinese New Year!
> 
> I am planning to submit a topic about the challenges tomcat faces 
> under high throughputs production systems, how we troubleshooting 
> issues under such scenarios and how we fix it.
> 
> This is the first time for me to submit a proposal, therefore it 
> would be glad if someone can offer me help on finishing it.
> 
> I am planning to submit the proposal today or tomorrow, any 
> suggestions will be appreciated.

Go ahead and submit the proposal -- you can submit the slides later, so
all you need is a 4-sentence abstract.

I'd be happy to have a look at what you have so far (if you have
anything), or to help you get started. Feel free to email me directly.

- -chris

> --
>  From:Christopher Schultz  Time:2017
> Feb 2 (Thu) 03:09 To:Tomcat Users List  
> Subject:Re: TomcatCon @ ApacheCon
> 
> 
> All,
> 
> On 1/17/17 5:04 PM, Mark Thomas wrote:
>> On 09/01/2017 11:57, Mark Thomas wrote:
> 
>> 
> 
>>> I look forward to hearing your topic ideas.
> 
>> Thanks for all the great ideas so far. I've tried to pull them
>> all together here: 
>> https://cwiki.apache.org/confluence/display/TOMCAT/TomcatCon+NA+2017
>
>>
>> 
I'm waiting to hear back from the ApacheCon folks as to how this
>> would work. I'll update this thread with info as I get it.
> 
> Only 10 more days before the CFP closes:
> 
> http://events.linuxfoundation.org/events/apachecon-north-america/progr
am
>
>
> 
/cfp
> 
> I haven't seen anyone specifically mention that they had submitted
> a proposal. I'm just really hoping that it's not markt and me 
> tag-teaming a whole day of presentations (again).
> 
> I am certainly willing to help someone get started if they'd like
> to do a presentation and aren't sure how to proceed.
> 
> -chris
> 
> -
>
>
> 
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org 
> -
>
>
> 
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
> 
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBCAAGBQJYlJCnAAoJEBzwKT+lPKRYPWMP/Rmkdn3rT+REEE/V7kY28JN2
Icvn6MbZP9hTH5tF+jr358bTSkgWRtQz6rhvmpFZwM29RodHa7wOP4/fTpw4mjpA
BYCimDXXwvf/B3mV5W8ienX1Zm0TUhCEoodBLsrrEtz0OggYp+hYUSqQWm3WGHnm
Wy3zPBzHVkzzlKb293Y8h5znyHMYHc/8giPevSQNaNXO2xYFBSQbE9k2v/y7bC6t
UW9J2BudbeyeZw7BWZRaFWpTj4J/tlQHuSaBWvvWyaMsyo8Yb00OMJrZElFclg00
smRSZPN7qo/SRcQsS4KD3WdlCaMnAA9y/Em4f4ka3isHVaZzQLvvWorEWlSLYfoP
E+O0qoOytxfwGmHbNKcJmVmPFDpkcRJEON5VpbZdCfgcwJEJOgps+8jsv4gfrI7p
EwG0GcCmPEqYcAFnZcO1pkb5jGSO9QiBxsoqU0+EejXeZB/VXirzerGxbdaXy68b
y9mBqHbKlEBDi0pNOirIvPnDZMMKht1+pY1TOBcFWGRGYPg9dROZsl/My88yT91j
ZQ7YV/rSG1HuTcUpRTXaeXFq9s5Q5uwOlXlxXm2p6lv2QMyys/OKQxFPXNDFyubQ
Aeg08N0FtRvp0YJVOYfqy5KCj1CyH2XvxdlOkvfYsjES6Mmr8P1qlrYgtl+AO6PA
1rJugsSZTGwcjBBZNZ9f
=khAi
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: TomcatCon @ ApacheCon

[Emmanuel Bourg]

Le 1/02/2017 à 20:20, Coty Sutherland a écrit :
> I'm still planning to submit for the linux packaging talk (though I
> haven't heard anything else form the other distro maintainers), just
> haven't done it yet. I suppose I could volunteer for one of the
> others, I'll check the list.

Hi Coty,

If you already have a draft of your presentation you can send it to me
and I'll insert a few slides about Debian.

Emmanuel Bourg


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: TomcatCon @ ApacheCon

[Christopher Schultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Huxing,

On 2/2/17 9:59 PM, Huxing Zhang wrote:
> Happy Chinese New Year!
> 
> I am planning to submit a topic about the challenges tomcat faces 
> under high throughputs production systems, how we troubleshooting 
> issues under such scenarios and how we fix it.
> 
> This is the first time for me to submit a proposal, therefore it 
> would be glad if someone can offer me help on finishing it.
> 
> I am planning to submit the proposal today or tomorrow, any 
> suggestions will be appreciated.

Go ahead and submit the proposal -- you can submit the slides later, so
all you need is a 4-sentence abstract.

I'd be happy to have a look at what you have so far (if you have
anything), or to help you get started. Feel free to email me directly.

- -chris

> --
>  From:Christopher Schultz  Time:2017
> Feb 2 (Thu) 03:09 To:Tomcat Users List  
> Subject:Re: TomcatCon @ ApacheCon
> 
> 
> All,
> 
> On 1/17/17 5:04 PM, Mark Thomas wrote:
>> On 09/01/2017 11:57, Mark Thomas wrote:
> 
>> 
> 
>>> I look forward to hearing your topic ideas.
> 
>> Thanks for all the great ideas so far. I've tried to pull them
>> all together here: 
>> https://cwiki.apache.org/confluence/display/TOMCAT/TomcatCon+NA+2017
>
>>
>> 
I'm waiting to hear back from the ApacheCon folks as to how this
>> would work. I'll update this thread with info as I get it.
> 
> Only 10 more days before the CFP closes:
> 
> http://events.linuxfoundation.org/events/apachecon-north-america/progr
am
>
>
> 
/cfp
> 
> I haven't seen anyone specifically mention that they had submitted
> a proposal. I'm just really hoping that it's not markt and me 
> tag-teaming a whole day of presentations (again).
> 
> I am certainly willing to help someone get started if they'd like
> to do a presentation and aren't sure how to proceed.
> 
> -chris
> 
> -
>
>
> 
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org 
> -
>
>
> 
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
> 
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBCAAGBQJYlJCnAAoJEBzwKT+lPKRYPWMP/Rmkdn3rT+REEE/V7kY28JN2
Icvn6MbZP9hTH5tF+jr358bTSkgWRtQz6rhvmpFZwM29RodHa7wOP4/fTpw4mjpA
BYCimDXXwvf/B3mV5W8ienX1Zm0TUhCEoodBLsrrEtz0OggYp+hYUSqQWm3WGHnm
Wy3zPBzHVkzzlKb293Y8h5znyHMYHc/8giPevSQNaNXO2xYFBSQbE9k2v/y7bC6t
UW9J2BudbeyeZw7BWZRaFWpTj4J/tlQHuSaBWvvWyaMsyo8Yb00OMJrZElFclg00
smRSZPN7qo/SRcQsS4KD3WdlCaMnAA9y/Em4f4ka3isHVaZzQLvvWorEWlSLYfoP
E+O0qoOytxfwGmHbNKcJmVmPFDpkcRJEON5VpbZdCfgcwJEJOgps+8jsv4gfrI7p
EwG0GcCmPEqYcAFnZcO1pkb5jGSO9QiBxsoqU0+EejXeZB/VXirzerGxbdaXy68b
y9mBqHbKlEBDi0pNOirIvPnDZMMKht1+pY1TOBcFWGRGYPg9dROZsl/My88yT91j
ZQ7YV/rSG1HuTcUpRTXaeXFq9s5Q5uwOlXlxXm2p6lv2QMyys/OKQxFPXNDFyubQ
Aeg08N0FtRvp0YJVOYfqy5KCj1CyH2XvxdlOkvfYsjES6Mmr8P1qlrYgtl+AO6PA
1rJugsSZTGwcjBBZNZ9f
=khAi
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org