Maxim, Thank you for your reply. I will look to see if "netstat -s" detects any listen queue overflows.
John -----Original Message----- From: nginx [mailto:nginx-boun...@nginx.org] On Behalf Of Maxim Dounin Sent: Tuesday, March 27, 2018 6:55 AM To: nginx@nginx.org Subject: Re: Nginx throttling issue? Hello! On Mon, Mar 26, 2018 at 08:21:27PM +0000, John Melom wrote: > I am load testing our system using Jmeter as a load generator. > We execute a script consisting of an https request executing in a > loop. The loop does not contain a think time, since at this point I > am not trying to emulate a “real user”. I want to get a quick look at > our system capacity. Load on our system is increased by increasing > the number of Jmeter threads executing our script. Each Jmeter thread > references different data. > > Our system is in AWS with an ELB fronting Nginx, which serves as a > reverse proxy for our Docker Swarm application cluster. > > At moderate loads, a subset of our https requests start experiencing > to a 1 second delay in addition to their normal response time. The > delay is not due to resource contention. > System utilizations remain low. The response times cluster around 4 > values: 0 millilseconds, 50 milliseconds, 1 second, and 1.050 > seconds. Right now, I am most interested in understanding and > eliminating the 1 second delay that gives the clusters at 1 second and > 1.050 seconds. > > The attachment shows a response time scatterplot from one of our runs. > The x-axis is the number of seconds into the run, the y-axis is the > response time in milliseconds. The plotted data shows the response > time of requests at the time they occurred in the run. > > If I run the test bypassing the ELB and Nginx, this delay does not > occur. > If I bypass the ELB, but include Nginx in the request path, the delay > returns. > > This leads me to believe the 1 second delay is coming from Nginx. There are no magic 1 second delays in nginx - unless you've configured something explicitly. Most likely, the 1 second delay is coming from TCP retransmission timeout during connection establishment due to listen queue overflows. Check "netstat -s" to see if there are any listen queue overflows on your hosts. [...] -- Maxim Dounin http://mdounin.ru/ _______________________________________________ nginx mailing list nginx@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx ________________________________ NOTE: This email message and any attachments are for the sole use of the intended recipient(s) and may contain confidential and/or privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you have received this e-mail in error, please contact the sender by replying to this email, and destroy all copies of the original message and any material included with this email. _______________________________________________ nginx mailing list nginx@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx
