Hello! On Sat, Jul 13, 2019 at 09:50:50AM -0400, heythisisom wrote:
> Hi Maxim, > > The nginx reverse proxy and uWSGI runs on the same host. Each nginx reverse > proxies are connected to only one single Instance of the uWSGI backend. > > But in the uWSGI backend, I'm running 4 workers in total based on the > configuration 2 workers can be handled by 1 VCPU. Essentially the Instance I > run has 2 VCPUs hence It translates to 4 workers. 4 uWSGI workers means that only 4 requests can be handled in parallel. As long as your code uses external resources, such as databases or external requests, this may be way too low. > The listen queue length of > my backend is sufficiently high i.e. 4096 and I have set my somaxconn > parameter to 32768. So I think everything with respect to the backend seems > fine. To large listen queue length might be a reason which actually causes the 499 errors you've observed. For example, assuming processing each request requires 1 second on your backend, and given you have 4 backend workers, listen queue length of 4096 translates to 1000 seconds delay due to queueing. You may want to monitor actual number of queued connection requests in the listening socket queue when the issue happens. Assuming you are using Linux, try "ss -nlt" see queue sizes and numbers of currently queue connection requests. > Only when I disable logging in nginx I could see this Issue happen. Once I > enable it, my hosts never raised Timeout error at all. Also, note that this > Issue happens more often when the server is in an Idle state and not when > the server is in peak. Disabling/enabled logging implies slightly different load pattern, might change various timings and/or OS scheduler behaviour, and hence change the observed results - either by triggering bugs in various places (including kernel, nginx, and your backend) or simply by making things less efficient. What exactly happens requires further investigation. You may start with looking at the "ss -nlt" numbers as suggested above. -- Maxim Dounin http://mdounin.ru/ _______________________________________________ nginx mailing list nginx@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx
