Hello, On a system with a load of about 500-600 URI/sec I see some unexpected behaviour when using "aio threads" option in the configuration.
System setup: The system runs on RHEL6.6 with 3 workers running nginx 1.9.6 with thread support. Content is cached and populated by a proxied-upstream. The cache location is a tmpfs file system with more then enough space at all times. Proxy buffer size 8k. The output buffer is default (no config item, so 2 32k). Keepalive timeout 75s. Sendfile is enabled. Seen behaviour: On the WAF in front of this system I see occasional hangs on resources (mainly larger files like js, jpeg, ..). Seen in the WAF log is that this WAF waits for the transfer to be completed until nginx closes the connection at the keepalive time of 75s. In the nginx access.log I see the entry served from cache (upstream server '-') with the correct content length. In the tcp dump I see the response of this call to contain a content-length header with the correct length, a server time header over 1 minute older then the tcpdump timestamp (all servers are ntp-connected). The served jpeg is half-way in its cache lifetime at that time and there are previous served entries from cache without incomplete transfers. In the tcp dump the jpeg file starts to differ from the original after 32168 bytes and misses 8192 bytes after which the remaining content is served (which is identical to original). From the tcpdump I can extract the file which is missing 8192 bytes. We have also a dump when during the proxied call this same behaviour was seen. The upstream call is started to get a jpeg from the origin. After a few packets the data is sent to the WAF. The complete upstream file is retrieved (can be validated in the tcpdump that the jpeg is complete and correctly retrieved), but not all the data is sent to the listening socket to the WAF. If I change the setup to "aio on" or "aio off" this behaviour is not seen. This is the only change in the configuration between the tests. It looks like this behaviour only affects bigger files. I have not seen this effect on small files or proxied responses. Does anyone have the same experience with this option. And what is the best way to proceed in tracing this? Regards, B. _______________________________________________ nginx-devel mailing list nginx-devel@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel
