Hello again. Maxim Dounin Wrote: ------------------------------------------------------- > Hello! > > On Thu, Feb 25, 2016 at 05:46:32AM -0500, vergil wrote: > > > vedranf Wrote: > > ------------------------------------------------------- > > > Hello, > > > > > > I'm having an issue where nginx (1.8) cache manager suddenly just > > > stops deleting content thus the disk soon ends up being full until > I > > > restart it by hand. After it is restarted, it works normally for a > > > couple of days, but then it happens again. Cache has some 30-40k > > > files, nothing huge. Relevant config lines are: > > [...] > > > We have the same problem, but i'm not sure, that this is caused by > often > > nginx restarts. > > This particular case was traced to segmentation faults, likely > caused by 3rd party modules. > > [...] > > > Also, i think it's somehow related to write connection leak. (see > image > > link) > > > > > https://s3.eu-central-1.amazonaws.com/drive-public-eu/nginx/betelgeuse > _nginx_connections.PNG > > [...] > > > As you see write connections continuously grows. (When we had to > power off > > the machine it's reached ~60k). > > > > For counting nginx connections we use standard > http_stub_status_module. > > > > I think that nginx "reference counter" could be broken, because > total > > established TCP connection remains the same all the time. > > Writing connections will grow due to segmentation faults as well, > so you are likely have the same problem. See basic > recommendations in my initial answer in this threads.
I've maded custom nginx build using latest version (1.9.13) without 3rd party modules: nginx -V nginx version: nginx/1.9.13 built by gcc 4.8.4 (Ubuntu 4.8.4-2ubuntu1~14.04.1) built with OpenSSL 1.0.1f 6 Jan 2014 TLS SNI support enabled configure arguments: --with-cc-opt='-g -O2 -fPIE -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2' --with-ld-opt='-Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now' --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-debug --with-pcre-jit --with-ipv6 --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_addition_module --with-http_dav_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_v2_module --with-http_sub_module --with-http_xslt_module --with-stream --with-stream_ssl_module --with-mail --with-mail_ssl_module --with-threads Nothing changed: connections continuously grow, cache manager works fine and not filled the disk yet, but i think it's a matter of 2-3 days. PIDs didn't changed since the start, and log didn't contain "worker process exited ..." messages. Posted at Nginx Forum: https://forum.nginx.org/read.php?2,264599,266142#msg-266142 _______________________________________________ nginx mailing list [email protected] http://mailman.nginx.org/mailman/listinfo/nginx
