Re: no live upstreams and NO previous error
Hey Drookie, Can you let me know how you solved this issue? We are facing a similar issue which results in 502 gateway error and I'm not able to find a solution for it. Below is the error log from /var/log/nginx/error.log 2018/03/22 01:34:49 [error] 8779#0: *13133 no live upstreams while connecting to upstream, client: 207.189.113.68, server: 192.168.10.4, request: "POST /upload_app HTTP/1.1", upstream: "uwsgi://upload_clusters", host: "173.255.9.88" 2018/03/22 01:34:50 [error] 8779#0: *13136 no live upstreams while connecting to upstream, client: 207.189.113.68, server: 192.168.10.4, request: "POST /upload_app HTTP/1.1", upstream: "uwsgi://upload_clusters", host: "173.255.9.88" 2018/03/22 01:34:50 [error] 8779#0: *13143 no live upstreams while connecting to upstream, client: 207.189.113.68, server: 192.168.10.4, request: "POST /upload_app HTTP/1.1", upstream: "uwsgi://upload_clusters", host: "173.255.9.88" 2018/03/22 01:34:50 [error] 8779#0: *13144 no live upstreams while connecting to upstream, client: 207.189.113.68, server: 192.168.10.4, request: "POST /upload_app HTTP/1.1", upstream: "uwsgi://upload_clusters", host: "173.255.9.88" We dont get this issue for all requests but for some random requests in between. Below is the access log from /var/log/nginx/access.log 207.189.113.68 - - [22/Mar/2018:02:15:29 -0700] "POST /upload_app HTTP/1.1" 502 198 "-" "Apache-HttpClient/4.0.3 (java 1.5)" 207.189.113.68 - - [22/Mar/2018:02:15:29 -0700] "POST /upload_app HTTP/1.1" 200 162 "-" "Apache-HttpClient/4.0.3 (java 1.5)" 207.189.113.68 - - [22/Mar/2018:02:15:29 -0700] "POST /upload_app HTTP/1.1" 200 162 "-" "Apache-HttpClient/4.0.3 (java 1.5)" 207.189.113.68 - - [22/Mar/2018:02:15:29 -0700] "POST /upload_app HTTP/1.1" 502 198 "-" "Apache-HttpClient/4.0.3 (java 1.5)" 207.189.113.68 - - [22/Mar/2018:02:15:29 -0700] "POST /upload_app HTTP/1.1" 502 198 "-" "Apache-HttpClient/4.0.3 (java 1.5)" 207.189.113.68 - - [22/Mar/2018:02:15:29 -0700] "POST /upload_app HTTP/1.1" 200 162 "-" "Apache-HttpClient/4.0.3 (java 1.5)" 207.189.113.68 - - [22/Mar/2018:02:15:34 -0700] "POST /upload_app HTTP/1.1" 200 162 "-" "Apache-HttpClient/4.0.3 (java 1.5)" 207.189.113.68 - - [22/Mar/2018:02:15:34 -0700] "POST /upload_app HTTP/1.1" 200 162 "-" "Apache-HttpClient/4.0.3 (java 1.5)" 207.189.113.68 - - [22/Mar/2018:02:15:36 -0700] "POST /upload_app HTTP/1.1" 200 162 "-" "Apache-HttpClient/4.0.3 (java 1.5)" 207.189.113.68 - - [22/Mar/2018:02:15:37 -0700] "POST /upload_app HTTP/1.1" 200 162 "-" "Apache-HttpClient/4.0.3 (java 1.5)" 207.189.113.68 - - [22/Mar/2018:02:15:39 -0700] "POST /upload_app HTTP/1.1" 200 162 "-" "Apache-HttpClient/4.0.3 (java 1.5)" 207.189.113.68 - - [22/Mar/2018:02:15:42 -0700] "POST /upload_app HTTP/1.1" 200 162 "-" "Apache-HttpClient/4.0.3 (java 1.5)" 207.189.113.68 - - [22/Mar/2018:02:15:42 -0700] "POST /upload_app HTTP/1.1" 200 162 "-" "Apache-HttpClient/4.0.3 (java 1.5)" 207.189.113.68 - - [22/Mar/2018:02:15:45 -0700] "POST /upload_app HTTP/1.1" 200 162 "-" "Apache-HttpClient/4.0.3 (java 1.5)" 207.189.113.68 - - [22/Mar/2018:02:15:46 -0700] "POST /upload_app HTTP/1.1" 502 198 "-" "Apache-HttpClient/4.0.3 (java 1.5)" 207.189.113.68 - - [22/Mar/2018:02:15:46 -0700] "POST /upload_app HTTP/1.1" 200 162 "-" "Apache-HttpClient/4.0.3 (java 1.5)" As you can see some requests got 200(success) response and some got 502(gateway error). I have tried parameter tuning by increasing timeouts but no help. It would be really helpful if you can let me know what worked for you. Thanks. Posted at Nginx Forum: https://forum.nginx.org/read.php?2,269577,279148#msg-279148 ___ nginx mailing list nginx@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx
Re: no live upstreams and NO previous error
We have json parsing data in between via api's. Anybody here to advise??? Posted at Nginx Forum: https://forum.nginx.org/read.php?2,269577,272345#msg-272345 ___ nginx mailing list nginx@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx
Re: no live upstreams and NO previous error
Oh, solved. Upstreams do respond with 500. Posted at Nginx Forum: https://forum.nginx.org/read.php?2,269577,269584#msg-269584 ___ nginx mailing list nginx@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx
Re: no live upstreams and NO previous error
(yup, it's still the author of the original post, but my other browser just remembers another set of credentials). If I increase verbosity of the error_log, I'm seeing additional messages in log, like upstream server temporarily disabled while reading response header from but this message doesn't explain why the upstream server was disabled. I understand that the error occured, but what exaclty ? I'm used to see timeouts instead, or some other explicit problem. This looks totally mysterios for me. Could someone shine some light on it ? Posted at Nginx Forum: https://forum.nginx.org/read.php?2,269577,269583#msg-269583 ___ nginx mailing list nginx@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx
no live upstreams and NO previous error
Hi. I've set up a multiple upstream configuration with nginx as a load balancer. And yup, I'm getting 'no live upstreams' in the error log. Like in 1-3% of requests. And yes, I know how this works: nginx is marking a backend in an upstream as dead when he receives an error from it, and these errors are configured with proxy_next_upstream; plus, when all of the servers in an upstream group are under such holddowns, you will get this error. So if you're getting these errors, basically all you need is to fix the root cause of them, like timeouts and various 50x, and 'no live upstreams' will be long gone. But in my case I'm getting these like all of a sudden. I would be happy to see some timeouts, or 50x from backends and so on. Nope, I'm getting these: 2016/09/14 20:27:58 [error] 46898#100487: *49484 no live upstreams while connecting to upstream, client: xx.xx.xx.xx, server: foo.bar, request: "POST /mixed/json HTTP/1.1", upstream: "http://backends/mixed/json;, host: "foo.bar" And in the access log these: xx.xx.xx.xx - - [14/Sep/2016:20:27:58 +0300] foo.bar "POST /mixed/json HTTP/1.1" 502 198 "-" "-" 0.015 backends 502 - and the most funny thing is that I'm getting a bulk of these requests, and previous ones are 200. It really looks like the upstream group is switching for no reason to a dead state, and, since I don't believe in miracles, I think that there must be a cause for that, only that nginx for some reason doesn't log it. So, my question is, if this isn't caused by the HTTP errors (since I don't see the errors on the backends) - can this be caused by a sudden lack of l3 connectivity ? Like tcp connections dropped, some intermediary packet filters and so on ? Thanks. Posted at Nginx Forum: https://forum.nginx.org/read.php?2,269577,269577#msg-269577 ___ nginx mailing list nginx@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx
