yangxikun opened a new issue #5622:
URL: https://github.com/apache/apisix/issues/5622


   ### Issue description
   
   APISIX Pod occasional failed on readiness probe(GET /healthz). I wrote a 
program to do http health check and log netstat output.
   
   My program error log:
   
   ```
   2021/11/26 01:10:45 err:  Get "http://localhost:8080/healthz": context 
deadline exceeded (Client.Timeout exceeded while awaiting headers)
   2021/11/26 01:10:45 2021-11-26 01:10:43.345216628 +0000 UTC 
m=+39831.465577142 Get Conn: localhost:8080
   2021-11-26 01:10:43.345260592 +0000 UTC m=+39831.465621171 DNS Start: 
{Host:localhost}
   2021-11-26 01:10:43.345368086 +0000 UTC m=+39831.465728613 DNS Info: 
{Addrs:[{IP:127.0.0.1 Zone:} {IP:::1 Zone:}] Err:<nil> Coalesced:false}
   2021-11-26 01:10:43.34539942 +0000 UTC m=+39831.465759942 Connect Start: tcp 
127.0.0.1:8080
   2021-11-26 01:10:43.649157665 +0000 UTC m=+39831.769518188 Connect Start: 
tcp [::1]:8080
   2021-11-26 01:10:43.649219174 +0000 UTC m=+39831.769579704 Connect Done: tcp 
[::1]:8080 dial tcp [::1]:8080: connect: cannot assign requested address
   2021-11-26 01:10:45.345542547 +0000 UTC m=+39833.465903093 Connect Done: tcp 
127.0.0.1:8080 dial tcp 127.0.0.1:8080: i/o timeout
   ```
   
   AND the netstat show APISIX is not read network packet(recv-q > 0):
   
   
![image](https://user-images.githubusercontent.com/3364160/143543610-86b7ca04-8b17-4507-9692-994001864dc6.png)
   
   
   ### Environment
   
   - apisix version (cmd: `apisix version`): 2.10.0
   - OS (cmd: `uname -a`): 4.14.105-1-tlinux3-0016 #1 SMP Fri Jul 31 16:16:47 
CST 2020 x86_64 x86_64 x86_64 GNU/Linux
   - OpenResty / Nginx version (cmd: `nginx -V` or `openresty -V`): nginx 
version: openresty/1.19.9.1
   built by gcc 9.3.1 20200408 (Red Hat 9.3.1-2) (GCC)
   built with OpenSSL 1.1.1k  25 Mar 2021 (running with OpenSSL 1.1.1l  24 Aug 
2021)
   TLS SNI support enabled
   configure arguments: --prefix=/usr/local/openresty/nginx --with-cc-opt='-O2 
-DNGX_LUA_ABORT_AT_PANIC -I/usr/local/openresty/zlib/include 
-I/usr/local/openresty/pcre/include -I/usr/local/openresty/openssl111/include' 
--add-module=../ngx_devel_kit-0.3.1 --add-module=../echo-nginx-module-0.62 
--add-module=../xss-nginx-module-0.06 --add-module=../ngx_coolkit-0.2 
--add-module=../set-misc-nginx-module-0.32 
--add-module=../form-input-nginx-module-0.12 
--add-module=../encrypted-session-nginx-module-0.08 
--add-module=../srcache-nginx-module-0.32 --add-module=../ngx_lua-0.10.20 
--add-module=../ngx_lua_upstream-0.07 
--add-module=../headers-more-nginx-module-0.33 
--add-module=../array-var-nginx-module-0.05 
--add-module=../memc-nginx-module-0.19 --add-module=../redis2-nginx-module-0.15 
--add-module=../redis-nginx-module-0.3.7 --add-module=../ngx_stream_lua-0.0.10 
--with-ld-opt='-Wl,-rpath,/usr/local/openresty/luajit/lib 
-L/usr/local/openresty/zlib/lib -L/usr/local/openresty/pcre/lib -L/usr/loc
 al/openresty/openssl111/lib 
-Wl,-rpath,/usr/local/openresty/zlib/lib:/usr/local/openresty/pcre/lib:/usr/local/openresty/openssl111/lib'
 --with-cc='ccache gcc -fdiagnostics-color=always' --with-pcre-jit 
--with-stream --with-stream_ssl_module --with-stream_ssl_preread_module 
--with-http_v2_module --without-mail_pop3_module --without-mail_imap_module 
--without-mail_smtp_module --with-http_stub_status_module 
--with-http_realip_module --with-http_addition_module 
--with-http_auth_request_module --with-http_secure_link_module 
--with-http_random_index_module --with-http_gzip_static_module 
--with-http_sub_module --with-http_dav_module --with-http_flv_module 
--with-http_mp4_module --with-http_gunzip_module --with-threads --with-compat 
--with-stream --with-http_ssl_module
   - etcd version, if have (cmd: run `curl 
http://127.0.0.1:9090/v1/server_info` to get the info from server-info API):
   - apisix-dashboard version, if have:
   - the plugin runner version, if the issue is about a plugin runner (cmd: 
depended on the kind of runner):
   - luarocks version, if the issue is about installation (cmd: `luarocks 
--version`):
   
   
   ### Steps to reproduce
   
   It's occasionally, I cannot provide reproduce steps.
   
   ### Actual result
   
   APISIX blocking temporary. Cause some request fail.
   
   ### Error log
   
   No any error log.
   
   ### Expected result
   
   _No response_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


Reply via email to