jaysonsantos opened a new issue, #11779:
URL: https://github.com/apache/apisix/issues/11779
### Current Behavior
apisix uses a singleton to load the service account file and kubernetes
rotates roughly every 90 days and after that time, the discovery will fail to
get new pods with Unauthorized returned from kubernetes' API leading to stale
pods in memory and nginx making calls to pods that do not exist anymore (in
case deployments were rolled out)
### Expected Behavior
apisix should re-read the token file every X days
### Error Logs
```
apisix-57c57fd48b-hqzq9 apisix 2024/11/22 13:34:23 [error] 57#57: *509002587
[lua] informer_factory.lua:295: list failed, kind: Endpoints, reason:
Unauthorized, message :
{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"Unauthorized","reason":"Unauthorized","code":401}
apisix-57c57fd48b-xgcjv apisix 2024/11/22 13:34:24 [error] 57#57: *508864131
[lua] informer_factory.lua:295: list failed, kind: Endpoints, reason:
Unauthorized, message :
{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"Unauthorized","reason":"Unauthorized","code":401}
apisix-57c57fd48b-zgp7b apisix 2024/11/22 13:34:25 [error] 56#56: *508946548
[lua] informer_factory.lua:295: list failed, kind: Endpoints, reason:
Unauthorized, message :
{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"Unauthorized","reason":"Unauthorized","code":401}
```
### Steps to Reproduce
If there is a way to rotate the service account, this would reproduce it but
i am not sure it is possible
### Environment
- APISIX version (run `apisix version`):
/usr/local/openresty//luajit/bin/luajit ./apisix/cli/apisix.lua version
3.5.0
- Operating system (run `uname -a`):
Linux apisix-7bd7684cdf-2k524 5.10.220-209.869.amzn2.x86_64 #1 SMP Wed Jul
17 15:10:20 UTC 2024 x86_64 GNU/Linux
- OpenResty / Nginx version (run `openresty -V` or `nginx -V`):
```
nginx version: openresty/1.21.4.2
built by gcc 10.2.1 20210110 (Debian 10.2.1-6)
built with OpenSSL 1.1.1s 1 Nov 2022
TLS SNI support enabled
configure arguments: --prefix=/usr/local/openresty/nginx --with-cc-opt='-O2
-DAPISIX_BASE_VER=1.21.4.2.0
-DNGX_GRPC_CLI_ENGINE_PATH=/usr/local/openresty/libgrpc_engine.so
-DNGX_HTTP_GRPC_CLI_ENGINE_PATH=/usr/local/openresty/libgrpc_engine.so
-DNGX_LUA_ABORT_AT_PA
NIC -I/usr/local/openresty/zlib/include -I/usr/local/openresty/pcre/include
-I/usr/local/openresty/openssl111/include' --add-module=../ngx_devel_kit-0.3.2
--add-module=../echo-nginx-module-0.63 --add-module=../xss-nginx-module-0.06
--add-module=../ngx_coolkit-0.2
--add-module=../set-misc-nginx-module-0.33
--add-module=../form-input-nginx-module-0.12
--add-module=../encrypted-session-nginx-module-0.09
--add-module=../srcache-nginx-module-0.33 --add-module=../ngx_lua-0.10.25
--add-module=../ngx_lua_upstream-0.07 --add-modu
le=../headers-more-nginx-module-0.34
--add-module=../array-var-nginx-module-0.06
--add-module=../memc-nginx-module-0.19 --add-module=../redis2-nginx-module-0.15
--add-module=../redis-nginx-module-0.3.9 --add-module=../ngx_stream_lua-0.0.13
--with-ld-opt='-Wl,-rpa
th,/usr/local/openresty/luajit/lib
-Wl,-rpath,/usr/local/openresty/wasmtime-c-api/lib
-L/usr/local/openresty/zlib/lib -L/usr/local/openresty/pcre/lib
-L/usr/local/openresty/openssl111/lib
-Wl,-rpath,/usr/local/openresty/zlib/lib:/usr/local/openresty/pcre/lib:/usr
/local/openresty/openssl111/lib'
--add-module=/tmp/tmp.0EeoYgSz2t/openresty-1.21.4.2/../mod_dubbo-1.0.2
--add-module=/tmp/tmp.0EeoYgSz2t/openresty-1.21.4.2/../ngx_multi_upstream_module-1.1.1
--add-module=/tmp/tmp.0EeoYgSz2t/openresty-1.21.4.2/../apisix-nginx-modu
le-1.14.0
--add-module=/tmp/tmp.0EeoYgSz2t/openresty-1.21.4.2/../apisix-nginx-module-1.14.0/src/stream
--add-module=/tmp/tmp.0EeoYgSz2t/openresty-1.21.4.2/../apisix-nginx-module-1.14.0/src/meta
--add-module=/tmp/tmp.0EeoYgSz2t/openresty-1.21.4.2/../wasm-nginx-mod
ule-0.6.5
--add-module=/tmp/tmp.0EeoYgSz2t/openresty-1.21.4.2/../lua-var-nginx-module-v0.5.3
--add-module=/tmp/tmp.0EeoYgSz2t/openresty-1.21.4.2/../grpc-client-nginx-module-v0.4.3
--with-poll_module --with-pcre-jit --with-stream --with-stream_ssl_module
--with-st
ream_ssl_preread_module --with-http_v2_module --without-mail_pop3_module
--without-mail_imap_module --without-mail_smtp_module
--with-http_stub_status_module --with-http_realip_module
--with-http_addition_module --with-http_auth_request_module --with-http_secure_
link_module --with-http_random_index_module --with-http_gzip_static_module
--with-http_sub_module --with-http_dav_module --with-http_flv_module
--with-http_mp4_module --with-http_gunzip_module --with-threads --with-compat
--with-stream --with-http_ssl_module
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
