sandy420 opened a new issue #4160:
URL: https://github.com/apache/apisix/issues/4160
### Issue description
### Environment
* apisix 2.4
* centos 7.6
* nginx version: openresty/1.19.3.1
* etcdctl version: 3.4.13
* apisix-dashboard 2.5
My APISIX 2.4 Prometheus plug-in reported "500 Internal Server Error" after
running for a period of time. After restarting apisix, it returned to normal,
and a 500 error was reported after a while
1.error.log
`2021/04/29 18:11:58 [warn] 17627#17627: *17251 [lua] ctx.lua:153:
__index(): failed to fetch cookie value by key: cookie_ymmoa_passport error: no
cookie foun
d in the current request, client: 10.13.65.71, server: , request: "POST
/inner/loaner/query-user-auth-info HTTP/1.1", host: "loaner.dev-ag.56qq.com"
2021/04/29 18:11:58 [warn] 17627#17627: *17251 [lua] ctx.lua:153: __index():
failed to fetch cookie value by key: cookie_dev_passport error: no cookie found
in the current request, client: 10.13.65.71, server: , request: "POST
/inner/loaner/query-user-auth-info HTTP/1.1", host: "loaner.dev-ag.56qq.com"
2021/04/29 18:12:00 [error] 17627#17627: *18598 lua entry thread aborted:
runtime error: ...seidon/apisix-2.4/apisix/plugins/prometheus/exporter.lua:206:
att
empt to compare number with string
stack traceback:
coroutine 0:
...seidon/apisix-2.4/apisix/plugins/prometheus/exporter.lua: in
function 'set_modify_index'
...seidon/apisix-2.4/apisix/plugins/prometheus/exporter.lua:229: in
function 'etcd_modify_index'
...seidon/apisix-2.4/apisix/plugins/prometheus/exporter.lua:295: in
function 'handler'
/home/poseidon/apisix-2.4/apisix/api_router.lua:130: in function
'handler'
...eidon/apisix-2.4//deps/share/lua/5.1/resty/radixtree.lua:720: in
function 'dispatch'
/home/poseidon/apisix-2.4/apisix/api_router.lua:165: in function 'match'
/home/poseidon/apisix-2.4/apisix/init.lua:292: in function
'http_access_phase'
access_by_lua(nginx.conf:205):2: in main chunk, client: 10.10.10.100,
server: , request: "GET /apisix/prometheus/metrics HTTP/1.1", host:
"10.10.10.100:80"
2021/04/29 18:12:15 [error] 17627#17627: *19213 lua entry thread aborted:
runtime error: ...seidon/apisix-2.4/apisix/plugins/prometheus/exporter.lua:206:
att
empt to compare number with string
stack traceback:
coroutine 0:
...seidon/apisix-2.4/apisix/plugins/prometheus/exporter.lua: in
function 'set_modify_index'
...seidon/apisix-2.4/apisix/plugins/prometheus/exporter.lua:229: in
function 'etcd_modify_index'
...seidon/apisix-2.4/apisix/plugins/prometheus/exporter.lua:295: in
function 'handler'
/home/poseidon/apisix-2.4/apisix/api_router.lua:130: in function
'handler'
...eidon/apisix-2.4//deps/share/lua/5.1/resty/radixtree.lua:720: in
function 'dispatch'
/home/poseidon/apisix-2.4/apisix/api_router.lua:165: in function 'match'
/home/poseidon/apisix-2.4/apisix/init.lua:292: in function
'http_access_phase'
access_by_lua(nginx.conf:205):2: in main chunk, client: 10.10.10.100,
server: , request: "GET /apisix/prometheus/metrics HTTP/1.1", host:
"10.10.10.100:80"
`
2.my config.yaml:
etcd:
host:
- "http://10.10.0.10:2379";
apisix:
admin_key:
- name: "admin"
# yamllint disable rule:comments-indentation
key: 123456789 # using fixed API token has security risk, please
update it when you deploy to production environment
# yamllint enable rule:comments-indentation
role: admin
plugins:
- prometheus
- log-rotate
- proxy-rewrite
- redirect
plugin_attr:
prometheus:
export_uri: /apisix/prometheus/metrics
server-info:
report_interval: 60
report_ttl: 3600
log-rotate:
interval: 86400
max_kept: 30
nginx_config:
http:
access_log_format:
"$time_iso8601|$msec|$status|$request_completion|$bytes_sent|$body_bytes_sent|$realip_remote_addr|$remote_addr|$http_x_forwarded_for|$
remote_user|$host|$server_name|$server_port|$server_protocol|$scheme|$request_method|$request_length|$request_time|$request_uri|$uri|$content_length|$content_type|$http_referer|$http_user_agent|$http_app_jb|$http_client_info|$upstream_addr|$upstream_connect_time|$upstream_header_time|$upstream_response_time|$upstream_status|$upstream_bytes_received|$upstream_cache_status|$upstream_http_content_type|$upstream_http_content_length|$upstream_http_content_disposition|$http_x_cat_parent_id|$upstream_scheme://$upstream_host$upstream_uri"
Seek help !How to solve this problem?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
