riverwoo opened a new issue, #1912:
URL: https://github.com/apache/apisix-ingress-controller/issues/1912
### Issue description
Kubernetes deploys APISIX, Apisix-ingress-Controller has a large number of
error, the Image version is 1.6.1, and the APISIXROUTE cannot be seen in
dashboard. The APISIX version is 3.4.1, and the Dashboard version is 3.0.1
```
[root@k8s-dev-master yunwei]# kubectl logs -f -n ingress-apisix
apisix-ingress-controller-5fb55f45df-lqvdn
Defaulted container "apisix-ingress-controller" out of:
apisix-ingress-controller, wait-apisix-admin (init)
2023-08-01T19:24:23+08:00 info ingress/ingress.go:128 init apisix
ingress controller
2023-08-01T19:24:23+08:00 info ingress/ingress.go:130 version:
Version: 1.6.1
Git SHA: no-git-module
Go Version: go1.19.8
Building OS/Arch: linux/amd64
Running OS/Arch: linux/amd64
2023-08-01T19:24:23+08:00 info ingress/ingress.go:140 use
configuration
{
"cert_file": "/etc/webhook/certs/cert.pem",
"key_file": "/etc/webhook/certs/key.pem",
"log_level": "info",
"log_output": "stderr",
"log_rotate_output_path": "",
"log_rotation_max_size": 100,
"log_rotation_max_age": 0,
"log_rotation_max_backups": 0,
"http_listen": ":8080",
"https_listen": ":8443",
"ingress_publish_service": "",
"ingress_status_address": [],
"enable_profiling": true,
"kubernetes": {
"kubeconfig": "",
"resync_interval": "6h0m0s",
"namespace_selector": [],
"election_id": "ingress-apisix-leader",
"ingress_class": "apisix",
"ingress_version": "networking/v1",
"watch_endpoint_slices": false,
"api_version": "apisix.apache.org/v2",
"enable_gateway_api": false
},
"apisix": {
"admin_api_version": "v2",
"default_cluster_name": "default",
"default_cluster_base_url":
"http://apisix-admin.default.svc.cluster.local:9180/apisix/admin";,
"default_cluster_admin_key": "******"
},
"apisix-resource-sync-interval": "5m0s",
"plugin_metadata_cm": ""
}
2023-08-01T19:24:23+08:00 warn api/server.go:76 failed to load
x509 key pair, will not start admission server {"Error": "open
/etc/webhook/certs/cert.pem: no such file or directory", "CertFilePath":
"/etc/webhook/certs/cert.pem", "KeyFilePath": "/etc/webhook/certs/key.pem"}
2023-08-01T19:24:23+08:00 info ingress/ingress.go:152 start ingress
controller
I0801 19:24:23.920356 1 leaderelection.go:248] attempting to acquire
leader lease ingress-apisix/ingress-apisix-leader...
2023-08-01T19:24:23+08:00 info providers/controller.go:151 start
api server
2023-08-01T19:24:23+08:00 info providers/controller.go:137
LeaderElection {"message": "apisix-ingress-controller-5fb55f45df-lqvdn became
leader", "event_type": "Normal"}
I0801 19:24:23.933770 1 leaderelection.go:258] successfully acquired
lease ingress-apisix/ingress-apisix-leader
2023-08-01T19:24:23+08:00 warn providers/controller.go:176 found a
new leader apisix-ingress-controller-5fb55f45df-lqvdn
2023-08-01T19:24:23+08:00 info providers/controller.go:360
controller tries to leading ... {"namespace": "ingress-apisix", "pod":
"apisix-ingress-controller-5fb55f45df-lqvdn"}
2023-08-01T19:24:23+08:00 info apisix/stream_route.go:38
resource stream_routes is disabled
2023-08-01T19:24:23+08:00 warn apisix/cluster.go:344 waiting cluster
default to ready, it may takes a while
2023-08-01T19:24:23+08:00 info apisix/cluster.go:172 syncing cache
{"cluster": "default"}
2023-08-01T19:24:23+08:00 info apisix/cluster.go:384 syncing schema
{"cluster": "default"}
2023-08-01T19:24:23+08:00 error apisix/ssl.go:120 failed to list
ssl: unexpected status code 404; error message: {"error_msg":"Unsupported
resource type: ssl"}
2023-08-01T19:24:23+08:00 error apisix/cluster.go:232 failed to list
ssl in APISIX: unexpected status code 404; error message:
{"error_msg":"Unsupported resource type: ssl"}
2023-08-01T19:24:24+08:00 info [email protected]/context.go:173 path:
/healthz, status: 200, method: GET, query: , ip: 172.26.100.4, user-agent:
kube-probe/1.24, errors: , cost: 145.193µs
2023-08-01T19:24:25+08:00 error apisix/ssl.go:120 failed to list
ssl: unexpected status code 404; error message: {"error_msg":"Unsupported
resource type: ssl"}
2023-08-01T19:24:25+08:00 error apisix/cluster.go:232 failed to list
ssl in APISIX: unexpected status code 404; error message:
{"error_msg":"Unsupported resource type: ssl"}
2023-08-01T19:24:27+08:00 error apisix/ssl.go:120 failed to list
ssl: unexpected status code 404; error message: {"error_msg":"Unsupported
resource type: ssl"}
2023-08-01T19:24:27+08:00 error apisix/cluster.go:232 failed to list
ssl in APISIX: unexpected status code 404; error message:
{"error_msg":"Unsupported resource type: ssl"}
2023-08-01T19:24:29+08:00 error apisix/ssl.go:120 failed to list
ssl: unexpected status code 404; error message: {"error_msg":"Unsupported
resource type: ssl"}
2023-08-01T19:24:29+08:00 error apisix/cluster.go:232 failed to list
ssl in APISIX: unexpected status code 404; error message:
{"error_msg":"Unsupported resource type: ssl"}
2023-08-01T19:24:31+08:00 error apisix/ssl.go:120 failed to list
ssl: unexpected status code 404; error message: {"error_msg":"Unsupported
resource type: ssl"}
2023-08-01T19:24:31+08:00 error apisix/cluster.go:232 failed to list
ssl in APISIX: unexpected status code 404; error message:
{"error_msg":"Unsupported resource type: ssl"}
2023-08-01T19:24:31+08:00 error apisix/cluster.go:182 failed to sync
cache {"cost_time": "8.028115228s", "cluster": "default"}
2023-08-01T19:24:31+08:00 error providers/controller.go:388 failed
to wait the default cluster to be ready: unexpected status code 404; error
message: {"error_msg":"Unsupported resource type: ssl"}
2023-08-01T19:24:31+08:00 info apisix/stream_route.go:38
resource stream_routes is disabled
2023-08-01T19:24:31+08:00 info apisix/cluster.go:384 syncing schema
{"cluster": "default"}
2023-08-01T19:24:31+08:00 info apisix/cluster.go:172 syncing cache
{"cluster": "default"}
2023-08-01T19:24:31+08:00 error apisix/plugin.go:46 failed to list
plugins' names: Get
"http://apisix-admin.default.svc.cluster.local:9180/apisix/admin/plugins?all=true":
context canceled
2023-08-01T19:24:31+08:00 error apisix/route.go:118 failed to list
routes: Get
"http://apisix-admin.default.svc.cluster.local:9180/apisix/admin/routes":
context canceled
2023-08-01T19:24:31+08:00 error apisix/cluster.go:404 failed to list
plugin names in APISIX: Get
"http://apisix-admin.default.svc.cluster.local:9180/apisix/admin/plugins?all=true":
context canceled
2023-08-01T19:24:31+08:00 error apisix/cluster.go:222 failed to list
routes in APISIX: Get
"http://apisix-admin.default.svc.cluster.local:9180/apisix/admin/routes":
context canceled
2023-08-01T19:24:31+08:00 error apisix/cluster.go:367 failed to sync
schema: Get
"http://apisix-admin.default.svc.cluster.local:9180/apisix/admin/plugins?all=true":
context canceled
2023-08-01T19:24:31+08:00 error apisix/cluster.go:182 failed to sync
cache {"cost_time": "72.435µs", "cluster": "default"}
2023-08-01T19:24:31+08:00 info providers/controller.go:190
controller now is running as a candidate {"namespace": "ingress-apisix",
"pod": "apisix-ingress-controller-5fb55f45df-lqvdn"}
I0801 19:24:31.973464 1 leaderelection.go:248] attempting to acquire
leader lease ingress-apisix/ingress-apisix-leader...
2023-08-01T19:24:31+08:00 info providers/controller.go:137
LeaderElection {"message": "apisix-ingress-controller-5fb55f45df-lqvdn became
leader", "event_type": "Normal"}
I0801 19:24:31.978263 1 leaderelection.go:258] successfully acquired
lease ingress-apisix/ingress-apisix-leader
2023-08-01T19:24:31+08:00 info providers/controller.go:360
controller tries to leading ... {"namespace": "ingress-apisix", "pod":
"apisix-ingress-controller-5fb55f45df-lqvdn"}
2023-08-01T19:24:31+08:00 info apisix/stream_route.go:38
resource stream_routes is disabled
2023-08-01T19:24:31+08:00 warn apisix/cluster.go:344 waiting cluster
default to ready, it may takes a while
2023-08-01T19:24:31+08:00 info apisix/cluster.go:384 syncing schema
{"cluster": "default"}
2023-08-01T19:24:31+08:00 info apisix/cluster.go:172 syncing cache
{"cluster": "default"}
2023-08-01T19:24:31+08:00 error apisix/ssl.go:120 failed to list
ssl: unexpected status code 404; error message: {"error_msg":"Unsupported
resource type: ssl"}
2023-08-01T19:24:31+08:00 error apisix/cluster.go:232 failed to list
ssl in APISIX: unexpected status code 404; error message:
{"error_msg":"Unsupported resource type: ssl"}
2023-08-01T19:24:33+08:00 info [email protected]/context.go:173 path:
/healthz, status: 200, method: GET, query: , ip: 172.26.100.4, user-agent:
kube-probe/1.24, errors: , cost: 43.301µs
2023-08-01T19:24:33+08:00 info [email protected]/context.go:173 path:
/healthz, status: 200, method: GET, query: , ip: 172.26.100.4, user-agent:
kube-probe/1.24, errors: , cost: 53.528µs
2023-08-01T19:24:33+08:00 error apisix/ssl.go:120 failed to list
ssl: unexpected status code 404; error message: {"error_msg":"Unsupported
resource type: ssl"}
2023-08-01T19:24:33+08:00 error apisix/cluster.go:232 failed to list
ssl in APISIX: unexpected status code 404; error message:
{"error_msg":"Unsupported resource type: ssl"}
2023-08-01T19:24:35+08:00 error apisix/ssl.go:120 failed to list
ssl: unexpected status code 404; error message: {"error_msg":"Unsupported
resource type: ssl"}
2023-08-01T19:24:35+08:00 error apisix/cluster.go:232 failed to list
ssl in APISIX: unexpected status code 404; error message:
{"error_msg":"Unsupported resource type: ssl"}
```
```
[root@k8s-dev-master yunwei]# kubectl logs -f apisix-576f67c7d8-457h4
10.233.74.207 - - [01/Aug/2023:11:27:36 +0000]
apisix-admin.default.svc.cluster.local:9180 "GET
/apisix/admin/schema//plugins/limit-conn HTTP/1.1" 200 978 0.000 "-"
"Go-http-client/1.1" - - - "http://apisix-admin.default.svc.cluster.local:9180";
10.233.74.207 - - [01/Aug/2023:11:27:36 +0000]
apisix-admin.default.svc.cluster.local:9180 "GET
/apisix/admin/schema//plugins/ldap-auth HTTP/1.1" 200 703 0.000 "-"
"Go-http-client/1.1" - - - "http://apisix-admin.default.svc.cluster.local:9180";
10.233.74.207 - - [01/Aug/2023:11:27:36 +0000]
apisix-admin.default.svc.cluster.local:9180 "GET
/apisix/admin/schema//plugins/proxy-cache HTTP/1.1" 200 1539 0.000 "-"
"Go-http-client/1.1" - - - "http://apisix-admin.default.svc.cluster.local:9180";
10.233.74.207 - - [01/Aug/2023:11:27:36 +0000]
apisix-admin.default.svc.cluster.local:9180 "GET
/apisix/admin/schema//plugins/degraphql HTTP/1.1" 200 648 0.000 "-"
"Go-http-client/1.1" - - - "http://apisix-admin.default.svc.cluster.local:9180";
10.233.74.207 - - [01/Aug/2023:11:27:36 +0000]
apisix-admin.default.svc.cluster.local:9180 "GET
/apisix/admin/schema//plugins/splunk-hec-logging HTTP/1.1" 200 1174 0.000 "-"
"Go-http-client/1.1" - - - "http://apisix-admin.default.svc.cluster.local:9180";
10.233.74.207 - - [01/Aug/2023:11:27:36 +0000]
apisix-admin.default.svc.cluster.local:9180 "GET
/apisix/admin/schema//plugins/elasticsearch-logger HTTP/1.1" 200 1472 0.000 "-"
"Go-http-client/1.1" - - - "http://apisix-admin.default.svc.cluster.local:9180";
10.233.74.207 - - [01/Aug/2023:11:27:36 +0000]
apisix-admin.default.svc.cluster.local:9180 "GET
/apisix/admin/schema//plugins/ext-plugin-pre-req HTTP/1.1" 200 678 0.000 "-"
"Go-http-client/1.1" - - - "http://apisix-admin.default.svc.cluster.local:9180";
10.233.74.207 - - [01/Aug/2023:11:27:36 +0000]
apisix-admin.default.svc.cluster.local:9180 "GET
/apisix/admin/schema//plugins/api-breaker HTTP/1.1" 200 1395 0.000 "-"
"Go-http-client/1.1" - - - "http://apisix-admin.default.svc.cluster.local:9180";
10.233.74.207 - - [01/Aug/2023:11:27:36 +0000]
apisix-admin.default.svc.cluster.local:9180 "GET
/apisix/admin/schema//plugins/serverless-pre-function HTTP/1.1" 200 653 0.000
"-" "Go-http-client/1.1" - - -
"http://apisix-admin.default.svc.cluster.local:9180";
10.233.74.207 - - [01/Aug/2023:11:27:36 +0000]
apisix-admin.default.svc.cluster.local:9180 "GET
/apisix/admin/schema//plugins/tcp-logger HTTP/1.1" 200 1093 0.000 "-"
"Go-http-client/1.1" - - - "http://apisix-admin.default.svc.cluster.local:9180";
10.233.74.207 - - [01/Aug/2023:11:27:36 +0000]
apisix-admin.default.svc.cluster.local:9180 "GET
/apisix/admin/schema//plugins/limit-req HTTP/1.1" 200 880 0.000 "-"
"Go-http-client/1.1" - - - "http://apisix-admin.default.svc.cluster.local:9180";
10.233.74.207 - - [01/Aug/2023:11:27:36 +0000]
apisix-admin.default.svc.cluster.local:9180 "GET
/apisix/admin/schema//plugins/request-id HTTP/1.1" 200 859 0.000 "-"
"Go-http-client/1.1" - - - "http://apisix-admin.default.svc.cluster.local:9180";
10.233.74.207 - - [01/Aug/2023:11:27:36 +0000]
apisix-admin.default.svc.cluster.local:9180 "GET
/apisix/admin/schema//plugins/authz-casdoor HTTP/1.1" 200 724 0.000 "-"
"Go-http-client/1.1" - - - "http://apisix-admin.default.svc.cluster.local:9180";
10.233.74.207 - - [01/Aug/2023:11:27:36 +0000]
apisix-admin.default.svc.cluster.local:9180 "GET
/apisix/admin/schema//plugins/datadog HTTP/1.1" 200 834 0.000 "-"
"Go-http-client/1.1" - - - "http://apisix-admin.default.svc.cluster.local:9180";
10.233.74.207 - - [01/Aug/2023:11:27:36 +0000]
apisix-admin.default.svc.cluster.local:9180 "GET
/apisix/admin/schema//plugins/opa HTTP/1.1" 200 1001 0.000 "-"
"Go-http-client/1.1" - - - "http://apisix-admin.default.svc.cluster.local:9180";
10.233.74.207 - - [01/Aug/2023:11:27:38 +0000]
apisix-admin.default.svc.cluster.local:9180 "GET /apisix/admin/routes HTTP/1.1"
200 33 0.002 "-" "Go-http-client/1.1" - - -
"http://apisix-admin.default.svc.cluster.local:9180";
10.233.74.207 - - [01/Aug/2023:11:27:38 +0000]
apisix-admin.default.svc.cluster.local:9180 "GET /apisix/admin/upstreams
HTTP/1.1" 200 33 0.001 "-" "Go-http-client/1.1" - - -
"http://apisix-admin.default.svc.cluster.local:9180";
10.233.74.207 - - [01/Aug/2023:11:27:38 +0000]
apisix-admin.default.svc.cluster.local:9180 "GET /apisix/admin/ssl HTTP/1.1"
404 58 0.000 "-" "Go-http-client/1.1" - - -
"http://apisix-admin.default.svc.cluster.local:9180";
```
```
[root@apisix-dashboard-5486d46f6f-pdwbj logs]# tail -f access.log
2023-08-01T12:06:46.465Z filter/logging.go:45 /ping {"status": 200,
"host": "10.233.74.205:9000", "query": "", "requestId": "", "latency": 0,
"remoteIP": "172.26.100.4", "method": "GET", "errs": []}
2023-08-01T12:06:46.465Z filter/logging.go:45 /ping {"status": 200,
"host": "10.233.74.205:9000", "query": "", "requestId": "", "latency": 0,
"remoteIP": "172.26.100.4", "method": "GET", "errs": []}
2023-08-01T12:06:56.464Z filter/logging.go:45 /ping {"status": 200,
"host": "10.233.74.205:9000", "query": "", "requestId": "", "latency": 0,
"remoteIP": "172.26.100.4", "method": "GET", "errs": []}
2023-08-01T12:06:56.464Z filter/logging.go:45 /ping {"status": 200,
"host": "10.233.74.205:9000", "query": "", "requestId": "", "latency": 0,
"remoteIP": "172.26.100.4", "method": "GET", "errs": []}
2023-08-01T12:07:06.464Z filter/logging.go:45 /ping {"status": 200,
"host": "10.233.74.205:9000", "query": "", "requestId": "", "latency": 0,
"remoteIP": "172.26.100.4", "method": "GET", "errs": []}
2023-08-01T12:07:06.464Z filter/logging.go:45 /ping {"status": 200,
"host": "10.233.74.205:9000", "query": "", "requestId": "", "latency": 0,
"remoteIP": "172.26.100.4", "method": "GET", "errs": []}
2023-08-01T12:07:16.465Z filter/logging.go:45 /ping {"status": 200,
"host": "10.233.74.205:9000", "query": "", "requestId": "", "latency": 0,
"remoteIP": "172.26.100.4", "method": "GET", "errs": []}
2023-08-01T12:07:16.465Z filter/logging.go:45 /ping {"status": 200,
"host": "10.233.74.205:9000", "query": "", "requestId": "", "latency": 0,
"remoteIP": "172.26.100.4", "method": "GET", "errs": []}
2023-08-01T12:07:26.465Z filter/logging.go:45 /ping {"status": 200,
"host": "10.233.74.205:9000", "query": "", "requestId": "", "latency": 0,
"remoteIP": "172.26.100.4", "method": "GET", "errs": []}
2023-08-01T12:07:26.465Z filter/logging.go:45 /ping {"status": 200,
"host": "10.233.74.205:9000", "query": "", "requestId": "", "latency": 0,
"remoteIP": "172.26.100.4", "method": "GET", "errs": []}
```
### Environment
- your apisix-ingress-controller version (output of
apisix-ingress-controller version --long):
1.6.1
- your Kubernetes cluster version (output of kubectl version):
1.24.3
- if you run apisix-ingress-controller in Bare-metal environment, also show
your OS version (uname -a):
Linux k8s-dev-master 3.10.0-1160.88.1.el7.x86_64 #1 SMP Tue Mar 7 15:41:52
UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
