zkl94 opened a new issue, #11166: URL: https://github.com/apache/apisix/issues/11166
### Current Behavior it is not able to send request logs to google cloud logging now due to this bug. ### Expected Behavior The logs should be sent to google cloud logging successfully ### Error Logs 2024/04/18 19:03:44 [error] 55#55: *67159 [lua] oauth.lua:62: refresh_access_token(): failed to refresh google oauth access token, 20: unable to get local issuer certificate, context: ngx.timer, client: 10.186.156.41, server: 0.0.0.0:9443 2024/04/18 19:03:44 [error] 55#55: *67159 [lua] batch-processor.lua:95: Batch Processor[google-cloud-logging] failed to process entries: failed to get google oauth token, context: ngx.timer, client: 10.186.156.41, server: 0.0.0.0:9443 ### Steps to Reproduce 1. run apisix helm chart https://artifacthub.io/packages/helm/apisix/apisix on k8s 1.27 on GKE 2. create the below ApisixGlobalRule CR ```yaml apiVersion: apisix.apache.org/v2 kind: ApisixGlobalRule metadata: name: google-cloud-logging spec: plugins: - name: google-cloud-logging enable: true config: auth_config: client_email: "xxxxxx" private_key_id: "xxxxx" client_id: "xxxxx" private_key: "xxxxx" project_id: "xxxxx" token_uri: "https://oauth2.googleapis.com/token"; entries_uri: "https://logging.googleapis.com/v2/entries:write"; auth_uri: "https://accounts.google.com/o/oauth2/auth"; ssl_verify: false batch_max_size: 1 retry_delay: 1 inactive_timeout: 10 max_retry_count: 10 ``` 3. just in case it is not applied globally, I added the google-cloud-logging section to ApisixRoute as well: ``` apiVersion: apisix.apache.org/v2 kind: ApisixRoute metadata: name: apisix-dashboard spec: http: - name: dashboard match: hosts: - xxxx.example.com paths: - /* backends: - serviceName: apisix-dashboard servicePort: 80 plugins: - name: redirect enable: true config: http_to_https: true - name: google-cloud-logging enable: true config: auth_config: client_email: "xxxxx" private_key_id: "xxxx" client_id: "xxx" private_key: "xxxx" project_id: "xxxxx" token_uri: "https://oauth2.googleapis.com/token"; entries_uri: "https://logging.googleapis.com/v2/entries:write"; auth_uri: "https://accounts.google.com/o/oauth2/auth"; ssl_verify: false batch_max_size: 1 retry_delay: 1 inactive_timeout: 10 max_retry_count: 10 ``` 4. visit xxxx.example.com on browser multiple times 5. check the logs of apisix pods inside kubernetes and we get the error logs: ``` 2024/04/18 19:03:53 [error] 55#55: *67552 [lua] batch-processor.lua:95: Batch Processor[google-cloud-logging] failed to process entries: failed to get google oauth token, context: ngx.timer, client: 10.186.156.41, server: 0.0.0.0:9443 2024/04/18 19:03:53 [error] 55#55: *67552 [lua] batch-processor.lua:104: Batch Processor[google-cloud-logging] exceeded the max_retry_count[11] dropping the entries, context: ngx.timer, client: 10.186.156.41, server: 0.0.0.0:9443 2024/04/18 19:03:53 [error] 55#55: *67554 [lua] oauth.lua:62: refresh_access_token(): failed to refresh google oauth access token, 20: unable to get local issuer certificate, context: ngx.timer, client: 10.186.156.41, server: 0.0.0.0:9443 2024/04/18 19:03:53 [error] 55#55: *67554 [lua] batch-processor.lua:95: Batch Processor[google-cloud-logging] failed to process entries: failed to get google oauth token, context: ngx.timer, client: 10.186.156.41, server: 0.0.0.0:9443 ``` 6. I checked google-cloud-logging as well, the request logs are not received ### Environment - APISIX version (run `apisix version`): 3.8.0 - Operating system (run `uname -a`): (I run it inside the apisix pod) Linux apisix-6bc8f67474-mk5xj 5.15.133+ SMP Wed Nov 8 17:30:28 UTC 2023 x86_64 GNU/Linux - OpenResty / Nginx version (run `openresty -V` or `nginx -V`): nginx version: openresty/1.21.4.2 - etcd version, if relevant (run `curl http://127.0.0.1:9090/v1/server_info`): - APISIX Dashboard version, if relevant: - Plugin runner version, for issues related to plugin runners: - LuaRocks version, for installation issues (run `luarocks --version`): -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
