zar3bski opened a new issue, #7920: URL: https://github.com/apache/apisix/issues/7920
### Current Behavior I can't use a [python plugin](https://github.com/apache/apisix-python-plugin-runner) as a `ext-plugin-post-req`. When I deactivate the plugin,the request reaches the upstream service. Here are the processes in the container ``` ps PID USER TIME COMMAND 1 root 0:00 {openresty} nginx: master process /usr/local/openresty/bin/openresty -p /usr/local/apisix -g daemon off; 46 nobody 0:00 {openresty} nginx: worker process ..... 75 nobody 0:00 {openresty} nginx: worker process 76 nobody 0:00 {openresty} nginx: worker process 77 nobody 0:00 {openresty} nginx: worker process 78 nobody 0:00 {openresty} nginx: cache manager process 80 root 0:00 {openresty} nginx: privileged agent process 81 root 0:00 python3 /usr/share/extras/py-apisix/py-runner start ``` It seems that both apisix and my plugin run as root so they should be able to communicate through `/usr/local/apisix/conf/apisix-1.sock`: ``` ls -la /usr/local/apisix/conf/apisix-1.sock srwxr-xr-x 1 root root 0 Sep 14 13:17 /usr/local/apisix/conf/apisix-1.sock ``` I just can't make sense of this behavior ### Expected Behavior Requests should reach the upstream service enriched with the headers defined in the plugin module **apisix-python-plugin-runner/apisix/plugins/rewrite.py** ```python def filter(self, conf: Any, request: Request, response: Response): """ The plugin executes the main function :param conf: plugin configuration after parsing :param request: request parameters and information :param response: response parameters and information :return: """ # print plugin configuration print(conf) # Fetch request nginx variable `host` host = request.get_var("host") print(host) # Fetch request body body = request.get_body() print(body) # Rewrite request headers request.set_header("X-Resp-A6-Runner", "Python") # Rewrite request args request.set_arg("a6_runner", "Python") # Rewrite request path request.set_uri("/a6/python/runner") ``` ### Error Logs ``` 2022/09/14 13:47:01 [crit] 47#47: *795708 connect() to unix:/usr/local/apisix/conf/apisix-1.sock failed (13: Permission denied), client: 192.168.1.139, server: _, request: "GET /mock/ HTTP/1.1", host: "localhost:32674" 2022/09/14 13:47:01 [error] 47#47: *795708 [lua] init.lua:726: phase_func(): failed to connect to the unix socket unix:/usr/local/apisix/conf/apisix-1.sock: permission denied, client: 192.168.1.139, server: _, request: "GET /mock/ HTTP/1.1", host: "localhost:32674" 2022/09/14 13:47:01 [warn] 47#47: *795708 [lua] plugin.lua:759: run_plugin(): ext-plugin-post-req exits with http status code 503, client: 192.168.1.139, server: _, request: "GET /mock/ HTTP/1.1", host: "localhost:32674" ``` ### Steps to Reproduce 1. Deploy the chart with the following values.yaml ```yaml # # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. apisix: # Enable or disable Apache APISIX itself # Set it to false and ingress-controller.enabled=true will deploy only ingress-controller enabled: true admin_key: - name: admin key: edd1c9f034335f136f87ad84b625c8f1 # using fixed API token has security risk, please update it when you deploy to production environment role: admin # Enable nginx IPv6 resolver enableIPv6: true # Use Pod metadata.uid as the APISIX id. setIDFromPodUID: false customLuaSharedDicts: [] # - name: foo # size: 10k # - name: bar # size: 1m luaModuleHook: enabled: false # extend lua_package_path to load third party code luaPath: "" # the hook module which will be used to inject third party code into APISIX # use the lua require style like: "module.say_hello" hookPoint: "" # configmap that stores the codes configMapRef: name: "" # mounts decides how to mount the codes to the container. mounts: - key: "" path: "" enableCustomizedConfig: false customizedConfig: {} image: repository: apisix/apisix #apache/apisix pullPolicy: Always # Overrides the image tag whose default is the chart appVersion. tag: 2.13.3-alpine # Use a `DaemonSet` or `Deployment` kind: Deployment # kind is DaemonSet, replicaCount not become effective replicaCount: 1 podAnnotations: {} podSecurityContext: {} # fsGroup: 2000 securityContext: {} # capabilities: # drop: # - ALL # readOnlyRootFilesystem: true # runAsNonRoot: true # runAsUser: 1000 # See https://kubernetes.io/docs/tasks/run-application/configure-pdb/ for more details podDisruptionBudget: enabled: false minAvailable: 90% maxUnavailable: 1 resources: {} # We usually recommend not to specify default resources and to leave this as a conscious # choice for the user. This also increases chances charts run on environments with little # resources, such as Minikube. If you do want to specify resources, uncomment the following # lines, adjust them as necessary, and remove the curly braces after 'resources:'. # limits: # cpu: 100m # memory: 128Mi # requests: # cpu: 100m # memory: 128Mi hostNetwork: false nodeSelector: {} tolerations: [] affinity: {} # If true, it will sets the anti-affinity of the Pod. podAntiAffinity: enabled: false # timezone is the timezone where apisix uses. # For example: "UTC" or "Asia/Shanghai" # This value will be set on apisix container's environment variable TZ. # You may need to set the timezone to be consistent with your local time zone, # otherwise the apisix's logs may used to retrieve event maybe in wrong timezone. timezone: "" # extraEnvVars An array to add extra env vars # e.g: # extraEnvVars: # - name: FOO # value: "bar" # - name: FOO2 # valueFrom: # secretKeyRef: # name: SECRET_NAME # key: KEY extraEnvVars: [] nameOverride: "" fullnameOverride: "" gateway: type: NodePort # If you want to keep the client source IP, you can set this to Local. # ref: https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/#preserving-the-client-source-ip externalTrafficPolicy: Cluster # type: LoadBalancer # annotations: # service.beta.kubernetes.io/aws-load-balancer-type: nlb externalIPs: [] http: enabled: true servicePort: 80 containerPort: 9080 tls: enabled: false servicePort: 443 containerPort: 9443 existingCASecret: "" certCAFilename: "" http2: enabled: true sslProtocols: "TLSv1.2 TLSv1.3" # L4 proxy (TCP/UDP) stream: enabled: false only: false tcp: [] udp: [] ingress: enabled: false annotations: {} # kubernetes.io/ingress.class: nginx # kubernetes.io/tls-acme: "true" hosts: - host: apisix.local paths: [] tls: [] # - secretName: apisix-tls # hosts: # - chart-example.local admin: # Enable Admin API enabled: true # admin service type type: ClusterIP # loadBalancerIP: a.b.c.d # loadBalancerSourceRanges: # - "143.231.0.0/16" externalIPs: [] # port: 9180 servicePort: 9180 # Admin API support CORS response headers cors: true # Admin API credentials credentials: admin: edd1c9f034335f136f87ad84b625c8f1 viewer: 4054f7cf07e344346cd3f287985e76a2 allow: # The ip range for allowing access to Apache APISIX ipList: - 127.0.0.1/24 nginx: workerRlimitNofile: "20480" workerConnections: "10620" workerProcesses: auto enableCPUAffinity: true # APISIX plugins to be enabled plugins: - api-breaker - authz-keycloak - basic-auth - batch-requests - consumer-restriction - cors - echo - fault-injection - grpc-transcode - hmac-auth - http-logger - ip-restriction - ua-restriction - jwt-auth # - kafka-logger - key-auth - limit-conn - limit-count - limit-req - node-status - openid-connect - authz-casbin - prometheus - proxy-cache - proxy-mirror - proxy-rewrite - redirect - referer-restriction - request-id - request-validation - response-rewrite - serverless-post-function - serverless-pre-function - sls-logger - syslog - tcp-logger - udp-logger - uri-blocker - wolf-rbac - zipkin - traffic-split - gzip - real-ip - ext-plugin-pre-req - ext-plugin-post-req stream_plugins: - mqtt-proxy - ip-restriction - limit-conn pluginAttrs: {} extPlugin: enabled: true cmd: ["python3","/usr/share/extras/py-apisix/py-runner", "start"] # customPlugins allows you to mount your own HTTP plugins. customPlugins: enabled: false # the lua_path that tells APISIX where it can find plugins, # note the last ';' is required. luaPath: "/opts/custom_plugins/?.lua" plugins: # plugin name. - name: "" # plugin attrs attrs: {} # plugin codes can be saved inside configmap object. configMap: # name of configmap. name: "" # since keys in configmap is flat, mountPath allows to define the mount # path, so that plugin codes can be mounted hierarchically. mounts: - key: "" path: "" - key: "" path: "" updateStrategy: {} # type: RollingUpdate extraVolumes: - name: apisix-lib hostPath: path: /home/zar3bski/Documents/Code/octaave/test_files/apisix-python-plugin-runner/apisix - name: apisix-bin hostPath: path: /home/zar3bski/Documents/Code/octaave/test_files/apisix-python-plugin-runner/bin - name: apisix-conf hostPath: path: /home/zar3bski/Documents/Code/octaave/test_files/apisix-python-plugin-runner/conf extraVolumeMounts: - name: apisix-lib mountPath: /usr/lib/python3.9/site-packages/apisix readOnly: true - name: apisix-bin mountPath: /usr/share/extras/py-apisix readOnly: true - name: apisix-conf mountPath: /usr/share/extras/conf readOnly: true discovery: enabled: false registry: # Integration service discovery registry. E.g eureka\dns\nacos\consul_kv # reference: # https://apisix.apache.org/docs/apisix/discovery#configuration-for-eureka # https://apisix.apache.org/docs/apisix/discovery/dns#service-discovery-via-dns # https://apisix.apache.org/docs/apisix/discovery/consul_kv#configuration-for-consul-kv # https://apisix.apache.org/docs/apisix/discovery/nacos#configuration-for-nacos # # an eureka example: # eureka: # host: # - "http://${username}:${password}@${eureka_host1}:${eureka_port1}"; # - "http://${username}:${password}@${eureka_host2}:${eureka_port2}"; # prefix: "/eureka/" # fetch_interval: 30 # weight: 100 # timeout: # connect: 2000 # send: 2000 # read: 5000 # access log and error log configuration logs: enableAccessLog: true accessLog: "/dev/stdout" accessLogFormat: '$remote_addr - $remote_user [$time_local] $http_host \"$request\" $status $body_bytes_sent $request_time \"$http_referer\" \"$http_user_agent\" $upstream_addr $upstream_status $upstream_response_time \"$upstream_scheme://$upstream_host$upstream_uri\"' accessLogFormatEscape: default errorLog: "/dev/stderr" errorLogLevel: "warn" dns: resolvers: - 127.0.0.1 - 172.20.0.10 - 114.114.114.114 - 223.5.5.5 - 1.1.1.1 - 8.8.8.8 validity: 30 timeout: 5 initContainer: image: busybox tag: 1.28 autoscaling: enabled: false minReplicas: 1 maxReplicas: 100 targetCPUUtilizationPercentage: 80 targetMemoryUtilizationPercentage: 80 # Custom configuration snippet. configurationSnippet: main: | httpStart: | httpEnd: | httpSrv: | httpAdmin: | stream: | # Observability configuration. # ref: https://apisix.apache.org/docs/apisix/plugins/prometheus/ serviceMonitor: enabled: false # namespace where the serviceMonitor is deployed, by default, it is the same as the namespace of the apisix namespace: "" # name of the serviceMonitor, by default, it is the same as the apisix fullname name: "" # interval at which metrics should be scraped interval: 15sfalse metricPrefix: apisix_ # container port where the metrics are exposed containerPort: 9091 # @param serviceMonitor.labels ServiceMonitor extra labels labels: {} # @param serviceMonitor.annotations ServiceMonitor annotations annotations: {} # etcd configuration # use the FQDN address or the IP of the etcd etcd: # install etcd(v3) by default, set false if do not want to install etcd(v3) together enabled: true host: # host or ip e.g. http://172.20.128.89:2379 - http://etcd.host:2379 prefix: "/apisix" timeout: 30 # if etcd.enabled is true, set more values of bitnami/etcd helm chart auth: rbac: # No authentication by default create: false user: "" password: "" tls: enabled: false existingSecret: "" certFilename: "" certKeyFilename: "" verify: true sni: "" service: port: 2379 replicaCount: 3 dashboard: enabled: true replicaCount: 1 image: repository: apache/apisix-dashboard pullPolicy: IfNotPresent # Overrides the image tag whose default is the chart appVersion. tag: 2.13-alpine imagePullSecrets: [] nameOverride: "" fullnameOverride: "" serviceAccount: # Specifies whether a service account should be created create: true # Annotations to add to the service account annotations: {} # The name of the service account to use. # If not set and create is true, a name is generated using the fullname template name: "" podAnnotations: {} podSecurityContext: {} # fsGroup: 2000 securityContext: {} # capabilities: # drop: # - ALL # readOnlyRootFilesystem: true # runAsNonRoot: true # runAsUser: 1000 config: conf: listen: host: 0.0.0.0 port: 9000 etcd: # Supports defining multiple etcd host addresses for an etcd cluster endpoints: - apisix-etcd:2379 # apisix configurations prefix prefix: "/apisix" # Etcd basic auth info username: ~ password: ~ log: errorLog: level: warn filePath: /dev/stderr accessLog: filePath: /dev/stdout authentication: secret: secret expireTime: 3600 users: - username: admin password: admin service: type: ClusterIP port: 80 ingress: enabled: false # Kubernetes 1.18+ support ingressClassName attribute className: "" annotations: {} # kubernetes.io/ingress.class: nginx # kubernetes.io/tls-acme: "true" # domain access apisix example: # hosts: # - host: apisix-dashboard.local # paths: # - /* hosts: - host: apisix-dashboard.local paths: [] tls: [] # - secretName: chart-example-tls # hosts: # - chart-example.local resources: {} # We usually recommend not to specify default resources and to leave this as a conscious # choice for the user. This also increases chances charts run on environments with little # resources, such as Minikube. If you do want to specify resources, uncomment the following # lines, adjust them as necessary, and remove the curly braces after 'resources:'. # limits: # cpu: 100m # memory: 128Mi # requests: # cpu: 100m # memory: 128Mi autoscaling: enabled: false minReplicas: 1 maxReplicas: 100 targetCPUUtilizationPercentage: 80 # targetMemoryUtilizationPercentage: 80 nodeSelector: {} tolerations: [] affinity: {} ingress-controller: enabled: false ``` 2. create a route ```json { "uri": "/mock/*", "name": "mock", "methods": [ "GET", "POST", "PUT", "DELETE", "PATCH", "HEAD", "OPTIONS", "CONNECT", "TRACE" ], "plugins": { "ext-plugin-post-req": { "conf": [ { "name": "rewrite", "value": "{\"enable\":\"feature\"}" } ], "disable": false }, "ext-plugin-pre-req": { "disable": true } }, "upstream": { "nodes": [ { "host": "172.17.0.1", "port": 8081, "weight": 1 } ], "timeout": { "connect": 6, "send": 6, "read": 6 }, "type": "roundrobin", "scheme": "http", "pass_host": "pass", "keepalive_pool": { "idle_timeout": 60, "requests": 1000, "size": 320 } }, "status": 1 } ``` 3. query the route `curl http://localhost:${mapped port of apisix gateway}/mock/` ### Environment - APISIX version (run `apisix version`): 2.13.3 - Operating system (run `uname -a`): Linux apisix-6878f46cb5-hps57 5.15.0-47-generic #51-Ubuntu SMP Thu Aug 11 07:51:15 UTC 2022 x86_64 Linux (????? The image is supposed to be Alpine based) - OpenResty / Nginx version (run `openresty -V` or `nginx -V`): 1.21.4.1 - Plugin runner version, for issues related to plugin runners: python plugin v 0.2.0 - LuaRocks version, for installation issues (run `luarocks --version`): -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
