Owen-CH-Leung commented on PR #33306: URL: https://github.com/apache/airflow/pull/33306#issuecomment-1694285270
@potiuk @eladkal Apology - I just tested locally using breeze with a custom `KubernetesPodOperator` dag and with istio 1.18.2 enabled. It's still not killing the istio-proxy. I've figured out the fix though (just small code change). Will be creating PR in 1 or 2 hours. Setting up local airflow env to run KPO with istio enabled isn't so straight forward, so I'll document the steps that I've done here for record also. (FYI I'm using the docker-desktop which comes with k8s cluster and kubectl client) 1. use breeze to start up airflow as usual (`breeze start-airflow --backend postgres --load-example-dags --db-reset`) 2. run the command `kubectl config view --minify --flatten -o json --context=docker-desktop`. This should output a long json format 3. Copy the json format output that you got before, and in airflow UI, under the `Admin/Connections` tab, you should see a `kubernetes_default` row. Click edit record 4. You should see a row `Kube Config (JSON format)`. Paste your json output into that row and saved. 5. The example dags that breeze creates didn't have any DAG that uses a KPO. So you'll have to create your own DAG. Feel free to use mine. [my_k8s_dag.txt](https://github.com/apache/airflow/files/12445241/my_k8s_dag.txt) 6. rename this file into a `.py` file and put it under `files/dags` in your local airflow directory 7. Restart your airflow (schedule, trigger, webserver...everything). Now you should see your DAG appear in the airflow UI 8. Download istio. (follow this [link](https://istio.io/latest/docs/setup/getting-started/#download) for detailed instructions). Make sure `istioctl` is accessible 9. Don't forget to run `kubectl label namespace airflow istio-injection=enabled` to enable `istio-proxy` sidecar injection in the namespace 10. You are now ready to run the custom KPO Dag =) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
