mik-laj commented on a change in pull request #10393: URL: https://github.com/apache/airflow/pull/10393#discussion_r489371534
########## File path: UPDATING.md ########## @@ -154,6 +153,480 @@ The Old and New provider configuration keys that have changed are as follows For more information, visit https://flask-appbuilder.readthedocs.io/en/latest/security.html#authentication-oauth +### Changes to the KubernetesExecutor + +#### The KubernetesExecutor Will No Longer Read from the airflow.cfg for Base Pod Configurations + +In Airflow 2.0, the KubernetesExecutor will require a base pod template written in yaml. This file can exist +anywhere on the host machine and will be linked using the `pod_template_file` configuration in the airflow.cfg. + +The `airflow.cfg` will still accept values for the `worker_container_repository`, the `worker_container_tag`, and +the default namespace. + +#### The `executor_config` Will Now Expect a `kubernetes.client.models.V1Pod` Class When Launching Tasks + +In Airflow 1.10.x, users could modify task pods at runtime by passing a dictionary to the `executor_config` variable. +Users will now have full access the Kubernetes API via the `kubernetes.client.models.V1Pod`. + +While in the deprecated version a user would mount a volume using the following dictionary: + +```python +second_task = PythonOperator( + task_id="four_task", + python_callable=test_volume_mount, + executor_config={ + "KubernetesExecutor": { + "volumes": [ + { + "name": "example-kubernetes-test-volume", + "hostPath": {"path": "/tmp/"}, + }, + ], + "volume_mounts": [ + { + "mountPath": "/foo/", + "name": "example-kubernetes-test-volume", + }, + ] + } + } +) +``` + +In the new model a user can accomplish the same thing using the following code: + +```python +from kubernetes.client import models as k8s + +second_task = PythonOperator( + task_id="four_task", + python_callable=test_volume_mount, + executor_config={"KubernetesExecutor": k8s.V1Pod( + spec=k8s.V1PodSpec( + containers=[ + k8s.V1Container( + name="base", + volume_mounts=[ + k8s.V1VolumeMount( + mount_path="/foo/", + name="example-kubernetes-test-volume" + ) + ] + ) + ], + volumes=[ + k8s.V1Volume( + name="example-kubernetes-test-volume", + host_path=k8s.V1HostPathVolumeSource( + path="/tmp/" + ) + ) + ] + ) + ) + } +) +``` +For Airflow 2.0, the traditional `executor_config` will continue operation with a deprecation warning, Review comment: Do we have this deprecation warning? I didn't see it in the code, but I might have missed it. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
