dimberman commented on a change in pull request #10393: URL: https://github.com/apache/airflow/pull/10393#discussion_r489482962
########## File path: UPDATING.md ########## @@ -154,6 +153,480 @@ The Old and New provider configuration keys that have changed are as follows For more information, visit https://flask-appbuilder.readthedocs.io/en/latest/security.html#authentication-oauth +### Changes to the KubernetesExecutor + +#### The KubernetesExecutor Will No Longer Read from the airflow.cfg for Base Pod Configurations + +In Airflow 2.0, the KubernetesExecutor will require a base pod template written in yaml. This file can exist +anywhere on the host machine and will be linked using the `pod_template_file` configuration in the airflow.cfg. + +The `airflow.cfg` will still accept values for the `worker_container_repository`, the `worker_container_tag`, and +the default namespace. + +#### The `executor_config` Will Now Expect a `kubernetes.client.models.V1Pod` Class When Launching Tasks + +In Airflow 1.10.x, users could modify task pods at runtime by passing a dictionary to the `executor_config` variable. +Users will now have full access the Kubernetes API via the `kubernetes.client.models.V1Pod`. + +While in the deprecated version a user would mount a volume using the following dictionary: + +```python +second_task = PythonOperator( + task_id="four_task", + python_callable=test_volume_mount, + executor_config={ + "KubernetesExecutor": { + "volumes": [ + { + "name": "example-kubernetes-test-volume", + "hostPath": {"path": "/tmp/"}, + }, + ], + "volume_mounts": [ + { + "mountPath": "/foo/", + "name": "example-kubernetes-test-volume", + }, + ] + } + } +) +``` + +In the new model a user can accomplish the same thing using the following code: + +```python +from kubernetes.client import models as k8s + +second_task = PythonOperator( + task_id="four_task", + python_callable=test_volume_mount, + executor_config={"KubernetesExecutor": k8s.V1Pod( + spec=k8s.V1PodSpec( + containers=[ + k8s.V1Container( + name="base", + volume_mounts=[ + k8s.V1VolumeMount( + mount_path="/foo/", + name="example-kubernetes-test-volume" + ) + ] + ) + ], + volumes=[ + k8s.V1Volume( + name="example-kubernetes-test-volume", + host_path=k8s.V1HostPathVolumeSource( + path="/tmp/" + ) + ) + ] + ) + ) + } +) +``` +For Airflow 2.0, the traditional `executor_config` will continue operation with a deprecation warning, Review comment: Hi @mik-laj, the deprecation warning was already merged so it didn't show up in the PR https://github.com/apache/airflow/pull/10393/files#diff-868ed785b2a336f20cb6a577dde2502aR198-R201 ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
