kaxil commented on a change in pull request #10393: URL: https://github.com/apache/airflow/pull/10393#discussion_r488149744
########## File path: UPDATING.md ########## @@ -154,6 +153,480 @@ The Old and New provider configuration keys that have changed are as follows For more information, visit https://flask-appbuilder.readthedocs.io/en/latest/security.html#authentication-oauth +### Changes to the KubernetesExecutor + +#### The KubernetesExecutor Will No Longer Read from the airflow.cfg for Base Pod Configurations + +In Airflow 2.0, the KubernetesExecutor will require a base pod template written in yaml. This file can exist +anywhere on the host machine and will be linked using the `pod_template_file` configuration in the airflow.cfg. + +The airflow.cfg will still accept values for the `worker_container_repository`, the `worker_container_tag`, and Review comment: ```suggestion The `airflow.cfg` will still accept values for the `worker_container_repository`, the `worker_container_tag`, and ``` ########## File path: UPDATING.md ########## @@ -154,6 +153,480 @@ The Old and New provider configuration keys that have changed are as follows For more information, visit https://flask-appbuilder.readthedocs.io/en/latest/security.html#authentication-oauth +### Changes to the KubernetesExecutor + +#### The KubernetesExecutor Will No Longer Read from the airflow.cfg for Base Pod Configurations + +In Airflow 2.0, the KubernetesExecutor will require a base pod template written in yaml. This file can exist +anywhere on the host machine and will be linked using the `pod_template_file` configuration in the airflow.cfg. + +The airflow.cfg will still accept values for the `worker_container_repository`, the `worker_container_tag`, and +the default namespace. + +#### The executor_config Will Now Expect a `kubernetes.client.models.V1Pod` Class When Launching Tasks + +In airflow 1.10, users could modify task pods at runtime by passing a dictionary to the `executor_config` variable. Review comment: ```suggestion In Airflow 1.10.x, users could modify task pods at runtime by passing a dictionary to the `executor_config` variable. ``` ########## File path: UPDATING.md ########## @@ -154,6 +153,480 @@ The Old and New provider configuration keys that have changed are as follows For more information, visit https://flask-appbuilder.readthedocs.io/en/latest/security.html#authentication-oauth +### Changes to the KubernetesExecutor + +#### The KubernetesExecutor Will No Longer Read from the airflow.cfg for Base Pod Configurations + +In Airflow 2.0, the KubernetesExecutor will require a base pod template written in yaml. This file can exist +anywhere on the host machine and will be linked using the `pod_template_file` configuration in the airflow.cfg. + +The airflow.cfg will still accept values for the `worker_container_repository`, the `worker_container_tag`, and +the default namespace. + +#### The executor_config Will Now Expect a `kubernetes.client.models.V1Pod` Class When Launching Tasks + +In airflow 1.10, users could modify task pods at runtime by passing a dictionary to the `executor_config` variable. +Users will now have full access the Kubernetes API via the `kubernetes.client.models.V1Pod`. + +While in the deprecated version a user would mount a volume using the following dictionary: + +```python +second_task = PythonOperator( + task_id="four_task", + python_callable=test_volume_mount, + executor_config={ + "KubernetesExecutor": { + "volumes": [ + { + "name": "example-kubernetes-test-volume", + "hostPath": {"path": "/tmp/"}, + }, + ], + "volume_mounts": [ + { + "mountPath": "/foo/", + "name": "example-kubernetes-test-volume", + }, + ] + } + } +) +``` + +In the new model a user can accomplish the same thing using the following code: + +```python +from kubernetes.client import models as k8s + +second_task = PythonOperator( + task_id="four_task", + python_callable=test_volume_mount, + executor_config={"KubernetesExecutor": k8s.V1Pod( + spec=k8s.V1PodSpec( + containers=[ + k8s.V1Container( + name="base", + volume_mounts=[ + k8s.V1VolumeMount( + mount_path="/foo/", + name="example-kubernetes-test-volume" + ) + ] + ) + ], + volumes=[ + k8s.V1Volume( + name="example-kubernetes-test-volume", + host_path=k8s.V1HostPathVolumeSource( + path="/tmp/" + ) + ) + ] + ) + ) + } +) +``` +For Airflow 2.0, the traditional `executor_config` will continue operation with a deprecation warning, +but will be removed in a future version. + +### Changes to the KubernetesPodOperator + +Much like the KubernetesExecutor, the KubernetesPodOperator will no longer take Airflow custom classes and will Review comment: ```suggestion Much like the `KubernetesExecutor`, the `KubernetesPodOperator` will no longer take Airflow custom classes and will ``` ########## File path: UPDATING.md ########## @@ -154,6 +153,480 @@ The Old and New provider configuration keys that have changed are as follows For more information, visit https://flask-appbuilder.readthedocs.io/en/latest/security.html#authentication-oauth +### Changes to the KubernetesExecutor + +#### The KubernetesExecutor Will No Longer Read from the airflow.cfg for Base Pod Configurations + +In Airflow 2.0, the KubernetesExecutor will require a base pod template written in yaml. This file can exist +anywhere on the host machine and will be linked using the `pod_template_file` configuration in the airflow.cfg. + +The airflow.cfg will still accept values for the `worker_container_repository`, the `worker_container_tag`, and +the default namespace. + +#### The executor_config Will Now Expect a `kubernetes.client.models.V1Pod` Class When Launching Tasks Review comment: ```suggestion #### The `executor_config` Will Now Expect a `kubernetes.client.models.V1Pod` Class When Launching Tasks ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
