karenbraganz commented on issue #46714: URL: https://github.com/apache/airflow/issues/46714#issuecomment-2738019101
You are experiencing this issue because the `executor_config` property for mapped tasks does not have a defined setter. This has been reported before in [#24547](https://github.com/apache/airflow/issues/24547) and [#37013](https://github.com/apache/airflow/issues/37013). PRs [#28313](https://github.com/apache/airflow/pull/28313) and [#37828](https://github.com/apache/airflow/pull/37828) fixed this issue for some properties by defining setters for them. However, the `executor_config` property does not have a setter defined yet. I also experienced this issue when I was trying to implement a task policy for `executor_config` and was able to work around this by explicitly overriding `executor_config` for the `MappedOperator` and `DecoratedMappedOperator`in `partial_kwargs`. ``` from airflow.policies import hookimpl from kubernetes.client import models as k8s @hookimpl def task_policy(task: "BaseOperator"): ke_pod_default = { "pod_override": k8s.V1Pod( spec=k8s.V1PodSpec( containers=[ k8s.V1Container( name="base", resources=k8s.V1ResourceRequirements( requests={"cpu": 1, "memory": "2048Mi", "ephemeral-storage": "1Gi"}, limits={"cpu": 1, "memory": "2048Mi", "ephemeral-storage": "1Gi"} ) ) ] ) ) } if task.__class__.__name__ in ["MappedOperator", "DecoratedMappedOperator"]: if task.partial_kwargs["executor_config"] == {}: task.partial_kwargs["executor_config"] = ke_pod_default else: if task.executor_config == {}: task.executor_config = ke_pod_default ``` I recommend implementing this workaround until we are able to add a setter for the `executor_config` property. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
