akkik04 opened a new pull request, #59292:
URL: https://github.com/apache/airflow/pull/59292

   
   ---
   Closes #56459
   
   ## Problem
   
   `@task.kubernetes_cmd` breaks when used with TaskFlow-style mapping inside a 
TaskGroup.
   The root cause is that we didn't mark TaskFlow arguments (op_args / 
op_kwargs) as template_fields, thus the validation of the return value of the 
Python callable occurs before those fields are resolved by the TaskFlow/mapping 
machinery.
   
   As a result, when `group_task.expand(...)` is used, the name parameter that 
reaches the task (`echo_cmd` is the task being used in the repro linked within 
the issue) is still a `MappedArgument` object, and the type check inside 
`_generate_cmds` sees a non-`str` element and raises the `TypeError: Expected 
echo_cmd to return a list of strings, but got ['echo', MappedArgument(...)]`.
   
   ## Solution
   
   I aligned `@task.kubernetes_cmd` with `@task.kubernetes` with regards to how 
templated fields are handled, so TaskFlow mappings behave consistently. 
Concretely, I extended `template_fields` to include `op_args` and `op_kwargs` 
alongside the existing Kubernetes-specific fields inherited from 
`KubernetesPodOperator.template_fields`. This ensures that TaskFlow 
arguments—including mapping metadata such as `MappedArgument` coming from 
`TaskGroup.expand(...)` will be run through the normal TaskFlow resolution 
scheme and reach the user function as plain Python values instead of unresolved 
mapping objects.
   
   Finally, I add a focused unit test, 
`test_kubernetes_cmd_template_fields_include_taskflow_args`, which creates a 
simple `@task.kubernetes_cmd` TaskFlow function inside `self.dag_maker`, 
instantiates it, and asserts that the underlying operator’s `template_fields` 
include at least `"op_args"` and `"op_kwargs"`. This guards against future 
refactors accidentally dropping TaskFlow arguments from `template_fields` and 
reintroducing the original `MappedArgument` type error for 
`TaskGroup.expand(...)` with `@task.kubernetes_cmd`.
   
   Open to discussion!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to