hamzamazhar opened a new issue, #77137:
URL: https://github.com/apache/airflow/issues/77137

   ### Under which category would you file this issue?
   
   Task SDK
   
   ### Apache Airflow version
   
   3.x.x
   
   ### What happened and how to reproduce it?
   
   ## Issue
   I have a custom operator that inherits from `PythonOperator` and passes 
arguments to its callable through `op_kwargs`.
   
   Following the custom-operator testing pattern in the 
[documentation](https://airflow.apache.org/docs/apache-airflow/stable/best-practices.html),
 I instantiate the operator and call:
   ```operator.execute(context={})```
   
   This works with Airflow `2.11.0`, but fails with Airflow `3.0.0` (also 
tested with `Airflow 3.3.1`) because the callable receives none of its required 
arguments.
   
   Error:
   ```
   TypeError: MyCustomPythonOperator.print_arg() missing 1 required positional 
arguments: 'arg_1'
   ```
   
   Passing a non-empty context makes the same test pass:
   ```operator.execute(context={"test_mode": True})```
   
   Calling the callable directly with the configured arguments also works:
   ```operator.python_callable(**operator.op_kwargs)```
   
   ## How to reproduce
   
   ### Custom Python Operator
   
   ```
   
   # Change the import path for airflow 2.11
   from airflow.providers.standard.operators.python import PythonOperator
   
   class MyCustomPythonOperator(PythonOperator):
       def __init__(
           self,
           task_id: str,
           arg_1: str,
           **kwargs,
       ):
           super().__init__(
               task_id=task_id,
               python_callable=print_arg,
               op_kwargs={
                   "arg_1": arg_1,
               },
               **kwargs,
           )
   
       def print_arg(arg_1):
           print(arg_1)
           return 1
   ```
   
   ### Test file
   
   ```
   from my_custom_operators import MyCustomPythonOperator
   
   def test_my_custom_operator_execute():
       op = MyCustomPythonOperator(task_id="my_custom_operator_task", 
arg_1="abc")
   
       assert op.execute(context={}) == 1
   ```
   
   ## Debugging
   
   I looked at the source code for both [Airflow 
2.11](https://github.com/apache/airflow/blob/2.11.2/airflow/utils/context.py) 
and 
[main](https://github.com/apache/airflow/blob/main/task-sdk/src/airflow/sdk/definitions/context.py)
 and found out a difference in the `context_merge` function for the `execute()` 
method of the operator. In Airflow 3.x, there is an additional check, which 
doesn't exist on Airflow 2.11. I suspect this ignores any args and kwargs if 
the context is empty:
   ```
       if not context:
           context = Context()
   ```
   
   ### What you think should happen instead?
   
   This should work as described in the example [documentation 
here](https://airflow.apache.org/docs/apache-airflow/stable/best-practices.html)
 for Airflow 3 as well:
   ```
   def test_my_custom_operator_execute():
       op = MyCustomOperator(task_id="my_custom_operator_task", 
prefix="s3://bucket/some/prefix")
   
       assert op.execute(context={}) == "expected return value"
   ```
   
   ### Operating System
   
   _No response_
   
   ### Deployment
   
   None
   
   ### Apache Airflow Provider(s)
   
   _No response_
   
   ### Versions of Apache Airflow Providers
   
   _No response_
   
   ### Official Helm Chart version
   
   Not Applicable
   
   ### Kubernetes Version
   
   _No response_
   
   ### Helm Chart configuration
   
   _No response_
   
   ### Docker Image customizations
   
   _No response_
   
   ### Anything else?
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [x] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [x] I agree to follow this project's [Code of 
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

  • [I] PythonOperator.execute(con... hamzamazhar (via GitHub) via commits

Reply via email to