yuqian90 opened a new pull request #11922:
URL: https://github.com/apache/airflow/pull/11922
`PythonOperator` accepts `python_callable` in this form, i.e. the callable
can request any number of named arguments in `context`:
```
def python_callable(execution_date, ds_nodash):
...
```
However, `ExternalTaskSensor` accepts only `execution_date_fn` in this form:
```
def execution_date_fn(execution_date):
...
```
A more recent change #8702 added support for this form as well:
```
def execution_date_fn(execution_date, context):
...
```
However #8702 introduced an unpleasant side-effect. Because it enforces the
number of arguments is either 1 or 2, it stops people from using common
techniques such as loop variable capturing. E.g. this comment illustrate a
problem: https://github.com/apache/airflow/pull/8702/files#r497897460
This PR addresses the problem by making `execution_date_fn` work the same
way as `python_callable`. `execution_date_fn` is now much more flexible. It can
accept any variable number of keyword arguments from `context`. The change is
backward compatible. Existing usage continues to work.
E.g. the following callables are all legitimate execution_date_fn after this
PR:
```
def execution_date_fn(dt):
...
def execution_date_fn(execution_date):
...
def execution_date_fn(execution_date, context):
...
def execution_date_fn(execution_date, ds_nodash):
...
def execution_date_fn(execution_date, ds_nodash, dag):
...
```
The callable facility in `PythonOperator` has been refactored into
`airflow.utils.helper.make_kwargs_callable`. So callables in other places can
be made to work the same way if we wish to do that in the future.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
