turbaszek commented on a change in pull request #8805:
URL: https://github.com/apache/airflow/pull/8805#discussion_r422651378
##########
File path: airflow/models/baseoperator.py
##########
@@ -60,9 +60,25 @@
ScheduleInterval = Union[str, timedelta, relativedelta]
+class BaseOperatorMeta(type):
+ """
+ Base metaclass of BaseOperator.
+ """
+ def __call__(cls, *args, **kwargs):
+ """
+ Called when you call BaseOperator(). In this way we are able to
perform an action
+ after initializing an operator no matter where the
``super().__init__`` is called
+ (before or after assign of new attributes in a custom operator).
+ """
+ obj = type.__call__(cls, *args, **kwargs)
+ # Set upstream task defined by XComArgs passed to template fields of
an operator
+ obj._set_xcomargs_dependencies() # pylint: disable=protected-access
Review comment:
> 2. we can just finalize Operator implementation when we add it to dag
( thus no metaclass is required)
User can either provide the `dag` argument to operator or if it's `None`
then we call `DagContext.get_current_dag()` . And setting DAG (via
`@dag.setter`) is done during `__init__` call, so I'm not sure if I understand
when the "add to dag" moment happens.
Additionally, I think we will have to introduce this metaclass now or later.
Here's @mik-laj PR where he tried to address the `@apply_defaults`:
https://github.com/apache/airflow/pull/7450
One of the reasons to do that (from [devlist
thread](https://lists.apache.org/thread.html/r4cf535e5dec1fbf8330f3c86031519092ca4820b382d3738413b23cd%40%3Cdev.airflow.apache.org%3E)):
> I saw that PR today and it made me realize I've not been using
apply_defaults in much of my code! Look forward to this being an automaic
operation via a MetaClass, great work :)
However, this point is very valid:
> meta_class introduce backward compatibility problems for users who already
have metaclass defined for their operator ( they will have to inherit from
BaseOperatorMeta)
I suppose that we can add some refactor step once we have 2.0 and
https://github.com/apache/airflow/issues/8765 .
Of course we can try to add this in `DagBag.bag_dag` (like policy):
https://github.com/apache/airflow/blob/e1cc17e84856bd121724c634cb1809581c9c97db/airflow/models/dagbag.py#L322-L323
or in `DAG.create_dagrun` but this in my opinion is not the best approach.
Happy to discuss @evgenyshulman @potiuk @kaxil
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
