potiuk commented on code in PR #25012:
URL: https://github.com/apache/airflow/pull/25012#discussion_r919815471
##########
docs/apache-airflow/tutorial.rst:
##########
@@ -102,11 +102,28 @@ define a ``schedule_interval`` of 1 day for the DAG.
:start-after: [START instantiate_dag]
:end-before: [END instantiate_dag]
+Operators
+---------
+
+An operator defines a unit of work for Airflow to complete. They are the most
basic building blocks of DAGs.
+
+All operators inherit from the BaseOperator, which includes all of the
required arguments for
Review Comment:
I also do not agree they are most basic building blocks. Hooks IMHO are much
more of that.
We should treat thius chapter (which is important to add I agree, not only
as describing the current (or rather past approach) but also to start passing a
message to our users about where we want to steer their way of developing DAGs
I think (and hope) we will manage to transiton out from using Operators "by
default" but using more Task Flow where Operator concept does not exist and is
replaced by a much more flexible and volatile "task". There are a few things
missing for that (lineage for hooks for one) to be able to say "taskf flow and
hoos are the BEST way", but I think eventually we will get there, so I would
not like to stress that Operators are "most basic" in the official
documentation. I would like rather to gently steer away from using operators
and converting to use most task flow approach - and initially combine it with
operators but finally fully transition to it.
What I would like to mention here, is to make sure that we:
* mention that operators are "classic" approach but using Task Flow and
Hooks is more flexible and in many cases simpler and that they can be combined
with task-flow based tasks.
* I think we should mention that Operators are important but not necessary
to build DAGs. there are a way of doing it, but "most basic building block" is
not a message we want to pass.
* I think it's worth mentioning here that more modern and more Pythonic way
is task flow and link to that (while mentioning that "classic" operators are
still ok in a number of cases..
Those are the words (and message) I think we should pass:
* Operators are 'classic" and they are heading into direction of "legacy"
* TaskFlow is more "modern" and "flexible" and "more Pythonic"
* We can interoperate between the "classic" and "modern" approach in a
single DAG and generally using more taskflow where it makes sense is advised.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
