o-nikolas commented on PR #27559:
URL: https://github.com/apache/airflow/pull/27559#issuecomment-1310768362
Very interesting indeed!
Personally, I think the main benefits that operators provide are:
1. Simplification, e.g.:
- Defaulting SDK params where possible so that API exposed to the caller
is simpler
- Specific error and response handling to clarify results received from
the AWS API, which are often obtuse
2. Unification of one or more SDK calls into a single common recipe
encapsulated in an Operator.
3. Abstraction, the user doesn't necessarily need to be familiar with the
Boto3 SDK (or even what an SDK is) to get some dags up and running (think of
non-sde users here).
If we make the abstraction layer as thin and generic as proposed here I
think you lose many of these benefits of Operators. If users have to go look at
the SDK/API to find the method names and the parameters and also provide their
own result handler, etc I think it's actually easier and more straight forward
for them to just use TaskFlow or Custom Operators and the Boto3 API directly
(retrieved from the base hook if they'd like to have that generate the
connection for them, which I'd recommend).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
