I am +1 on it. While there are still open discussions on details there, I think the overall general direction is hashed out "good enough". Certainly in the "what we want to achieve" and "what limitations it will bring" but also "what interaction it will have with other AIPs" and we do not have to make all the detailed decisions today. Some of them will be taken during implementation, and some of them can be adapted and changed as we progress.
J. On Thu, Jul 11, 2024 at 1:03 PM Ash Berlin-Taylor <a...@apache.org> wrote: > > I’m calling for a vote on this AIP > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-72+Task+Execution+Interface+aka+Task+SDK > > Previous discussion thread was > https://lists.apache.org/thread/drfmwbx5brtycjkmtf51owrmhvmlx2lk > > Based on the recent discussions I have added the section titled "Dag Parsing > Changes”. > > This AIP (and the key point of removing direct DB access from tasks) will > cause some breaking changes to custom operators, and it’s hard for us to > judge how big an impact that change will have, but this AIP gives us so many > benefits that it is my view that even if we can’t help migrate custom > operators it’s worth it overall. > > A reminder of the main thing this AIP gives us: > > - No more direct DB access from workers — the number of active connections to > the DB is commonly a scaling pain point with larger Airflow deployments > - Improved security due to the above — No more can a task access any > connection it likes, or go and affect the state of other DAGs > - Gives us the AIP that enables Jens’ Remote Worker AIP to operate sensibly > - It will let users _natively_ write tasks in languages — a nicer UX than > just BashOperator. > > And for completness, the downsides as I see them: > - It will (possibly irreparably) break a small number of custom operators > - It is a lot of work (but achievable in the timeframe) > > The vote will run for 5 days and last till next Tuesday 16th of July 2024 > 11:00 UTC. > > Everyone is encouraged to vote, although only PMC members and Committer's > votes are considered binding. > > This is my +1. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org For additional commands, e-mail: dev-h...@airflow.apache.org
