The vote for "AIP-69: Remote Executor" has passed. 6 Binding +1s: - Kaxil Naik - Jarek Potiuk - Shahar Epstein - Vikram Koka - Niko Oliveira - Jens Scheffler
1 Non-binding +1s: - Buğra Öztürk I have noted two feedbacks regarding the naming. I re-think and propose a vote before finalizaton and would open the discussion for the naming. So far I am missing better convincing alternatives for "Remote Executor" and propose to change documentation to call the "non local executors" being "distributed executors". (see https://github.com/apache/airflow/pull/40900/files#diff-083c36387d854acf8cb3d04c6318aa068cccd2ab856400b4e4f4e988937b468a). Thanks everyone! Jens Scheffler Alliance: Enabler - Tech Lead (XC-AS/EAE-ADA-T) Robert Bosch GmbH | Hessbruehlstraße 21 | 70565 Stuttgart-Vaihingen | GERMANY | http://www.bosch.com/ Tel. +49 711 811-91508 | Mobil +49 160 90417410 | jens.scheff...@de.bosch.com Sitz: Stuttgart, Registergericht: Amtsgericht Stuttgart, HRB 14000; Aufsichtsratsvorsitzender: Prof. Dr. Stefan Asenkerschbaumer; Geschäftsführung: Dr. Stefan Hartung, Dr. Christian Fischer, Dr. Markus Forschner, Stefan Grosch, Dr. Markus Heyn, Dr. Frank Meyer, Dr. Tanja Rückert -----Original Message----- From: Buğra Öztürk <ozturkbugr...@gmail.com> Sent: Tuesday, July 23, 2024 11:58 PM To: dev@airflow.apache.org Subject: Re: [VOTE] (v2) AIP-69 Remote Executor +1 (non-binding) On Tue, 23 Jul 2024, 23:29 Kaxil Naik, <kaxiln...@gmail.com> wrote: > +1 binding > > On Sun, 21 Jul 2024 at 22:11, Jarek Potiuk <ja...@potiuk.com> wrote: > > > +1 (binding) > > > > On Sun, Jul 21, 2024 at 11:06 PM Shahar Epstein <sha...@apache.org> > wrote: > > > > > +1 binding :) > > > The AIP describes the feature well in all aspects, and I'm sure > > > that implementing it will take the executors a couple of steps forward. > > > I do agree with Niko regarding the overloading of "Remote", so > > terminology > > > should be revised. > > > > > > > > > On Wed, Jul 17, 2024 at 8:38 AM Scheffler Jens (XC-AS/EAE-ADA-T) > > > <jens.scheff...@de.bosch.com.invalid> wrote: > > > > > > > Hi Developers, > > > > > > > > After some further discussion time I’d like to call for a vote > > > > for > > > AIP-69. > > > > All details are described in: > > > > > > > > > > https://cwik/ > i.apache.org%2Fconfluence%2Fdisplay%2FAIRFLOW%2FAIP-69%2BRemote%2BExec > utor&data=05%7C02%7CJens.Scheffler%40de.bosch.com%7Caac8b855b1b743e55e > 1908dcab62e3a8%7C0ae51e1907c84e4bbb6d648ee58410f4%7C0%7C0%7C6385736884 > 14330373%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiL > CJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=66ZSeDGJjLR%2BLwMP57j > Hf5Jh5X36samUAU7jaEygsXY%3D&reserved=0 > > > > > > > > Note: > > > > > > > > * Compared to first VOTE in > > > > https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%25 > > > > 2Flists.apache.org%2Fthread%2Ftyfsrpjn12sz9dw50pbg16dsv6lmj610&d > > > > ata=05%7C02%7CJens.Scheffler%40de.bosch.com%7Caac8b855b1b743e55e > > > > 1908dcab62e3a8%7C0ae51e1907c84e4bbb6d648ee58410f4%7C0%7C0%7C6385 > > > > 73688414341854%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQI > > > > joiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=ixL > > > > 3e%2F%2FDKb4o3Ch4EzQsMriY2HY4BQsCWl%2FtXi7uObs%3D&reserved=0 > more > > > > details have been added > > > > * A PoC PR is available in > > > > https://github.com/apache/airflow/pull/40224 > > > > * Status of progress in > > > > > > > > > > > > > > https://gith/ > ub.com%2Fjscheffl%2Fairflow%2Fblob%2Ffeature%2Faip-69-poc%2Fairflow%2F > providers%2Fremote%2FTODO.md&data=05%7C02%7CJens.Scheffler%40de.bosch. > com%7Caac8b855b1b743e55e1908dcab62e3a8%7C0ae51e1907c84e4bbb6d648ee5841 > 0f4%7C0%7C0%7C638573688414354557%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wL > jAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sda > ta=ZdOj68Yf6V2UIEz3FCVDDLtJkufbaG6%2FLG0Wp15%2BiY0%3D&reserved=0 > > > > * Q&A session was hosted, Notes in > > > > https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%25 > > > > 2Flists.apache.org%2Fthread%2Fh2nxkto0lxgjnqj8yps0qsh7ppbccx6g&d > > > > ata=05%7C02%7CJens.Scheffler%40de.bosch.com%7Caac8b855b1b743e55e > > > > 1908dcab62e3a8%7C0ae51e1907c84e4bbb6d648ee58410f4%7C0%7C0%7C6385 > > > > 73688414360812%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQI > > > > joiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=ciT > > > > %2FsBJBjtTz0lvbioJXU4NLn0LTnlZZu7ClsZQPxFE%3D&reserved=0 > > > > > > > > Remote Executor should be a special executor for use cases where > > > > a distributed (non central) setup across different security > > > > perimeters > > need > > > > to > > > > be achieved and a worker accesses the central site only via HTTP(s). > It > > > > will > > > > leverage AIP-61 (Hybrid Execution) as well as builds on-top of > > > > AIP-44 > > (at > > > > least the parts needed for the worker, see PoC PR, it is already > > working > > > > on > > > > existing structures). > > > > Target is to deliver it with Airflow 2.10 as a Pre-Release. > > > > There it > > can > > > > be > > > > experienced/tested and incrementally be improved. It will > > > > integrate > in > > > > Airflow 3 with AIP-72 and replace AIP-44 task communication with > this. > > > > > > > > > > > > From the Q&A meeting main consent was elaborated in a direction of: > > > > > > > > - Remote Executor will be marked experimental, not contained in > default > > > > release in 2.10 line > > > > > > > > - Even if installed, remote endpoint will be disabled by default > > > > to minimize risk of exposure > > > > > > > > - We would release the provider package only with a version > > > > suffix > > "pre0" > > > > to > > > > PyPi such that an user must explicitly install a pre-release > > > > version > as > > > > manual install > > > > > > > > - Support and maintenance in Airflow 2.10++ will end with the > > > > feature being available in Airflow 3 to reduce double > > > > maintenance and as motivation > > to > > > > migrate > > > > > > > > > > > > > > > > Why already in 2.10? With the existing structures in Airflow > > > > 2.10 we > > can > > > > get > > > > started, it is already working with limitations. From there we > > > > can > use > > > it, > > > > learn on a running system and incrementally enhance and improve. > > > > > > > > > > > > > > > > The vote will run for 6 days and last till next Tuesday 23nd of > > > > July > > 2024 > > > > 8:00 UTC. > > > > > > > > > > > > > > > > Everyone is encouraged to vote, although only PMC members and > > Committer's > > > > votes are considered binding. > > > > > > > > > > > > > > > > This is my +1. > > > > > > > > Mit freundlichen Grüßen / Best regards > > > > > > > > Jens Scheffler > > > > > > > > Alliance: Enabler - Tech Lead (XC-AS/EAE-ADA-T) Robert Bosch > > > > GmbH | Hessbruehlstraße 21 | 70565 Stuttgart-Vaihingen | GERMANY > > > > | http://www.bosch.com/ Tel. +49 711 811-91508 | Mobil +49 160 90417410 > > > > | > > > > jens.scheff...@de.bosch.com<mailto:jens.scheff...@de.bosch.com> > > > > > > > > Sitz: Stuttgart, Registergericht: Amtsgericht Stuttgart, HRB > > > > 14000; > > > > Aufsichtsratsvorsitzender: Prof. Dr. Stefan Asenkerschbaumer; > > > > Geschäftsführung: Dr. Stefan Hartung, Dr. Christian Fischer, Dr. > Markus > > > > Forschner, > > > > Stefan Grosch, Dr. Markus Heyn, Dr. Frank Meyer, Dr. Tanja > > > > Rückert > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org For additional commands, e-mail: dev-h...@airflow.apache.org
