Re: [VOTE] Remove experimental API
For me, this is one of the experimental features that we can remove at any time according to our release process. For the users who are using it, I don't think they are using a recent version of Airflow because this API has been deprecated since 2.0.0 and we haven't added any features or fixes to it in over three years. +1 to remove it. But since we already have some -1 binding votes, I like to move it to a separate provider as an alternative solution. On Sun, Mar 17, 2024 at 3:34 AM Vikram Koka wrote: > -1 > > As much as I would like to see this removed, I feel the same way as Jed > above. > > In response to the question raised regarding "Experimental features", the > reason why this one seems different is because though this was marked as > "experimental", it was the only way to interact with Airflow before the > full fledged REST API and therefore a lot of users had baked this > experimental API into their automation processes. > > > On Sat, Mar 16, 2024 at 5:37 PM Daniel Imberman > > wrote: > > > As everyone above mentioned. I’m all for removing it but we should do so > as > > part of a major breaking release. Perhaps if we haven’t already we should > > at least add deprecation warnings? > > > > -1 but very down to add deprecation warnings > > > > On Sat, Mar 16, 2024 at 4:19 PM Bas Harenslak > > > wrote: > > > > > -1 for me too. > > > > > > Regardless of how we treat the “experimental” status, I often still see > > > people using the experimental API for triggering DAGs. IMO it would be > > too > > > much of a breaking change to remove it in a minor version, so I suggest > > > removing it in Airflow 3. > > > > > > Bas > > > > > > > Op 16 mrt 2024 om 14:24 heeft Andrey Anshin < > andrey.ans...@taragol.is> > > > het volgende geschreven: > > > > > > > > Asked because if it never was an experimental feature, then it can't > > be > > > > just removed until Airflow 3 which might never happen. > > > > In this case the vote should be canceled, and we need to continue to > > > > discuss moving it to a separate provider and suspend/remove the newly > > > > created provider. > > > > > > > > > > > > > > > >> On Sun, 17 Mar 2024 at 00:02, Andrey Anshin < > andrey.ans...@taragol.is > > > > > > >> wrote: > > > >> I just wonder if `Experimental` is covered by > > > >> > > > > > > https://airflow.apache.org/docs/apache-airflow/stable/release-process.html#experimental-features > > > >> ? > > > >> Or is it just another meaning of Experimental ? > > > >>> On Sat, 16 Mar 2024 at 23:39, Jarek Potiuk > wrote: > > > >>> Would you still vote `-1` of course was the question. > > > On Sat, Mar 16, 2024 at 8:37 PM Jarek Potiuk > > > wrote: > > > Question: Jed, Ash: Would you still vote If we move it to provider > > > (with > > > status "removed from maintenance except security fixes" - same as > we > > > did > > > with daskexecutor? > > > J. > > > On Sat, Mar 16, 2024 at 8:25 PM Ash Berlin-Taylor > > > > >>> wrote: > > > > As much as I would love to remove it I'm with Jed: if it worked > on > > > 2.0 > > > >>> it > > > > should work on all 2.x > > > > My vote is -1 > > > > On 16 March 2024 19:08:13 GMT, Jed Cunningham > > > >>> > > > > wrote: > > > >> I forgot to add the "why" - I view this as a breaking change > > still. > > > > > > - > > > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > > > For additional commands, e-mail: dev-h...@airflow.apache.org > > > > > > > > >
Re: [VOTE] Remove experimental API
-1 As much as I would like to see this removed, I feel the same way as Jed above. In response to the question raised regarding "Experimental features", the reason why this one seems different is because though this was marked as "experimental", it was the only way to interact with Airflow before the full fledged REST API and therefore a lot of users had baked this experimental API into their automation processes. On Sat, Mar 16, 2024 at 5:37 PM Daniel Imberman wrote: > As everyone above mentioned. I’m all for removing it but we should do so as > part of a major breaking release. Perhaps if we haven’t already we should > at least add deprecation warnings? > > -1 but very down to add deprecation warnings > > On Sat, Mar 16, 2024 at 4:19 PM Bas Harenslak > wrote: > > > -1 for me too. > > > > Regardless of how we treat the “experimental” status, I often still see > > people using the experimental API for triggering DAGs. IMO it would be > too > > much of a breaking change to remove it in a minor version, so I suggest > > removing it in Airflow 3. > > > > Bas > > > > > Op 16 mrt 2024 om 14:24 heeft Andrey Anshin > > het volgende geschreven: > > > > > > Asked because if it never was an experimental feature, then it can't > be > > > just removed until Airflow 3 which might never happen. > > > In this case the vote should be canceled, and we need to continue to > > > discuss moving it to a separate provider and suspend/remove the newly > > > created provider. > > > > > > > > > > > >> On Sun, 17 Mar 2024 at 00:02, Andrey Anshin > > > >> wrote: > > >> I just wonder if `Experimental` is covered by > > >> > > > https://airflow.apache.org/docs/apache-airflow/stable/release-process.html#experimental-features > > >> ? > > >> Or is it just another meaning of Experimental ? > > >>> On Sat, 16 Mar 2024 at 23:39, Jarek Potiuk wrote: > > >>> Would you still vote `-1` of course was the question. > > On Sat, Mar 16, 2024 at 8:37 PM Jarek Potiuk > > wrote: > > Question: Jed, Ash: Would you still vote If we move it to provider > > (with > > status "removed from maintenance except security fixes" - same as we > > did > > with daskexecutor? > > J. > > On Sat, Mar 16, 2024 at 8:25 PM Ash Berlin-Taylor > > >>> wrote: > > > As much as I would love to remove it I'm with Jed: if it worked on > > 2.0 > > >>> it > > > should work on all 2.x > > > My vote is -1 > > > On 16 March 2024 19:08:13 GMT, Jed Cunningham > > >>> > > > wrote: > > >> I forgot to add the "why" - I view this as a breaking change > still. > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > > For additional commands, e-mail: dev-h...@airflow.apache.org > > > > >
Re: [VOTE] Remove experimental API
As everyone above mentioned. I’m all for removing it but we should do so as part of a major breaking release. Perhaps if we haven’t already we should at least add deprecation warnings? -1 but very down to add deprecation warnings On Sat, Mar 16, 2024 at 4:19 PM Bas Harenslak wrote: > -1 for me too. > > Regardless of how we treat the “experimental” status, I often still see > people using the experimental API for triggering DAGs. IMO it would be too > much of a breaking change to remove it in a minor version, so I suggest > removing it in Airflow 3. > > Bas > > > Op 16 mrt 2024 om 14:24 heeft Andrey Anshin > het volgende geschreven: > > > > Asked because if it never was an experimental feature, then it can't be > > just removed until Airflow 3 which might never happen. > > In this case the vote should be canceled, and we need to continue to > > discuss moving it to a separate provider and suspend/remove the newly > > created provider. > > > > > > > >> On Sun, 17 Mar 2024 at 00:02, Andrey Anshin > >> wrote: > >> I just wonder if `Experimental` is covered by > >> > https://airflow.apache.org/docs/apache-airflow/stable/release-process.html#experimental-features > >> ? > >> Or is it just another meaning of Experimental ? > >>> On Sat, 16 Mar 2024 at 23:39, Jarek Potiuk wrote: > >>> Would you still vote `-1` of course was the question. > On Sat, Mar 16, 2024 at 8:37 PM Jarek Potiuk > wrote: > Question: Jed, Ash: Would you still vote If we move it to provider > (with > status "removed from maintenance except security fixes" - same as we > did > with daskexecutor? > J. > On Sat, Mar 16, 2024 at 8:25 PM Ash Berlin-Taylor > >>> wrote: > > As much as I would love to remove it I'm with Jed: if it worked on > 2.0 > >>> it > > should work on all 2.x > > My vote is -1 > > On 16 March 2024 19:08:13 GMT, Jed Cunningham > >>> > > wrote: > >> I forgot to add the "why" - I view this as a breaking change still. > > - > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > For additional commands, e-mail: dev-h...@airflow.apache.org > >
Re: [VOTE] Remove experimental API
-1 for me too. Regardless of how we treat the “experimental” status, I often still see people using the experimental API for triggering DAGs. IMO it would be too much of a breaking change to remove it in a minor version, so I suggest removing it in Airflow 3. Bas > Op 16 mrt 2024 om 14:24 heeft Andrey Anshin het > volgende geschreven: > > Asked because if it never was an experimental feature, then it can't be > just removed until Airflow 3 which might never happen. > In this case the vote should be canceled, and we need to continue to > discuss moving it to a separate provider and suspend/remove the newly > created provider. > > > >> On Sun, 17 Mar 2024 at 00:02, Andrey Anshin >> wrote: >> I just wonder if `Experimental` is covered by >> https://airflow.apache.org/docs/apache-airflow/stable/release-process.html#experimental-features >> ? >> Or is it just another meaning of Experimental ? >>> On Sat, 16 Mar 2024 at 23:39, Jarek Potiuk wrote: >>> Would you still vote `-1` of course was the question. On Sat, Mar 16, 2024 at 8:37 PM Jarek Potiuk wrote: Question: Jed, Ash: Would you still vote If we move it to provider (with status "removed from maintenance except security fixes" - same as we did with daskexecutor? J. On Sat, Mar 16, 2024 at 8:25 PM Ash Berlin-Taylor >>> wrote: > As much as I would love to remove it I'm with Jed: if it worked on 2.0 >>> it > should work on all 2.x > My vote is -1 > On 16 March 2024 19:08:13 GMT, Jed Cunningham >>> > wrote: >> I forgot to add the "why" - I view this as a breaking change still. - To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org For additional commands, e-mail: dev-h...@airflow.apache.org
Re: [VOTE] Remove experimental API
Asked because if it never was an experimental feature, then it can't be just removed until Airflow 3 which might never happen. In this case the vote should be canceled, and we need to continue to discuss moving it to a separate provider and suspend/remove the newly created provider. On Sun, 17 Mar 2024 at 00:02, Andrey Anshin wrote: > I just wonder if `Experimental` is covered by > https://airflow.apache.org/docs/apache-airflow/stable/release-process.html#experimental-features > ? > Or is it just another meaning of Experimental ? > > > > > On Sat, 16 Mar 2024 at 23:39, Jarek Potiuk wrote: > >> Would you still vote `-1` of course was the question. >> >> On Sat, Mar 16, 2024 at 8:37 PM Jarek Potiuk wrote: >> >> > Question: Jed, Ash: Would you still vote If we move it to provider (with >> > status "removed from maintenance except security fixes" - same as we did >> > with daskexecutor? >> > >> > J. >> > >> > On Sat, Mar 16, 2024 at 8:25 PM Ash Berlin-Taylor >> wrote: >> > >> >> As much as I would love to remove it I'm with Jed: if it worked on 2.0 >> it >> >> should work on all 2.x >> >> >> >> My vote is -1 >> >> >> >> On 16 March 2024 19:08:13 GMT, Jed Cunningham >> >> >> wrote: >> >> >I forgot to add the "why" - I view this as a breaking change still. >> >> >> > >> >
Re: [VOTE] Remove experimental API
I just wonder if `Experimental` is covered by https://airflow.apache.org/docs/apache-airflow/stable/release-process.html#experimental-features ? Or is it just another meaning of Experimental ? On Sat, 16 Mar 2024 at 23:39, Jarek Potiuk wrote: > Would you still vote `-1` of course was the question. > > On Sat, Mar 16, 2024 at 8:37 PM Jarek Potiuk wrote: > > > Question: Jed, Ash: Would you still vote If we move it to provider (with > > status "removed from maintenance except security fixes" - same as we did > > with daskexecutor? > > > > J. > > > > On Sat, Mar 16, 2024 at 8:25 PM Ash Berlin-Taylor > wrote: > > > >> As much as I would love to remove it I'm with Jed: if it worked on 2.0 > it > >> should work on all 2.x > >> > >> My vote is -1 > >> > >> On 16 March 2024 19:08:13 GMT, Jed Cunningham > > >> wrote: > >> >I forgot to add the "why" - I view this as a breaking change still. > >> > > >
Re: [VOTE] Remove experimental API
Would you still vote `-1` of course was the question. On Sat, Mar 16, 2024 at 8:37 PM Jarek Potiuk wrote: > Question: Jed, Ash: Would you still vote If we move it to provider (with > status "removed from maintenance except security fixes" - same as we did > with daskexecutor? > > J. > > On Sat, Mar 16, 2024 at 8:25 PM Ash Berlin-Taylor wrote: > >> As much as I would love to remove it I'm with Jed: if it worked on 2.0 it >> should work on all 2.x >> >> My vote is -1 >> >> On 16 March 2024 19:08:13 GMT, Jed Cunningham >> wrote: >> >I forgot to add the "why" - I view this as a breaking change still. >> >
Re: [VOTE] Remove experimental API
Question: Jed, Ash: Would you still vote If we move it to provider (with status "removed from maintenance except security fixes" - same as we did with daskexecutor? J. On Sat, Mar 16, 2024 at 8:25 PM Ash Berlin-Taylor wrote: > As much as I would love to remove it I'm with Jed: if it worked on 2.0 it > should work on all 2.x > > My vote is -1 > > On 16 March 2024 19:08:13 GMT, Jed Cunningham > wrote: > >I forgot to add the "why" - I view this as a breaking change still. >
Re: [VOTE] Remove experimental API
As much as I would love to remove it I'm with Jed: if it worked on 2.0 it should work on all 2.x My vote is -1 On 16 March 2024 19:08:13 GMT, Jed Cunningham wrote: >I forgot to add the "why" - I view this as a breaking change still.
Re: [VOTE] Remove experimental API
I forgot to add the "why" - I view this as a breaking change still.
Re: [VOTE] Remove experimental API
-1. Even though it's been deprecated for a really long time now, I don't think we should remove it in a minor 2 release. I think we should wait until the next major.
Re: [VOTE] Remove experimental API
+1 (binding) On Sat, Mar 16, 2024 at 1:14 PM Andrey Anshin wrote: > Greetings everyone! > > I would like to start a vote proces about removal of Experimental API > support into the next minor Airflow version, presumably 2.9, but it could > be postponed to 2.10. > > By default experimental REST API turned off, and we recommend to use stable > REST API: > > https://airflow.apache.org/docs/apache-airflow/stable/deprecated-rest-api-ref.html > > Discussion about deprecate and remove support of Experimental API started > in 1.10 > - Dev List: > https://lists.apache.org/thread/jdz9l7bsnsw5c3t27dxfrx5pd4wvjlxt > - Github Issues: https://github.com/apache/airflow/issues/10552 > > And recently: > - https://lists.apache.org/thread/khl7gvzpcv3kn99zc441wb9m2dyz4gp9 > > The vote will last until 13:00 GMT/UTC on March 22, 2024, and until at > least 3 binding votes have been cast. > > Please vote accordingly: > > [ ] + 1 approve > [ ] + 0 no opinion > [ ] - 1 disapprove with the reason > > Only votes from PMC members and committers are binding, but other members > of the community are encouraged to check the AIP and vote with > "(non-binding)". >
[VOTE] Remove experimental API
Greetings everyone! I would like to start a vote proces about removal of Experimental API support into the next minor Airflow version, presumably 2.9, but it could be postponed to 2.10. By default experimental REST API turned off, and we recommend to use stable REST API: https://airflow.apache.org/docs/apache-airflow/stable/deprecated-rest-api-ref.html Discussion about deprecate and remove support of Experimental API started in 1.10 - Dev List: https://lists.apache.org/thread/jdz9l7bsnsw5c3t27dxfrx5pd4wvjlxt - Github Issues: https://github.com/apache/airflow/issues/10552 And recently: - https://lists.apache.org/thread/khl7gvzpcv3kn99zc441wb9m2dyz4gp9 The vote will last until 13:00 GMT/UTC on March 22, 2024, and until at least 3 binding votes have been cast. Please vote accordingly: [ ] + 1 approve [ ] + 0 no opinion [ ] - 1 disapprove with the reason Only votes from PMC members and committers are binding, but other members of the community are encouraged to check the AIP and vote with "(non-binding)".
Re: [DISCUSS] Deprecate cli options in Airflow Configurations and airflow.api.client
Oh.. I've totally forgotten about this discussion.
We have some time before Airflow 2.9, so there is a chance that we could
remove it into 2.9.
Let me create a vote.
On Tue, 20 Feb 2024 at 23:10, Pierre Jeambrun wrote:
> +1, stable API should be favoured
>
> On Tue 20 Feb 2024 at 16:10, Jarek Potiuk wrote:
>
> > Oh God . What an archeology. Yeah I think we had a limite when we
> discussed
> > it then (looking at the discussion) we did not have a solid replacement
> yet
> > (and the discussion was about deprecating it in 1.10). Since then we
> marked
> > it as experimental and left it in 2.0.
> > But since then - we have - and it's well established and proven (REST
> > API + Python Client) which very much serves the same purpose that the
> > "remote" experimental CLI should serve (and I think that was the main
> > problem that it did not exist back then).
> >
> > I'd say it calls for removal and significant note ("If you were still
> using
> > it, please use the REST API and Python Client instead").
> >
> > On Tue, Feb 20, 2024 at 11:19 AM Andrey Anshin >
> > wrote:
> >
> > > BTW, I've found that this this top already discussed in the past, like
> > more
> > > that 3 years ago
> > >
> > > Github Issue https://github.com/apache/airflow/issues/10552
> (accidently
> > > found when set sorting on issues as Least recently updated)
> > > Dev List:
> > https://lists.apache.org/thread/jdz9l7bsnsw5c3t27dxfrx5pd4wvjlxt
> > >
> > >
> > >
> > > On Tue, 20 Feb 2024 at 00:14, Andrey Anshin
> > > wrote:
> > >
> > > > Just for clarification, the proposal is about depreciation or even
> > > removal
> > > > of something obsolete. If we could do some improvement at the same
> > moment
> > > > it would be nice, if not also nice, that mean no additional work
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Mon, 19 Feb 2024 at 22:55, Scheffler Jens (XC-AS/EAE-ADA-T)
> > > > wrote:
> > > >
> > > >> I do not habe a strong opinion on the topic but I am mostly with
> > Jarek.
> > > >>
> > > >> The API must be multi tenant and mask all internal DB access - where
> > we
> > > >> are close-by with AIP-44 and the other efforts of multi tenancy
> > > (missing a
> > > >> list of all planned efforts, would be cool to have a mata AIP
> keeping
> > > the
> > > >> completed and planned individual efforts)
> > > >>
> > > >> But in contrast I see the CLI only partly being a tool on tenant but
> > > most
> > > >> parts on the internal perimeter admin side. Might be a bit „gray
> > zone“.
> > > If
> > > >> we habe overlap to REST API (do we have a list of rhings like
> trigger
> > > sag
> > > >> or so?) we should remove and consolidate redundancy to API logic.
> But
> > > there
> > > >> is for sure some „black magic“ which must not be migrated and needs
> > > native
> > > >> DB access. E.g. resetting DB or triggering DB migration. For REST
> API
> > as
> > > >> well as internal API you need a web endpoint incl. minimal
> > > authentication.
> > > >> Some base admin tooling is needed for bootstrap and base admin. Like
> > > >> creating the first admin user is impossible if no user.
> > > >>
> > > >> So before a lengthy discussion I‘d propose to document details and
> > make
> > > >> an AIP to discuss, that might be easier then a reply-to-all thred to
> > > define
> > > >> the scope of cleanup. Also and especially for API cöient we need to
> > > check
> > > >> for impact if breaking changes appear.
> > > >>
> > > >> Jens
> > > >>
> > > >> Sent from Outlook for iOS
> > > >>
> > > >> From: Andrey Anshin
> > > >> Sent: Monday, February 19, 2024 3:54 PM
> > > >> To: dev@airflow.apache.org
> > > >> Subject: Re: [DISCUSS] Deprecate cli options in Airflow
> Configurations
> > > >> and airflow.api.client
> > > >>
> > > >> >Actually - it's not at all part of AIP-44 :). Airflow CLI was
> > > >> (consciously and deliberately) not part of it.
> > > >>
> > > >> Then better to check whether or not it accidentally uses it, add to
> my
> > > >> checklist
> > > >>
> > > >> > So I would be very much for removing that whole part - even in
> > Airflow
> > > >> 2.9.
> > > >>
> > > >> +1 for removal if it won't break our promises
> > > >>
> > > >>
> > > >>
> > > >>
> > > >> On Mon, 19 Feb 2024 at 16:01, Jarek Potiuk
> wrote:
> > > >>
> > > >> > Actually - it's not at all part of AIP-44 :). Airflow CLI was
> > > >> (consciously
> > > >> > and deliberately) not part of it.
> > > >> >
> > > >> > CLI is specifically treated as a "local" tool that is executed
> > inside
> > > >> the
> > > >> > security perimeter where direct database access is needed (i.e. it
> > > >> falls in
> > > >> > the same category as 'scheduler', 'webserver`, `internal-api`
> > > component
> > > >> -
> > > >> > all of which stil has direct DB access and AIP-44 does not change
> > > that.
> > > >> > AIP-44 is specifically about Worker, Triggerrer and Dag File
> > Processor
> > > >> > direct DB access only.
> > > >> >
> > > >> > I
