Thanks for the input everyone. I have updated the docs and changelog, and put the exact error message in the docs (both the new Taskflow API docs and the Changelog of the provider) . Hopefully Google indexes it and if users have the error message when trying to use the task.decorator - I think that should be "good enough".
The docs updated here: https://github.com/apache/airflow/pull/18613 J. On Thu, Sep 30, 2021 at 12:32 PM Jarek Potiuk <[email protected]> wrote: > > > On Thu, Sep 30, 2021 at 9:56 AM Kamil Breguła <[email protected]> wrote: > >> If we release this package, the user will not be able to test it in >> any released version as such a version does not exist yet. This could >> also potentially mean that it will never work, as unreleased versions >> of Airflow may have breaking API changes. We only maintain API >> compatibility between released versions of Airflow. Airflow developer >> versions may have breaking changes, and that's fine as long as this >> issue is fixed before a new version is released. >> >> > We will not bump the version now (as task.decorator is not silently > falling - I just double checked it). > > But even if we would - we should release it before with >= 2.2. It's quite > the opposite from what you say. Users who are currently testing beta and > RCS of 2.2 will be able to test both RC AND released versions of Docker > provider only when we prepare and release and make it ready for them to > test the RC. > It's by NOT releasing it earlier we do not give the users to test it > before 2.2 is released. By releasing it now, we give a chance to test it > with current beta, later rcs and if the users will find a problem we can > fix it in the future patchlevel of the provider. > > >> czw., 30 wrz 2021 o 08:32 Kaxil Naik <[email protected]> napisał(a): >> > >> > Agree with Kamil here. We should just adjust the versioning of the >> docker provider and bump the Minor version to indicate New Feature. >> > >> > To XD's question - we can release a patch version in that minor version >> series for Docker Provider if we find a critical fix. >> > >> > Regards, >> > Kaxil >> > >> > On Thu, Sep 30, 2021, 04:03 Jarek Potiuk <[email protected]> wrote: >> >>> >> >>> >> >>> I have no strong opinion on this, but would like to have >> clarification on one question: I understand you highlighted there is no >> changes nor fixes in the provider, but what if in the future we need to fix >> something for the “classic” operator which may still be used in Airflow 2.1 >> or even lower version installations? How will releases etc. be managed in >> such cases? >> >> >> >> >> >> Yep. Good question. This is about the only "potential" issue - >> currently we have no process for that as we've never had to do it. However >> it is always possible to do it if and when we need it. If we find that >> there is a critical fix for Docker Provider that we would need to fix, we >> can always branch-off from the last released version that works for 2.1 and >> cherry-pick and release a patched version. >> >> >> >> We did not see a need for that so far. We released all providers when >> we released 2.1.0 which are 2.1+ only (because of apply_defaults) and so >> far there was not a case where anyone raised a need to release any bugfix >> for previous versions. With our SemVer approach anyone should be "SAFE" >> migrating to newer version of Airflow (even 2.2) as they are all supposed >> to be backwards compatible, so I think our approach should be "If you need >> anything that we release for the DockerProvider in the future - migrate to >> 2.2. But this is a "default" approach I think and I see the case of a >> really critical issue that will result in releasing a patchlevel version >> for the previous release. >> >> >> >> Again - we have not done that yet, and we want to avoid it for as long >> as we can, but there is nothing (except the overhead on cherry-picking and >> releasing it) to prevent us from doing so. We have all history, tags, >> changelogs and mechanisms to selectively release only one provider if >> needed. But I think it should be really Ad-Hoc and when we REALLY REALLY >> need it. >> >> >> >> J. >> >> >> >>> I may have misunderstood or missed something, and be asking a dumb >> question. Please correct me in that case. >> >>> >> >>> Thanks! >> >>> >> >>> >> >>> XD >> >>> >> >>> On Wed, Sep 29, 2021 at 22:15 Jarek Potiuk <[email protected]> wrote: >> >>>> >> >>>> Hello Everyone, >> >>>> >> >>>> I am just about to release the September Providers, but Kamil raised >> a valid concern and he convinced me that it might be a problem if we >> release a new Docker Provider with minimum airflow version 2.1. >> >>>> >> >>>> The new provider has the new "Docker Decorator" feature which will >> be available only in 2.2. But other than that - it is backwards compatible >> - so you could install it on Airflow 2.1 and have the "classic" operator >> work. But if you try to use the decorator, you will be able to write a Dag >> with this decorator, but it simply won't work (and you will have no message >> about it). >> >>>> >> >>>> The Docker Provider has no other changes nor fixes. >> >>>> >> >>>> I am tempted (and quite convinced) to increase the min airflow >> version of the Docker Provider to 2.2 so that it can be only installed >> there (and we make it ready for the 2.2 release). >> >>>> >> >>>> Does anyone have a strong opinion on that ? >> >>>> >> >>>> J. >> >>>> >> >>>> >> >>>> >> >
