Unfortunately not - once release is done in PyPI we cannot do anything (except yanking the release which means "mark it as a buggy release so that it does not get installed by default". PyPI releases are "write-once-only" - basically immutable. When you delete a package, you cannot upload a new one (and this is a good idea for security and stability/reproducibility reasons).
So the most we can do - and I will make a PR for that is to write a warning about Python 3.12 in the upcoming 2.7.3 . I thought a bit about it and it is the "best" we can do - just print a clear warning Python 3.12 is not supported. I think the error is too much, while limiting 2.7.3 when 2.7.2 does not have that limit might be misleading, warning seems to be a good compromise. J. On Thu, Oct 26, 2023 at 10:15 AM Amogh Desai <amoghdesai....@gmail.com> wrote: > Hi Jarek, > > Thanks for this initiative. It is always a good idea to let the end users > know that "oh the software does not work with your current set of > dependencies, do ---- to handle it" > > I like the idea of limiting the versions to < 3.12 and am for it. While we > do this, we should not spend a lot of time and effort on this, because this > might be > a temporary workaround. > > Is there anything that we can do about 2.7.2 which is already released? > Might be too late in the game. We might have to clearly document this so > that users > don't have this experience at all. > > Thanks, > Amogh Desai > > On Wed, Oct 25, 2023 at 8:17 PM Damian Shaw <ds...@striketechnologies.com> > wrote: > > > > But it seems to be changing now. Also I think (by listening to some > > podcast) - Python community starts to be overwhelmed with keeping all the > > backwards-compatibility and they are changing their approach for future > > releases to be more aggressive in removals, separate out certain "core" > > features to outside-libraries and removing them out of the core Python > > > > I appreciate this is a little off topic, but I follow core dev of CPython > > closely and would just like to add I don't think this take is quite > > accurate. IMO Python core dev are focused on Python backwards > compatibility > > in general but say a few things happened: > > > > 1. Functions that have been marked as deprecated for many releases of > > Python are finally being cleaned up > > 2. A group of modules which are completely unmaintained got marked as > > deprecated in PEP 594 (the Steering Council made clear no similar PEPs > > would be accepted and each module will be treated on a case-by-case > basis) > > 3. Several large projects are working on the underlying implementation of > > CPython and they are more willing to propose larger changes to the C-API > > than has happened in the past > > > > There has been discussion implementing an official "soft depreciation", > > which will avoid the eventual removal of a function but no longer support > > any development: > > > https://discuss.python.org/t/formalize-the-concept-of-soft-deprecation-dont-schedule-removal-in-pep-387-backwards-compatibility-policy/27957 > > > > Damian > > > > -----Original Message----- > > From: Jarek Potiuk <ja...@potiuk.com> > > Sent: Wednesday, October 25, 2023 6:15 AM > > To: dev@airflow.apache.org > > Subject: Re: Limiting (or errorring out) Airflow for Python 3.12 until > our > > dependencies/we catch up > > > > > I agree, has there been a new release of Python that has worked with > > Airflow without at least some fixing? > > > > Generally yes. Most of the past bumps were mostly test fixing. I can't > > recall any "serious" changes in the code. Maybe in some single providers. > > Generally Python 3.7 -> 3.11 were extremely backwards compatible, there > > were really maybe 2 cases I remember where we had compatibility issues in > > airflow "core" and they were because of security fixes (some edge cases > of > > url parsing changed Is one that I remember) - and that was even in > > patchlevel releases - not even in the minor release. > > > > > I understand that Airflow is both a library and an application and I > > > do > > agree with not putting upper bounds on libraries unless required, but it > > seems like allowing an arbitrary upper bound of Python is never going to > be > > practical. If users really need to break this support version and install > > an older version of Airflow with a newer version of Python it isn't > > difficult to build Airflow locally with patches, and they will likely > need > > to add more patches to get Airflow to work anyway, so their cost is > already > > built in. > > > > To be honest, It's been practical for now, this is the first time it has > > happened for quite a while. But it seems to be changing now. Also I think > > (by listening to some podcast) - Python community starts to be > overwhelmed > > with keeping all the backwards-compatibility and they are changing their > > approach for future releases to be more aggressive in removals, separate > > out certain "core" features to outside-libraries and removing them out of > > the core Python (which is actually quite an inspiration for us in what we > > do by moving stuff - like executors, hopefully FAB in the future and > maybe > > few others from core to providers), so we can expect more of such > breakages > > in the moment. Having explicit upper-binding will be a good idea. > > > > I think we can do a hybrid solution. We can upper-bind for 2.8 but when > we > > release 2.7.3 we can implement raising error if someone wants to use > Python > > 3.12. I think that will be the best approach. > > > > J. > > > > > > > > On Mon, Oct 23, 2023 at 6:33 PM Damian Shaw < > ds...@striketechnologies.com> > > wrote: > > > > > I agree, has there been a new release of Python that has worked with > > > Airflow without at least some fixing? > > > > > > I understand that Airflow is both a library and an application and I > > > do agree with not putting upper bounds on libraries unless required, > > > but it seems like allows an arbitrary upper bound of Python is never > > > going to be practical. If users really need to break this support > > > version and install an older version of Airflow with a newer version > > > of Python it isn't difficult to build Airflow locally with patches, > > > and they will likely need to add more patches to get Airflow to work > > > anyway, so their cost is already built in. > > > > > > Even if just limiting to 3.11 or lower for 2.7.3 or higher I think the > > > benefits are there, Python 3.12 might be supported soon or there might > > > be more hidden issues that mean it isn't supported for 9+ months. > > > > > > Damian > > > > > > -----Original Message----- > > > From: Pierre Jeambrun <pierrejb...@gmail.com> > > > Sent: Monday, October 23, 2023 12:13 PM > > > To: dev@airflow.apache.org > > > Subject: Re: Limiting (or errorring out) Airflow for Python 3.12 until > > > our dependencies/we catch up > > > > > > I think that limiting to <3.12 makes sense. 2.7.2 is already out so > > > I'm not sure we can do anything for users trying to install 2.7.2 on > > > python 3.12. > > > > > > I believe there is no such thing as a python minor that is out of the > > > box working well for airflow. It seems that we always need extra > > > efforts to bring in a new version due to small (or big) breaking > > > change. Maybe limiting python version should be by default (always), > > > until we explicitly move that limit when we integrate a new version. > > > Airflow would not be installable on newest version that were not > > explicitly tested against. > > > > > > Le lun. 23 oct. 2023 à 14:19, Aritra Basu <aritrabasu1...@gmail.com> a > > > écrit : > > > > > > > I think I'm on the side of giving an error message saying 3.12 is > > > > not yet supported in 2.7.3, I would assume anyone seeing that would > > > > understand the implication that neither does 2.7.2 and thus they > > > > wouldn't try installing it. > > > > > > > > Though I would also think they would have the same understanding > > > > that if > > > > 2.7.3 doesn't > > > > list 3.12 as supported neither would 2.7.2. But I'd rather be > > > > explicitly told 2.7.3 and earlier don't work with 3.12. > > > > > > > > Thanks and Regards, > > > > Aritra Basu > > > > > > > > > > > > On Mon, Oct 23, 2023 at 4:54 PM Jarek Potiuk <ja...@potiuk.com> > wrote: > > > > > > > > > Hey everyone, > > > > > > > > > > I've opened a PR https://github.com/apache/airflow/pull/35123 to > > > > > limit Airflow to Python < 3.12 though I am not sure if this is the > > > > > best idea > > > > so I > > > > > seek devlist wisdom to decide whether we should do this, or maybe > > > > something > > > > > else like allowing airflow to be installed but produce a clean > > > > > error indicating 3.12 is not (yet) supported. > > > > > > > > > > Currently Airflow does not work for Python 3.12 - mainly because > > > > > of "distutils" removal https://peps.python.org/pep-0632/ and the > > > > > way how > > > > some > > > > > of the core dependencies - Pendulum for on - and likely some of > > > > > our providers have other breaking dependencies. While Andrey works > > > > > on > > > > Pendulum > > > > > upgrade as they released version 3 that is compatible, this might > > > > > take > > > > some > > > > > time and we might find out there are other problems as we are not > > > > > running the full test suite for 3.12 due to dependency issues. > > > > > > > > > > It looks like that Python 3.12 is a bit different than 3.7 - 3.11 > > > > releases > > > > > were - this is the first release that is more aggressive when it > > > > > comes to backwards compatibility. There are also some unit test > > > > > methods removal > > > > and > > > > > other change that might make our 3.12 migration a bit more > > > > > painful (more details here for example) > > > > > https://towardsdatascience.com/python3-12-98245ecd6a97 > > > > > > > > > > Currently people attempting to install Airflow with Python 3.12 > > > > > will get unobvious errors (like build errors or runtime errors > > > > > that are > > > cryptic). > > > > > This is already happening and users are reporting issues that > > > > > "airflow > > > > does > > > > > not work" which boils down to installing Airflow (or breeze) for > > > > > Python 3.12. > > > > > > > > > > I initially thought (hence the PR) that we can limit Airflow to < > > > > > 3.12 > > > > and > > > > > release it in 2.7.3 (and remove the limitation once we get > > > > > official > > > > > 3.12 support). > > > > > > > > > > So maybe a better solution will be to check the Python version at > > > > > startup time and fail Airflow with clear error. Or maybe we should > > > > > not worry > > > > about > > > > > it at all? > > > > > > > > > > The problem is that if we limit 2.7.3, users installing airflow on > > > > > 3.12 will install 2.7.2 because it has no limit for Python - and > > > > > will end up with the very same experience they have now. > > > > > > > > > > WDYT? > > > > > > > > > > J. > > > > > > > > > > > > ________________________________ > > > Strike Technologies, LLC (“Strike”) is part of the GTS family of > > > companies. Strike is a technology solutions provider, and is not a > > > broker or dealer and does not transact any securities related business > > > directly whatsoever. This communication is the property of Strike and > > > its affiliates, and does not constitute an offer to sell or the > > > solicitation of an offer to buy any security in any jurisdiction. It > > > is intended only for the person to whom it is addressed and may > > > contain information that is privileged, confidential, or otherwise > > protected from disclosure. > > > Distribution or copying of this communication, or the information > > > contained herein, by anyone other than the intended recipient is > > > prohibited. If you have received this communication in error, please > > > immediately notify Strike at i...@striketechnologies.com, and delete > > and destroy any copies hereof. > > > ________________________________ > > > > > > CONFIDENTIALITY / PRIVILEGE NOTICE: This transmission and any > > > attachments are intended solely for the addressee. This transmission > > > is covered by the Electronic Communications Privacy Act, 18 U.S.C > > > ''2510-2521. The information contained in this transmission is > > > confidential in nature and protected from further use or disclosure > > > under U.S. Pub. L. 106-102, 113 U.S. Stat. 1338 (1999), and may be > > > subject to attorney-client or other legal privilege. Your use or > > > disclosure of this information for any purpose other than that > > > intended by its transmittal is strictly prohibited, and may subject > > > you to fines and/or penalties under federal and state law. If you are > > > not the intended recipient of this transmission, please DESTROY ALL > > > COPIES RECEIVED and confirm destruction to the sender via return > > transmittal. > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > > > For additional commands, e-mail: dev-h...@airflow.apache.org > > > > > ________________________________ > > Strike Technologies, LLC (“Strike”) is part of the GTS family of > > companies. Strike is a technology solutions provider, and is not a broker > > or dealer and does not transact any securities related business directly > > whatsoever. This communication is the property of Strike and its > > affiliates, and does not constitute an offer to sell or the solicitation > of > > an offer to buy any security in any jurisdiction. It is intended only for > > the person to whom it is addressed and may contain information that is > > privileged, confidential, or otherwise protected from disclosure. > > Distribution or copying of this communication, or the information > contained > > herein, by anyone other than the intended recipient is prohibited. If you > > have received this communication in error, please immediately notify > Strike > > at i...@striketechnologies.com, and delete and destroy any copies > hereof. > > ________________________________ > > > > CONFIDENTIALITY / PRIVILEGE NOTICE: This transmission and any attachments > > are intended solely for the addressee. This transmission is covered by > the > > Electronic Communications Privacy Act, 18 U.S.C ''2510-2521. The > > information contained in this transmission is confidential in nature and > > protected from further use or disclosure under U.S. Pub. L. 106-102, 113 > > U.S. Stat. 1338 (1999), and may be subject to attorney-client or other > > legal privilege. Your use or disclosure of this information for any > purpose > > other than that intended by its transmittal is strictly prohibited, and > may > > subject you to fines and/or penalties under federal and state law. If you > > are not the intended recipient of this transmission, please DESTROY ALL > > COPIES RECEIVED and confirm destruction to the sender via return > > transmittal. > > >
