I agree, has there been a new release of Python that has worked with Airflow without at least some fixing?
I understand that Airflow is both a library and an application and I do agree with not putting upper bounds on libraries unless required, but it seems like allows an arbitrary upper bound of Python is never going to be practical. If users really need to break this support version and install an older version of Airflow with a newer version of Python it isn't difficult to build Airflow locally with patches, and they will likely need to add more patches to get Airflow to work anyway, so their cost is already built in. Even if just limiting to 3.11 or lower for 2.7.3 or higher I think the benefits are there, Python 3.12 might be supported soon or there might be more hidden issues that mean it isn't supported for 9+ months. Damian -----Original Message----- From: Pierre Jeambrun <pierrejb...@gmail.com> Sent: Monday, October 23, 2023 12:13 PM To: dev@airflow.apache.org Subject: Re: Limiting (or errorring out) Airflow for Python 3.12 until our dependencies/we catch up I think that limiting to <3.12 makes sense. 2.7.2 is already out so I'm not sure we can do anything for users trying to install 2.7.2 on python 3.12. I believe there is no such thing as a python minor that is out of the box working well for airflow. It seems that we always need extra efforts to bring in a new version due to small (or big) breaking change. Maybe limiting python version should be by default (always), until we explicitly move that limit when we integrate a new version. Airflow would not be installable on newest version that were not explicitly tested against. Le lun. 23 oct. 2023 à 14:19, Aritra Basu <aritrabasu1...@gmail.com> a écrit : > I think I'm on the side of giving an error message saying 3.12 is not > yet supported in 2.7.3, I would assume anyone seeing that would > understand the implication that neither does 2.7.2 and thus they > wouldn't try installing it. > > Though I would also think they would have the same understanding that > if > 2.7.3 doesn't > list 3.12 as supported neither would 2.7.2. But I'd rather be > explicitly told 2.7.3 and earlier don't work with 3.12. > > Thanks and Regards, > Aritra Basu > > > On Mon, Oct 23, 2023 at 4:54 PM Jarek Potiuk <ja...@potiuk.com> wrote: > > > Hey everyone, > > > > I've opened a PR https://github.com/apache/airflow/pull/35123 to > > limit Airflow to Python < 3.12 though I am not sure if this is the > > best idea > so I > > seek devlist wisdom to decide whether we should do this, or maybe > something > > else like allowing airflow to be installed but produce a clean error > > indicating 3.12 is not (yet) supported. > > > > Currently Airflow does not work for Python 3.12 - mainly because of > > "distutils" removal https://peps.python.org/pep-0632/ and the way > > how > some > > of the core dependencies - Pendulum for on - and likely some of our > > providers have other breaking dependencies. While Andrey works on > Pendulum > > upgrade as they released version 3 that is compatible, this might > > take > some > > time and we might find out there are other problems as we are not > > running the full test suite for 3.12 due to dependency issues. > > > > It looks like that Python 3.12 is a bit different than 3.7 - 3.11 > releases > > were - this is the first release that is more aggressive when it > > comes to backwards compatibility. There are also some unit test > > methods removal > and > > other change that might make our 3.12 migration a bit more painful > > (more details here for example) > > https://towardsdatascience.com/python3-12-98245ecd6a97 > > > > Currently people attempting to install Airflow with Python 3.12 will > > get unobvious errors (like build errors or runtime errors that are cryptic). > > This is already happening and users are reporting issues that > > "airflow > does > > not work" which boils down to installing Airflow (or breeze) for > > Python 3.12. > > > > I initially thought (hence the PR) that we can limit Airflow to < > > 3.12 > and > > release it in 2.7.3 (and remove the limitation once we get official > > 3.12 support). > > > > So maybe a better solution will be to check the Python version at > > startup time and fail Airflow with clear error. Or maybe we should > > not worry > about > > it at all? > > > > The problem is that if we limit 2.7.3, users installing airflow on > > 3.12 will install 2.7.2 because it has no limit for Python - and > > will end up with the very same experience they have now. > > > > WDYT? > > > > J. > > > ________________________________ Strike Technologies, LLC (“Strike”) is part of the GTS family of companies. Strike is a technology solutions provider, and is not a broker or dealer and does not transact any securities related business directly whatsoever. This communication is the property of Strike and its affiliates, and does not constitute an offer to sell or the solicitation of an offer to buy any security in any jurisdiction. It is intended only for the person to whom it is addressed and may contain information that is privileged, confidential, or otherwise protected from disclosure. Distribution or copying of this communication, or the information contained herein, by anyone other than the intended recipient is prohibited. If you have received this communication in error, please immediately notify Strike at i...@striketechnologies.com, and delete and destroy any copies hereof. ________________________________ CONFIDENTIALITY / PRIVILEGE NOTICE: This transmission and any attachments are intended solely for the addressee. This transmission is covered by the Electronic Communications Privacy Act, 18 U.S.C ''2510-2521. The information contained in this transmission is confidential in nature and protected from further use or disclosure under U.S. Pub. L. 106-102, 113 U.S. Stat. 1338 (1999), and may be subject to attorney-client or other legal privilege. Your use or disclosure of this information for any purpose other than that intended by its transmittal is strictly prohibited, and may subject you to fines and/or penalties under federal and state law. If you are not the intended recipient of this transmission, please DESTROY ALL COPIES RECEIVED and confirm destruction to the sender via return transmittal. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org For additional commands, e-mail: dev-h...@airflow.apache.org
