potiuk commented on PR #27264: URL: https://github.com/apache/airflow/pull/27264#issuecomment-1292298781
For those who are watching I made the CI images build. This is the first step that is needed. PyArrow was (of course) tip of the iceberg. But it FAR better than it was with Python 3.10 the day Python 3.10 was released (there almost all dependencies - including some core depednencies had problems). I think (let's see the tests) there are no problems with any of the dependencies that Airlfow Core has. Most of the problems come from our providers, which makes it viable to skip some providers from workng on 3.11- and we could release airflow without those providers working on 3.11. I had to disable quite a number of providers (8) - and big ones (Google, Snowflake, Azure among them). I also implemented a complete mechanism that will allow us in the future to decide whether we want to release Airflow with 3.11 even if some providers will not support it (basically excluding the providers in all the places/extras they were using. I opened (or commmented in existing ones) quite a number of issues in relevant dependencies that are blocking - some of them (like PyArrow) are quite close (they were working on their PR the whole day today). Some other might be tricky and we might not get it at all (for example cloudera "owns" but does not really support any changes in "sasl" library which is required for Hive Provider. They were in a process of handing the sasl library to some individual who voluntered to maintain it but I think it stallled so we might want to drop Hive (and possibly some other providers from 3.11 support). For Google Provider I belive we are the furthest. The only option to have 3.11 support will be to bump the old, pre-2.0.0 google-cloud libraries the Google Provider uses. This is loooong overdue and it will hold us from releaseing Google Provider for 3.11 if not done. And we will be able to release 3.11 without Google Provider support for 3.11, but that woudl be a huge bummer IMHO. Now - the next will be tests failing in this PR, Apart of those tests that are relevant to the excluded providers, I am not going to look at them yet. And there might be other issues. And mybe fixing some dependencies will reveal some other issues. I am subscribed to the issues and will update and rebase the PR as the dependencies get updated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
