erikogan opened a new issue, #41106: URL: https://github.com/apache/airflow/issues/41106
### Apache Airflow version Other Airflow 2 version (please specify below) ### If "Other Airflow 2 version" selected, which one? 2.9.2 ### What happened? I am investigating the viability of MWAA for a project. I’m using [aws-mwaa-local-runner](https://github.com/aws/aws-mwaa-local-runner)‘s `mwaa-local-env test-requirements`. I’ve generated the `requirements.txt` from a Poetry lock file using the `poetry export` plugin. However, the Airflow 2.9.2 / Python 3.11 [constraints file](https://raw.githubusercontent.com/apache/airflow/constraints-2.9.2/constraints-3.11.txt) specifies conflicting versions of some of my requirements: * `snowflake-connector-python==3.10.1`, which requires: * ~[`urllib3` < 2.0.0](https://github.com/snowflakedb/snowflake-connector-python/blob/v3.10.1/setup.cfg#57)~ * [`keyring` < 25.0](https://github.com/snowflakedb/snowflake-connector-python/blob/v3.10.1/setup.cfg#L99) * `urllib3==2.2.1` * `keyring==25.2.1` (Updating to `snowflake-connector-python==3.12.0` would solve the `keyring` constraint, ~but it still requires `urllib3` < 2.0.0~) I tried removing both `keyring` and `urllib3` from my `requirements.txt` after generation, which got me past the resolution while downloading dependencies, but the resolver still catches me during the actual install phase: ``` ERROR: Cannot install snowflake-connector-python[secure-local-storage]==3.10.1 because these package versions have conflicting dependencies. The conflict is caused by: snowflake-connector-python[secure-local-storage] 3.10.1 depends on keyring<25.0.0 and >=23.1.0; extra == "secure-local-storage" The user requested (constraint) keyring==25.2.1 To fix this you could try to: 1. loosen the range of package versions you've specified 2. remove package versions to allow pip attempt to solve the dependency conflict ``` **EDIT:** I just realized `snowflake-connector-python` only requires `urllib3` when the Python version is less than 3.10. I was doing my `poetry export` in a legacy Python 3.9 environment. So this is a non-issue in Python 3.11, though `keyring` still is. ### What you think should happen instead? The constraints file should not include conflicting versions of these modules, which prevents installation when `snowflake-connector-python` is required. I believe the only way to fix this is to either downgrade the version of `keyring` to 24.x.y, or upgrade the version of `snowflake-connector-python` to 3.12.0. My preference would be for the latter, but either would unblock the process. ### How to reproduce 1. Clone https://github.com/aws/aws-mwaa-local-runner 2. Check out `v2.9.2` branch (currently the default) 3. Add the connector (with secure-local-storage, requiring keyring): `echo 'snowflake-connector-python[secure-local-storage]==3.10.1' >> requirements/requirements.txt` 4. Build the MWAA image: `./mwaa-local-env build-image` 5. Test the requirements: `./mwaa-local-env test-requirements` ### Operating System Amazon Linux 2023.5.20240722 (In a Docker container) ### Versions of Apache Airflow Providers Just the base ones, though this might also happen with ``` apache-airflow-providers-snowflake==5.5.1 ``` ### Deployment Amazon (AWS) MWAA ### Deployment details I haven’t gotten to deployment yet, still trying to get it installed. ### Anything else? _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
