axelmagn opened a new issue, #32235: URL: https://github.com/apache/beam/issues/32235
### What needs to happen? It looks like `requests==2.32.*` was explicitly blocked at some point as a workaround for a docker bug: https://github.com/apache/beam/blob/e4f2322453dad98a0c4c5b30697b32d9404c2ed9/sdks/python/setup.py#L388 Unfortunately, this leads to dependency version conflicts in practice whenever a pipeline depends on other packages that make use of `requests`. In my case it is `huggingface/datasets`. Example: ``` $ poetry add datasets Using version ^2.21.0 for datasets Updating dependencies Resolving dependencies... (0.9s) Because no versions of apache-beam match >2.58,<2.58.1 || >2.58.1,<3.0 and apache-beam (2.58.0) depends on requests (>=2.24.0,<2.32.dev0 || >=2.33.dev0,<3.0.0), apache-beam (>=2.58,<2.58.1 || >2.58.1,<3.0) requires requests (>=2.24.0,<2.32.dev0 || >=2.33.dev0,<3.0.0). And because apache-beam (2.58.1) depends on requests (>=2.24.0,<2.32.dev0 || >=2.33.dev0,<3.0.0) and no versions of requests match >=2.33.dev0,<3.0.0, apache-beam (>=2.58,<3.0) requires requests (>=2.24.0,<2.32.dev0). Because no versions of datasets match >2.21.0,<3.0.0 and datasets (2.21.0) depends on requests (>=2.32.2), datasets (>=2.21.0,<3.0.0) requires requests (>=2.32.2). Thus, datasets (>=2.21.0,<3.0.0) is incompatible with apache-beam (>=2.58,<3.0). So, because XXX depends on both apache-beam (^2.58) and datasets (^2.21.0), version solving failed. ``` It also looks like the PR mentioned in the workaround has been resolved: https://github.com/docker/docker-py/pull/3257 I propose that this restriction is removed, but I am not familiar with the original issue that was worked around, so I'm not sure how to confirm that it's resolved. @tvalentyn FYI ### Issue Priority Priority: 2 (default / most normal work should be filed as P2) ### Issue Components - [X] Component: Python SDK - [ ] Component: Java SDK - [ ] Component: Go SDK - [ ] Component: Typescript SDK - [ ] Component: IO connector - [ ] Component: Beam YAML - [ ] Component: Beam examples - [ ] Component: Beam playground - [ ] Component: Beam katas - [ ] Component: Website - [ ] Component: Infrastructure - [ ] Component: Spark Runner - [ ] Component: Flink Runner - [ ] Component: Samza Runner - [ ] Component: Twister2 Runner - [ ] Component: Hazelcast Jet Runner - [ ] Component: Google Cloud Dataflow Runner -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
