bvolpato opened a new pull request, #41005:
URL: https://github.com/apache/beam/pull/41005
This PR upgrades the `vllm` extra from 0.10.1.1 to 0.32.0. The old pin dates
from August 2025 and declares `python_requires<3.13`, so the extra cannot
install on Python 3.13 or 3.14. vLLM 0.32.0 declares support for Python
versions below 3.15.
The new version needs newer dependencies, so this PR makes these changes:
- `setup.py`: Pin `vllm==0.32.0` and `openai>=2.25.0,<3.0.0`. Remove the
`triton` pin, because `torch` now selects `triton`. The `openai` cap matches
the `openai==2.44.0` pin in the vLLM CUDA test requirements.
- `vllm_tests_requirements.txt`: Replace `transformers==4.57.1` and raise
`openai`, because the old values conflict with vLLM 0.32.0.
- `common.gradle` and `gpu_image_requirements.txt`: Remove vLLM from the
generated GPU requirements.
**Decision for reviewers:** The GPU requirements no longer include vLLM.
vLLM 0.32.0 pins `torch==2.13.0`, and the `transformers` extra caps
`torch<2.12.0` because newer Linux wheels crash when imported after TensorFlow.
A list with both extras has no solution. I kept the cap and removed vLLM from
the list. No file in this repository reads the GPU requirements, and the vLLM
benchmark image installs `[gcp,vllm]`. People who use the list to get vLLM must
now install the extra. If vLLM must stay in the list, then the list must drop
TensorFlow or the cap must change.
**The generated files:** I did not run `generatePythonRequirementsAll`,
because it needs Linux x86_64 and a large install. I resolved the list with `uv
pip compile` for `x86_64-manylinux_2_28`, with the current pins as preferences.
The result removes vLLM and the 88 packages that only vLLM needed, and it
changes no other pin. The next scheduled regeneration will move the remaining
pins, including `torch`.
**Validated:**
- `[gcp,vllm]` resolves on Python 3.11 and 3.12 for Linux x86_64, with and
without `vllm_tests_requirements.txt`.
- `.[vllm]` from this branch installs on Python 3.11 in a Linux arm64
container. `uv pip check` reports only that `nvidia-cusparselt-cu13` was built
for a different platform.
- `vllm_inference_test.py` passes in that container.
- The vLLM 0.32.0 server CLI lists the `--model`, `--port`,
`--max-num-seqs`, `--gpu-memory-utilization`, `--served-model-name`, and
`--max-model-len` options. The container has no GPU, so I forced the vLLM CPU
platform plugin for this check.
- `VLLMCompletionsModelHandler` and `VLLMChatModelHandler` complete requests
against a fake OpenAI endpoint with `openai` 2.54.0.
**Not validated:** A real vLLM server, GPU inference, the Dynamo path, the
benchmark image build, and Dataflow. The vLLM wheel cannot start a server
without a GPU. Also, I did not install the extra on Python 3.13 or 3.14. The
vLLM benchmark workflow is the best check for the GPU path.
<!--
Thank you for your contribution! Follow this checklist to help us
incorporate your contribution quickly and easily:
- [ ] If this contribution is large, please file an Apache [Individual
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
- [x] Replace these instructions with a description of your change and why
you have made it.
- [ ] Mention the appropriate issue in your description (for example:
`addresses #123` or `fixes #456`).
- [x] Update `CHANGES.md` with noteworthy changes.
See [CONTRIBUTING.md]() for more details.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
