This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 5e6986b8198 [SPARK-45409][INFRA] Pin `torch<=2.0.1`
5e6986b8198 is described below
commit 5e6986b819821c4ea5e341217b6c901690d93e90
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Tue Oct 3 23:59:49 2023 -0700
[SPARK-45409][INFRA] Pin `torch<=2.0.1`
### What changes were proposed in this pull request?
This PR aims to pin `torch` to recover Python Linter failure.
### Why are the changes needed?
`torch 2.1.0` is released and breaks `mypy` in a weird way.
- https://github.com/apache/spark/actions/runs/6399720715/job/17372994437
```
starting mypy annotations test...
annotations failed mypy checks:
/usr/local/lib/python3.9/dist-packages/torch/_dynamo/variables/tensor.py:369:
error: INTERNAL ERROR -- Please try using mypy master on GitHub:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
If this issue continues with mypy master, please report a bug at
https://github.com/python/mypy/issues
version: 0.982
/usr/local/lib/python3.9/dist-packages/torch/_dynamo/variables/tensor.py:369: :
note: please use --show-traceback to print a traceback when reporting a bug
2
```
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Recover `Python Linter` in CIs.
I also manually verified this.
```
$ pip3 freeze | grep torch
torch==2.0.1
torcheval==0.0.7
torchvision==0.15.2
$ dev/lint-python
...
all lint-python tests passed!
```
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #43207 from dongjoon-hyun/traitlets.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
dev/infra/Dockerfile | 2 +-
dev/requirements.txt | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/dev/infra/Dockerfile b/dev/infra/Dockerfile
index d816ec5ec1b..001de613a92 100644
--- a/dev/infra/Dockerfile
+++ b/dev/infra/Dockerfile
@@ -91,7 +91,7 @@ RUN python3.9 -m pip install numpy pyarrow 'pandas<=2.1.1'
scipy unittest-xml-re
RUN python3.9 -m pip install 'grpcio>=1.48,<1.57' 'grpcio-status>=1.48,<1.57'
'protobuf==3.20.3' 'googleapis-common-protos==1.56.4'
# Add torch as a testing dependency for TorchDistributor
-RUN python3.9 -m pip install torch torchvision --index-url
https://download.pytorch.org/whl/cpu
+RUN python3.9 -m pip install 'torch<=2.0.1' torchvision --index-url
https://download.pytorch.org/whl/cpu
RUN python3.9 -m pip install torcheval
# Add Deepspeed as a testing dependency for DeepspeedTorchDistributor
RUN python3.9 -m pip install deepspeed
diff --git a/dev/requirements.txt b/dev/requirements.txt
index 88cb90da6a8..f4b036cae1c 100644
--- a/dev/requirements.txt
+++ b/dev/requirements.txt
@@ -61,7 +61,7 @@ googleapis-common-protos-stubs==2.2.0
grpc-stubs==1.24.11
# TorchDistributor dependencies
-torch
+torch<=2.0.1
torchvision
torcheval
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
