This is an automated email from the ASF dual-hosted git repository.
chaokunyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/fury.git
The following commit(s) were added to refs/heads/main by this push:
new a2b53f90 perf(python): upgrade pyarrow version to 15.0.0 (#2056)
a2b53f90 is described below
commit a2b53f905f0a46dc62c357d81cca8ee34f26ca8e
Author: PAN <[email protected]>
AuthorDate: Wed Feb 12 23:34:13 2025 +0800
perf(python): upgrade pyarrow version to 15.0.0 (#2056)
<!--
**Thanks for contributing to Fury.**
**If this is your first time opening a PR on fury, you can refer to
[CONTRIBUTING.md](https://github.com/apache/fury/blob/main/CONTRIBUTING.md).**
Contribution Checklist
- The **Apache Fury (incubating)** community has restrictions on the
naming of pr titles. You can also find instructions in
[CONTRIBUTING.md](https://github.com/apache/fury/blob/main/CONTRIBUTING.md).
- Fury has a strong focus on performance. If the PR you submit will have
an impact on performance, please benchmark it first and provide the
benchmark result here.
-->
## What does this PR do?
pip install -v -e .
Using pip 22.0.4 from
D:\code\PycharmProjects\project\venv\lib\site-packages\pip (python 3.9)
Obtaining file:///D:/code/fury/fury/python
Running command python setup.py egg_info
A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.0.2 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.
If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.
Traceback (most recent call last): File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "D:\code\fury\fury\python\setup.py", line 32, in <module>
import pyarrow as pa
File
"D:\code\PycharmProjects\project\venv\lib\site-packages\pyarrow\__init__.py",
line 65, in <module>
import pyarrow.lib as _lib
AttributeError: _ARRAY_API not found
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "D:\code\fury\fury\python\setup.py", line 32, in <module>
import pyarrow as pa
File
"D:\code\PycharmProjects\project\venv\lib\site-packages\pyarrow\__init__.py",
line 65, in <module>
import pyarrow.lib as _lib
File "pyarrow\lib.pyx", line 36, in init pyarrow.lib
ImportError: numpy.core.multiarray failed to import
error: subprocess-exited-with-error
× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a
problem with pip.
<!-- Describe the purpose of this PR. -->
## Related issues
<!--
Is there any related issue? Please attach here.
- #xxxx0
- #xxxx1
- #xxxx2
-->
## Does this PR introduce any user-facing change?
<!--
If any user-facing interface changes, please [open an
issue](https://github.com/apache/fury/issues/new/choose) describing the
need to do so and update the document if necessary.
-->
- [ ] Does this PR introduce any public API change?
- [ ] Does this PR introduce any binary protocol compatibility change?
## Benchmark
<!--
When the PR has an impact on performance (if you don't know whether the
PR will have an impact on performance, you can submit the PR first, and
if it will have impact on performance, the code reviewer will explain
it), be sure to attach a benchmark data here.
-->
---
ci/deploy.sh | 4 ++--
ci/run_ci.py | 6 ++----
docs/guide/DEVELOPMENT.md | 12 +++++++++---
java/fury-format/pom.xml | 2 +-
python/README.md | 8 +++++++-
python/setup.py | 2 +-
6 files changed, 22 insertions(+), 12 deletions(-)
diff --git a/ci/deploy.sh b/ci/deploy.sh
index cbf37204..59a4f396 100755
--- a/ci/deploy.sh
+++ b/ci/deploy.sh
@@ -164,8 +164,8 @@ install_pyarrow() {
pip install pyarrow==18.0.0
pip install numpy
else
- pip install pyarrow==14.0.0
- pip install "numpy<2.0.0"
+ pip install pyarrow==15.0.0
+ # Automatically install numpy
fi
}
diff --git a/ci/run_ci.py b/ci/run_ci.py
index cad0c73a..982eef11 100644
--- a/ci/run_ci.py
+++ b/ci/run_ci.py
@@ -27,9 +27,7 @@ import importlib
BAZEL_VERSION = "6.3.2"
-PYARROW_VERSION = "14.0.0"
-
-NUMPY_VERSION = "1.26.4"
+PYARROW_VERSION = "15.0.0"
PROJECT_ROOT_DIR = os.path.join(os.path.dirname(os.path.abspath(__file__)),
"../")
@@ -120,8 +118,8 @@ def _run_js():
def _install_cpp_deps():
_exec_cmd(f"pip install pyarrow=={PYARROW_VERSION}")
+ # Automatically install numpy
_exec_cmd("pip install psutil")
- _exec_cmd(f"pip install numpy=={NUMPY_VERSION}")
_install_bazel()
diff --git a/docs/guide/DEVELOPMENT.md b/docs/guide/DEVELOPMENT.md
index 5cbdd868..e00ff31c 100644
--- a/docs/guide/DEVELOPMENT.md
+++ b/docs/guide/DEVELOPMENT.md
@@ -24,7 +24,13 @@ mvn clean compile -DskipTests
```bash
cd python
-pip install pyarrow==14.0.0 Cython wheel numpy pytest
+# Uninstall numpy first so that when we install pyarrow, it will install the
correct numpy version automatically.
+# For Python versions less than 3.13, numpy 2 is not currently supported.
+pip uninstall -y numpy
+# Install necessary environment for Python < 3.13.
+pip install pyarrow==15.0.0 Cython wheel pytest
+# For Python 3.13, pyarrow 18.0.0 is available and requires numpy version
greater than 2.
+# pip install pyarrow==18.0.0 Cython wheel pytest
pip install -v -e .
```
@@ -37,14 +43,14 @@ pip install -v -e .
Build fury row format：
```bash
-pip install pyarrow==14.0.0
+pip install pyarrow==15.0.0
bazel build //cpp/fury/row:fury_row_format
```
Build fury row format encoder:
```bash
-pip install pyarrow==14.0.0
+pip install pyarrow==15.0.0
bazel build //cpp/fury/encoder:fury_encoder
```
diff --git a/java/fury-format/pom.xml b/java/fury-format/pom.xml
index 6e55c8fb..ac1527c6 100644
--- a/java/fury-format/pom.xml
+++ b/java/fury-format/pom.xml
@@ -49,7 +49,7 @@
<properties>
<maven.compiler.source>8</maven.compiler.source>
<maven.compiler.target>8</maven.compiler.target>
- <arrow.version>14.0.0</arrow.version>
+ <arrow.version>15.0.0</arrow.version>
<jackson-bom.version>2.16.0</jackson-bom.version>
<fury.java.rootdir>${basedir}/..</fury.java.rootdir>
</properties>
diff --git a/python/README.md b/python/README.md
index 4f3859d8..a3892513 100644
--- a/python/README.md
+++ b/python/README.md
@@ -6,7 +6,13 @@ Fury is a blazingly-fast multi-language serialization
framework powered by just-
```bash
cd python
-pip install pyarrow==14.0.0 Cython wheel numpy pytest
+# Uninstall numpy first so that when we install pyarrow, it will install the
correct numpy version automatically.
+# For Python versions less than 3.13, numpy 2 is not currently supported.
+pip uninstall -y numpy
+# Install necessary environment for Python < 3.13.
+pip install pyarrow==15.0.0 Cython wheel pytest
+# For Python 3.13, pyarrow 18.0.0 is available and requires numpy version
greater than 2.
+# pip install pyarrow==18.0.0 Cython wheel pytest
pip install -v -e .
```
diff --git a/python/setup.py b/python/setup.py
index e6502b66..d953ea00 100644
--- a/python/setup.py
+++ b/python/setup.py
@@ -37,7 +37,7 @@ try:
except FileExistsError:
pass
-pyarrow_version = "14.0.0" if sys.version_info.minor < 13 else "18.0.0"
+pyarrow_version = "15.0.0" if sys.version_info.minor < 13 else "18.0.0"
# Check if we're running 64-bit Python
if not sys.maxsize > 2**32:
raise RuntimeError("Not supported on 32-bit")
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
