zero323 commented on a change in pull request #33900:
URL: https://github.com/apache/spark/pull/33900#discussion_r727412156
##########
File path: examples/src/main/python/sql/arrow.py
##########
@@ -32,7 +32,7 @@
def dataframe_with_arrow_example(spark):
- import numpy as np # type: ignore[import]
+ import numpy as np
Review comment:
Sorry for a delay @HyukjinKwon. It too longer longer than I expected, to
test this properly (I had to isolate things, because mypy cache + some
undefined quirks, caused unpredictable behavior locally, even if tested within
conda / venv environments) .
Long story short ‒ this shouldn't really impact end users.
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -197,7 +197,7 @@ jobs:
- name: Install Python packages (Python 3.8)
if: (contains(matrix.modules, 'sql') && !contains(matrix.modules,
'sql-'))
run: |
- python3.8 -m pip install numpy 'pyarrow<5.0.0' pandas scipy xmlrunner
+ python3.8 -m pip install 'numpy>=1.20.0' 'pyarrow<5.0.0' pandas scipy
xmlrunner
Review comment:
Strictly speaking, we don't, or we could use package version bound
(>=1.7).
But it is useful to have things consistent (of course, unless we
intentionally test against different versions), for anyone who, like me, builds
dev environments using tests as a guideline. If that's something that doesn't
look right, I am happy to revert it.
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -460,7 +460,7 @@ jobs:
# See also https://github.com/sphinx-doc/sphinx/issues/7551.
# Jinja2 3.0.0+ causes error when building with Sphinx.
# See also https://issues.apache.org/jira/browse/SPARK-35375.
- python3.9 -m pip install 'sphinx<3.1.0' mkdocs numpy
pydata_sphinx_theme ipython nbsphinx numpydoc 'jinja2<3.0.0'
+ python3.9 -m pip install 'sphinx<3.1.0' mkdocs 'numpy>=1.20.0'
pydata_sphinx_theme ipython nbsphinx numpydoc 'jinja2<3.0.0'
python3.9 -m pip install sphinx_plotly_directive 'pyarrow<5.0.0'
pandas 'plotly>=4.8'
Review comment:
Good point!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
