[
https://issues.apache.org/jira/browse/SPARK-44964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-44964:
---------------------------------
Description:
{code}
./python/run-tests --python-executables=python3 --modules=pyspark-ml-connec
...
Finished test(python3): pyspark.ml.tests.connect.test_parity_torch_data_loader
(1s) ... 2 tests were skipped
Traceback (most recent call last):
File "/.../miniconda3/envs/vanilla-3.10/lib/python3.10/runpy.py", line 187,
in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
File "/.../miniconda3/envs/vanilla-3.10/lib/python3.10/runpy.py", line 110,
in _get_module_details
__import__(pkg_name)
File "/.../workspace/forked/spark/python/pyspark/ml/connect/__init__.py",
line 20, in <module>
from pyspark.ml.connect.base import (
File "/.../workspace/forked/spark/python/pyspark/ml/connect/base.py", line
20, in <module>
import pandas as pd
ModuleNotFoundError: No module named 'pandas'
{code}
was:
{code}
./python/run-tests --python-executables=python3 --modules=pyspark-ml-connec
...
Finished test(python3): pyspark.ml.tests.connect.test_parity_torch_data_loader
(1s) ... 2 tests were skipped
Traceback (most recent call last):
File "/.../miniconda3/envs/vanilla-3.10/lib/python3.10/runpy.py", line 187,
in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
File "/.../miniconda3/envs/vanilla-3.10/lib/python3.10/runpy.py", line 110,
in _get_module_details
__import__(pkg_name)
File "/.../workspace/forked/spark/python/pyspark/ml/connect/__init__.py",
line 20, in <module>
from pyspark.ml.connect.base import (
File "/.../workspace/forked/spark/python/pyspark/ml/connect/base.py", line
20, in <module>
import pandas as pd
ModuleNotFoundError: No module named 'pandas'
> Make PySpark (pyspark-ml-connect module) tests passing without any dependency
> -----------------------------------------------------------------------------
>
> Key: SPARK-44964
> URL: https://issues.apache.org/jira/browse/SPARK-44964
> Project: Spark
> Issue Type: Improvement
> Components: Connect, ML, PySpark, Tests
> Affects Versions: 4.0.0
> Reporter: Hyukjin Kwon
> Priority: Major
>
> {code}
> ./python/run-tests --python-executables=python3 --modules=pyspark-ml-connec
> ...
> Finished test(python3):
> pyspark.ml.tests.connect.test_parity_torch_data_loader (1s) ... 2 tests were
> skipped
> Traceback (most recent call last):
> File "/.../miniconda3/envs/vanilla-3.10/lib/python3.10/runpy.py", line 187,
> in _run_module_as_main
> mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
> File "/.../miniconda3/envs/vanilla-3.10/lib/python3.10/runpy.py", line 110,
> in _get_module_details
> __import__(pkg_name)
> File "/.../workspace/forked/spark/python/pyspark/ml/connect/__init__.py",
> line 20, in <module>
> from pyspark.ml.connect.base import (
> File "/.../workspace/forked/spark/python/pyspark/ml/connect/base.py", line
> 20, in <module>
> import pandas as pd
> ModuleNotFoundError: No module named 'pandas'
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
