a5creel commented on issue #2468:
URL: https://github.com/apache/arrow/issues/2468#issuecomment-1234826588
I've been working on this problem for a week now and am still getting this
error. Tried rearranging the import order, as well as uninstalling numpy and
pyarrow.
It worked on a previous machine and now fails on a new ec2 instance.
numpy 1.21.6
pyarrow 9.0.0
```
myPacker_df = dd.read_parquet('s3://XXX/YYY/summary_i/', engine = "pyarrow")
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
<ipython-input-2-9934e78b02e8> in <module>
----> 1 myPacker_df =
dd.read_parquet('s3://yse-bioecon/meat_packers/summary_i/', engine = "pyarrow")
~/anaconda3/lib/python3.7/site-packages/dask/dataframe/io/parquet/core.py in
read_parquet(path, columns, filters, categories, index, storage_options,
engine, gather_statistics, split_row_groups, chunksize, **kwargs)
208
209 if isinstance(engine, str):
--> 210 engine = get_engine(engine)
211
212 if hasattr(path, "name"):
~/anaconda3/lib/python3.7/site-packages/dask/dataframe/io/parquet/core.py in
get_engine(engine)
495 elif engine == "pyarrow" or engine == "arrow":
496 pa = import_required("pyarrow", "`pyarrow` not installed")
--> 497 from .arrow import ArrowEngine
498
499 if LooseVersion(pa.__version__) < "0.13.1":
~/anaconda3/lib/python3.7/site-packages/dask/dataframe/io/parquet/arrow.py
in <module>
7 import pyarrow as pa
8 import pyarrow.parquet as pq
----> 9 from pyarrow.compat import guid
10 from ....utils import natural_sort_key, getargspec
11 from ..utils import _get_pyarrow_dtypes, _meta_from_dtypes
ModuleNotFoundError: No module named 'pyarrow.compat'
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
