Re: AttributeError importing pyarrow 0.16.0
Thanks for linking to that. The Python there does seems problematic. Upgrading to TravisCI's "bionic" image (with Python 3.7.5 instead of 3.7.1) seems to have fixed it. Tom On Mon, Feb 10, 2020 at 1:34 PM Wes McKinney wrote: > hi Tom, > > Looks like it could be https://bugs.python.org/issue32973, but I'm not > sure. I wasn't able to reproduce locally. The Python version 3.7.1 > running in CI is also potentially suspicious. > > This class of error seems to have a lot of bug reports based on Google > searches > > Message isn't picklable so we should probably fix that regardless > > https://issues.apache.org/jira/browse/ARROW-7826 > > - Wes > > On Mon, Feb 10, 2020 at 12:17 PM Tom Augspurger > wrote: > > > > Hi all, > > > > I'm seeing a strange issue when importing pyarrow on the intake CI. I > get an > > exception saying > > > > AttributeError: type object 'pyarrow.lib.Message' has no attribute > > '__reduce_cython__' > > > > The full traceback is: > > > > __ test_arrow_import > > ___ > > > > def test_arrow_import(): > > > > > import pyarrow > > > > intake/cli/server/tests/test_server.py:32: > > > > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ _ > > _ _ > > > > > ../../../virtualenv/python3.7.1/lib/python3.7/site-packages/pyarrow/__init__.py:49: > > in > > > > from pyarrow.lib import cpu_count, set_cpu_count > > > > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ _ > > _ _ > > > > > ??? > > > > E AttributeError: type object 'pyarrow.lib.Message' has no attribute > > '__reduce_cython__' > > > > pyarrow/ipc.pxi:21: AttributeError > > > > _ TestServerV1Source.test_read_part_compressed > > _ > > > > > > I'm unable to reproduce this locally, and was wondering if anyone else > has > > seen something similar. > > Pyarrow was installed using pip / a wheel ( > > https://travis-ci.org/intake/intake/jobs/648523104#L311). > > > > A common cause of this error message is building with too old of a > Cython. > > While checking this, I noticed > > that some of the files are generated with Cython 0.29.8, while others > were > > generated with 0.29.14. > > I have no idea if this is a problem in general of if it's causing this > > specific issue. > > > > ``` > > _hdfs.cpp:1:/* Generated by Cython 0.29.14 */ > > include/arrow/python/pyarrow_lib.h:20:/* Generated by Cython 0.29.8 */ > > include/arrow/python/pyarrow_api.h:21:/* Generated by Cython 0.29.8 */ > > _plasma.cpp:1:/* Generated by Cython 0.29.14 */ > > _fs.cpp:1:/* Generated by Cython 0.29.14 */ > > lib_api.h:1:/* Generated by Cython 0.29.14 */ > > gandiva.cpp:1:/* Generated by Cython 0.29.14 */ > > _json.cpp:1:/* Generated by Cython 0.29.14 */ > > _parquet.cpp:1:/* Generated by Cython 0.29.14 */ > > _csv.cpp:1:/* Generated by Cython 0.29.14 */ > > _compute.cpp:1:/* Generated by Cython 0.29.14 */ > > _dataset.cpp:1:/* Generated by Cython 0.29.14 */ > > _flight.cpp:1:/* Generated by Cython 0.29.14 */ > > lib.cpp:1:/* Generated by Cython 0.29.14 */ > > ``` > > > > See the https://travis-ci.org/intake/intake/jobs/648523104 for the full > log. > > > > > > Thanks for any pointers! >
Re: AttributeError importing pyarrow 0.16.0
hi Tom, Looks like it could be https://bugs.python.org/issue32973, but I'm not sure. I wasn't able to reproduce locally. The Python version 3.7.1 running in CI is also potentially suspicious. This class of error seems to have a lot of bug reports based on Google searches Message isn't picklable so we should probably fix that regardless https://issues.apache.org/jira/browse/ARROW-7826 - Wes On Mon, Feb 10, 2020 at 12:17 PM Tom Augspurger wrote: > > Hi all, > > I'm seeing a strange issue when importing pyarrow on the intake CI. I get an > exception saying > > AttributeError: type object 'pyarrow.lib.Message' has no attribute > '__reduce_cython__' > > The full traceback is: > > __ test_arrow_import > ___ > > def test_arrow_import(): > > > import pyarrow > > intake/cli/server/tests/test_server.py:32: > > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ _ > > ../../../virtualenv/python3.7.1/lib/python3.7/site-packages/pyarrow/__init__.py:49: > in > > from pyarrow.lib import cpu_count, set_cpu_count > > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ _ > > > ??? > > E AttributeError: type object 'pyarrow.lib.Message' has no attribute > '__reduce_cython__' > > pyarrow/ipc.pxi:21: AttributeError > > _ TestServerV1Source.test_read_part_compressed > _ > > > I'm unable to reproduce this locally, and was wondering if anyone else has > seen something similar. > Pyarrow was installed using pip / a wheel ( > https://travis-ci.org/intake/intake/jobs/648523104#L311). > > A common cause of this error message is building with too old of a Cython. > While checking this, I noticed > that some of the files are generated with Cython 0.29.8, while others were > generated with 0.29.14. > I have no idea if this is a problem in general of if it's causing this > specific issue. > > ``` > _hdfs.cpp:1:/* Generated by Cython 0.29.14 */ > include/arrow/python/pyarrow_lib.h:20:/* Generated by Cython 0.29.8 */ > include/arrow/python/pyarrow_api.h:21:/* Generated by Cython 0.29.8 */ > _plasma.cpp:1:/* Generated by Cython 0.29.14 */ > _fs.cpp:1:/* Generated by Cython 0.29.14 */ > lib_api.h:1:/* Generated by Cython 0.29.14 */ > gandiva.cpp:1:/* Generated by Cython 0.29.14 */ > _json.cpp:1:/* Generated by Cython 0.29.14 */ > _parquet.cpp:1:/* Generated by Cython 0.29.14 */ > _csv.cpp:1:/* Generated by Cython 0.29.14 */ > _compute.cpp:1:/* Generated by Cython 0.29.14 */ > _dataset.cpp:1:/* Generated by Cython 0.29.14 */ > _flight.cpp:1:/* Generated by Cython 0.29.14 */ > lib.cpp:1:/* Generated by Cython 0.29.14 */ > ``` > > See the https://travis-ci.org/intake/intake/jobs/648523104 for the full log. > > > Thanks for any pointers!
AttributeError importing pyarrow 0.16.0
Hi all, I'm seeing a strange issue when importing pyarrow on the intake CI. I get an exception saying AttributeError: type object 'pyarrow.lib.Message' has no attribute '__reduce_cython__' The full traceback is: __ test_arrow_import ___ def test_arrow_import(): > import pyarrow intake/cli/server/tests/test_server.py:32: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../../../virtualenv/python3.7.1/lib/python3.7/site-packages/pyarrow/__init__.py:49: in from pyarrow.lib import cpu_count, set_cpu_count _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > ??? E AttributeError: type object 'pyarrow.lib.Message' has no attribute '__reduce_cython__' pyarrow/ipc.pxi:21: AttributeError _ TestServerV1Source.test_read_part_compressed _ I'm unable to reproduce this locally, and was wondering if anyone else has seen something similar. Pyarrow was installed using pip / a wheel ( https://travis-ci.org/intake/intake/jobs/648523104#L311). A common cause of this error message is building with too old of a Cython. While checking this, I noticed that some of the files are generated with Cython 0.29.8, while others were generated with 0.29.14. I have no idea if this is a problem in general of if it's causing this specific issue. ``` _hdfs.cpp:1:/* Generated by Cython 0.29.14 */ include/arrow/python/pyarrow_lib.h:20:/* Generated by Cython 0.29.8 */ include/arrow/python/pyarrow_api.h:21:/* Generated by Cython 0.29.8 */ _plasma.cpp:1:/* Generated by Cython 0.29.14 */ _fs.cpp:1:/* Generated by Cython 0.29.14 */ lib_api.h:1:/* Generated by Cython 0.29.14 */ gandiva.cpp:1:/* Generated by Cython 0.29.14 */ _json.cpp:1:/* Generated by Cython 0.29.14 */ _parquet.cpp:1:/* Generated by Cython 0.29.14 */ _csv.cpp:1:/* Generated by Cython 0.29.14 */ _compute.cpp:1:/* Generated by Cython 0.29.14 */ _dataset.cpp:1:/* Generated by Cython 0.29.14 */ _flight.cpp:1:/* Generated by Cython 0.29.14 */ lib.cpp:1:/* Generated by Cython 0.29.14 */ ``` See the https://travis-ci.org/intake/intake/jobs/648523104 for the full log. Thanks for any pointers!
