[ https://issues.apache.org/jira/browse/ARROW-7841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17036673#comment-17036673 ]
Kouhei Sutou commented on ARROW-7841: ------------------------------------- Oh, sorry. I missed the regression. I'll fix it. > pyarrow release 0.16.0 breaks `libhdfs.so` loading mechanism > ------------------------------------------------------------ > > Key: ARROW-7841 > URL: https://issues.apache.org/jira/browse/ARROW-7841 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Affects Versions: 0.16.0 > Reporter: Jack Fan > Priority: Major > > I have my env variable setup correctly according to the pyarrow README > {code:java} > $ ls $HADOOP_HOME/lib/native > libhadoop.a libhadooppipes.a libhadoop.so libhadoop.so.1.0.0 > libhadooputils.a libhdfs.a libhdfs.so libhdfs.so.0.0.0 {code} > Use the following script to reproduce > {code:java} > import pyarrow > pyarrow.hdfs.connect('hdfs://localhost'){code} > With pyarrow version 0.15.1 it is fine. > However, version 0.16.0 will give error > {code:java} > Traceback (most recent call last): > File "<string>", line 2, in <module> > File > "/home/jackwindows/anaconda2/lib/python2.7/site-packages/pyarrow/hdfs.py", > line 215, in connect > extra_conf=extra_conf) > File > "/home/jackwindows/anaconda2/lib/python2.7/site-packages/pyarrow/hdfs.py", > line 40, in __init__ > self._connect(host, port, user, kerb_ticket, driver, extra_conf) > File "pyarrow/io-hdfs.pxi", line 89, in > pyarrow.lib.HadoopFileSystem._connect > File "pyarrow/error.pxi", line 99, in pyarrow.lib.check_status > IOError: Unable to load libhdfs: /opt/hadoop/latest/libhdfs.so: cannot open > shared object file: No such file or directory {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)