[
https://issues.apache.org/jira/browse/ARROW-13535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17392675#comment-17392675
]
cheng pan commented on ARROW-13535:
-----------------------------------
I try this like like {{"hdfs:///"}}
fs.FileSystem.from_uri("hdfs:///nameservice1/user/tdops/1.parquet")
But it still failed,the error shows
oadFileSystems error:
(unable to get root cause for java.lang.NoClassDefFoundError)
(unable to get stack trace for java.lang.NoClassDefFoundError)
hdfsBuilderConnect(forceNewInstance=1, nn=hdfs://, port=0,
kerbTicketCachePath=(NULL), userName=(NULL)) error:
(unable to get root cause for java.lang.NoClassDefFoundError)
(unable to get stack trace for java.lang.NoClassDefFoundError)
/arrow/cpp/src/arrow/filesystem/hdfs.cc:51: Failed to disconnect hdfs client:
IOError: HDFS hdfsFS::Disconnect failed, errno: 9 (Bad file descriptor)
Traceback (most recent call last):
print(fs.FileSystem.from_uri("hdfs:///nameservice1/user/tdops/1.parquet"))
File "pyarrow/_fs.pyx", line 347, in pyarrow._fs.FileSystem.from_uri
File "pyarrow/error.pxi", line 122, in
pyarrow.lib.pyarrow_internal_check_status
File "pyarrow/error.pxi", line 99, in pyarrow.lib.check_status
OSError: HDFS connection failed
> connect hdfs error
> ------------------
>
> Key: ARROW-13535
> URL: https://issues.apache.org/jira/browse/ARROW-13535
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Affects Versions: 5.0.0
> Reporter: cheng pan
> Priority: Major
>
> when i use pyarrow to connect my hdfs, I meet error
> I use
> from pyarrow import fs
> print(fs.FileSystem.from_uri("hdfs://"))
> the error shows loadFileSystems error:
> (unable to get root cause for java.lang.NoClassDefFoundError)
> (unable to get stack trace for java.lang.NoClassDefFoundError)
> hdfsBuilderConnect(forceNewInstance=1, nn=hdfs://, port=0,
> kerbTicketCachePath=(NULL), userName=(NULL)) error:
> (unable to get root cause for java.lang.NoClassDefFoundError)
> (unable to get stack trace for java.lang.NoClassDefFoundError)
> /arrow/cpp/src/arrow/filesystem/hdfs.cc:51: Failed to disconnect hdfs client:
> IOError: HDFS hdfsFS::Disconnect failed, errno: 9 (Bad file descriptor)
> Traceback (most recent call last):
> File "/home/tdops/fucheng.pan/ray-code/read.py", line 15, in <module>
> print(fs.FileSystem.from_uri("hdfs:"))
> File "pyarrow/_fs.pyx", line 347, in pyarrow._fs.FileSystem.from_uri
> File "pyarrow/error.pxi", line 122, in
> pyarrow.lib.pyarrow_internal_check_status
> File "pyarrow/error.pxi", line 99, in pyarrow.lib.check_status
> OSError: HDFS connection failed
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
