[GitHub] [arrow] Nikhil-Khetani opened a new issue, #33913: Filesystem Interface: HadoopFileSystem unable to connect to hdfs - libhdfs not found

Sun, 29 Jan 2023 06:14:29 -0800 


Nikhil-Khetani opened a new issue, #33913:
URL: https://github.com/apache/arrow/issues/33913

   ### Describe the bug, including details regarding any error messages, 
version, and platform.
   
   I am running a hadoop distributed filesystem on my local computer and 
attempting to use the new Filesystem interface with HadoopFileSystem to connect 
to my hdfs. However, after verifying my environment variables were set 
correctly and that the libhdfs.so file exists in the specified directory, upon 
trying to connect, I get the error: 
   
   "OSError: Unable to load libhdfs: The specified module could not be found."
   
   Platform: Windows 10
   python == 3.11.1
   pyarrow==10.0.1
   Hadoop/hdfs version == 3.2.2
   Namenode port: 'localhost:9000'
   
   
![image](https://user-images.githubusercontent.com/67691373/215331205-6dcef419-eca8-452f-87b9-40aadf33f739.png)
   
   I have checked libhdfs.so exists and is not empty at specified location:
   
![image](https://user-images.githubusercontent.com/67691373/215332258-c9b1046c-2e0a-42e8-b717-c7b8d04abaef.png)
   
   
   I have also double checked the CLASSPATH variable is set correctly:
   
![image](https://user-images.githubusercontent.com/67691373/215331453-f62a6585-a3c1-45c3-9dd0-ba8fbac90e4f.png)
   
   Thanks, any help is appreciated.
   
   
   
   
   
   
   ### Component(s)
   
   C++, Python


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to