dpengpeng opened a new issue, #9258:
URL: https://github.com/apache/incubator-gluten/issues/9258
### Backend
VL (Velox)
### Bug description
I compiled a Gluten 1.3.0 release version using the Centos7 image. I enabled
the HDFS option and the compilation command is as follows:
`bash ./dev/buildbundle-veloxbe.sh --enable_hdfs=ON --enable_vcpkg=ON
--spark_version=3.4 --build_arrow=OFF`
I created an Iceberg table on HDFS and now use Spark to query the table
data. When I run the Gluten program in the K8S environment, I also use the
Centos 7 image, and the following exception information is displayed. Why does
it show that libhdfs.so cannot be found here? I have used the --enable_hdfs=ON
switch.
```
E20250408 16:29:51.768519 88 HdfsFileSystem.cpp:35] ConnectLibHdfs
failed due to: IOError: Unable to load libhdfs: ./libhdfs.so: cannot open
shared object file: No such file or directory
A fatal error has been detected by the Java Runtime Environment:
SIGSEGV (0xb) at pc=0x00007f41a4273850, pid=20, tid=0x00007f41ea17f700
JRE version: OpenJDK Runtime Environment (8.0_422-b05) (build 1.8.0_422-b05)
Java VM: OpenJDK 64-Bit Server VM (25.422-b05 mixed mode linux-amd64
compressed oops)
Problematic frame:
C [libvelox.so+0x2776850]
facebook::velox::filesystems::arrow::io::internal::LibHdfsShim::NewBuilder()+0x0
Core dump written. Default location: /opt/spark/work-dir/core or core.20
An error report file with more information is saved as:
/opt/spark/work-dir/hs_err_pid20.log
If you would like to submit a bug report, please visit:
http://bugreport.java.com/bugreport/crash.jsp
The crash happened outside the Java Virtual Machine in native code.
See problematic frame for where to report the bug.
```
At the end of 2024, I compiled a Gluten program of version 1.3.0-SNAPSHOT,
which can access HDFS normally. Now the release version seems to have changed
on HDFS. How can I access HDFS now?
### Spark version
Spark-3.4.x
### Spark configurations
_No response_
### System information
_No response_
### Relevant logs
```bash
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
