zhouyuan opened a new issue, #5103: URL: https://github.com/apache/incubator-gluten/issues/5103
### Description Velox hdfs connector is using libhdfs3 which is a C++ based hdfs client. The libhdfs3 is from HAWQ project and is not maintained anymore. In Gluten we used the CK libhdfs3 fork, and added the delegation token support. (https://github.com/oap-project/libhdfs3) However there are still some feature gaps like encrypted rpc(https://github.com/ClickHouse/ClickHouse/issues/22236), federated cluster In practice the HDFS team and Spark team is usually separated so it maybe difficult to introduce a new HDFS client. Gluten/Velox may also need to support JVM based libhdfs (hadoop default) as the second option, so there's no feature gap in HDFS connections. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
