FelixYBW opened a new issue, #12263: URL: https://github.com/apache/gluten/issues/12263
### Description To add Lance read/write support, we have two ways: - Use lance connector in Velox, offload the whole read/write operator to velox plan https://github.com/facebookincubator/velox/pull/16556 - Use lance-spark datasource. To read, gluten gets the arrow data from spark-spark, fwd to velox pipeline. To write, gluten gets the velox data from Velox, converts it as arrow, fwd to lance-spark. No C2R/R2C conversion in the whole process We are going to try lance-spark as the preferred way. ### Gluten version None -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
