FelixYBW commented on issue #11062: URL: https://github.com/apache/incubator-gluten/issues/11062#issuecomment-3535385478
@rui-mo mentioned when Spark pass the split info to Gluten, it has the file format into. So Velox will use the corresponding file reader to open the split. If we saw issue here, it's most because spark pass wrong file format info here for some reason. Velox itself can't detect the file format and create the right file reader. I'm not sure if it causes issue when a partition has mixed parquet and ORC files in Velox. We never tested it. If it has issues, we may repartition the files and make sure some partitions only has parquet, some ones only has ORC. @jinchengchenghh -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
