FelixYBW commented on issue #7860: URL: https://github.com/apache/incubator-gluten/issues/7860#issuecomment-2471587608
> Velox use `SpillReadFile` to read the file, it uses `FileInputStream` to read the file and `simd::memcpy` to copy the bytes, It will output batch RowVector one by one. `FileInputStream` uses `velox::LocalReadFile` `pread` or `preadv` to read the file. The optimal way should be we map like 1M each time, unmap it once accessed. Looks MemoryMappedFile::Region can implement it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
