ozgrakkurt commented on issue #5240: URL: https://github.com/apache/arrow-rs/issues/5240#issuecomment-1873752170
@tustvold so far I got initial version of this to work. The IO itself is 2x-3x as fast as tokio/stdlib io but the decoding is the main bottleneck like you said. I suspect I can tune decoding speed a lot, we currently plain encode columns with many many repetitions and then lz4 compress them so this is not very ideal I am guessing. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
