mrocklin commented on issue #38389: URL: https://github.com/apache/arrow/issues/38389#issuecomment-1777106272
Cool. What I'm reading from you are a couple of insights: 1. Arrow uses SNAPPY compression by default 2. SNAPPY is performing around 200 MB/s on these machines I'll add my understanding, which is that I also expect SNAPPY to operate at GB/s speeds, but for some reason it's not here. On Colab I also get 200MB/s as you do, but on my MacBook I get 767 MB/s This probably allows us to dive a bit deeper into the problem. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
