ianmcook commented on PR #35:
URL: https://github.com/apache/arrow-experiments/pull/35#issuecomment-2354292525
@felipecrv would you say that the below are valid interpretations of the
timings and compression ratios above?
- Using Arrow IPC buffer compression introduces less decompression latency
than using HTTP compression.
- If it's not an option to use Arrow IPC buffer compression (e.g. because
it's not implemented in the Arrow library you're using), then:
- If the network is very fast and data transfer costs are not a concern at
all, don't use any HTTP compression.
- If the network is fairly fast and data transfer costs are not a major
concern, zstd is often the best all-around balanced option (but YMMV so try it
yourself on a representative sample of datasets).
- If the network is slower or data transfer costs are a major concern, try
experimenting with other HTTP compression codecs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
