On 2023/04/23 09:38:02 "Yang, Yang10" wrote: > Hi, > > As discussed in this issue: https://github.com/apache/arrow/issues/35287, currently Arrow only supports one parameter: compression_level to be customized. We would like to make more compression parameters (such as window_bits) customizable when creating the Codec, given the variety of usage scenarios. As suggested by @kou, we may introduce a new options class such as Codec::Options to make the structure clear and easy to extend. But it may take some effort as this is more like a code structure refactor. Passing a parameter directly is another approach, easy to implement but may be hard to extend. So we would like some further discussion here. If you have any suggestion or comments, please share them on above issue or here. Thanks! > > Best, > Yang >
In most systems, including arrow, do not have a compression flag except for compression level. But I think we can provide a Codec::Options, like RocksDB [1]. With an options, we're able to config lz4 or zstd dictionary more flexible. So I think It's ok. Best regards, Xuwei Fu [1] https://github.com/facebook/rocksdb/blob/main/include/rocksdb/advanced_options.h#L86
