marioloko commented on issue #2988: URL: https://github.com/apache/arrow-rs/issues/2988#issuecomment-1299244888
I committed in my local fork the changes required to pass the option through `ReadOption`. However, passing the configuration down from `SerializedFileReader` to `create_codec` require changes in the signature of several functions. I tried not to break the public API by keeping the name of the existing functions, but I had to add new methods to public API, `new_with_config`. I also tried to make the new API methods as extensible as possible, by creating a Options and OptionsStruct method for each of the new API methods. Here it is the commit to check the changes in the API: https://github.com/marioloko/arrow-rs/commit/a94b2487263a036b3d95f118b54aa01f33c2b9fa So that the changes in the API can be verified in parallel while I add the compression methods. The important changes are: * Added `CodecOptions` struct to hold `Codec` configuration. * Added `backward_compatible_lz4` option in `CodecOptions`. * Added `CodecOptions` to `ReadOptions` to be able to configure `SerializedFileReader`. * Added `SerializedRowGroupReaderOptionsBuilder` with `CodecOptions` to be able to configure `SerializedRowGroupReader`, with extensible interface. * Added `SerializedPageReaderOptionsBuilder` with `CodecOptions` to be able to configure `SerializedPageReader`, with extensible interface. * Added `new_with_config` to `SerializedPageReader` API to be able to configure `SerializedFileReader` without breaking `new` API. * `CodecOptions` implements `CopyTrait` as it is composed by `Copy` types. If in the future it contains a non `Copy` type, maybe is better to create `CodecOptionsPtr = Arc<CodecOptions>`. * `CodecOptions` is only added in the read path, in the write path the default values are taken, as the options currently only affect the read path and have no effect on write path. If required to add to write path maybe it will be nice to add into `WriteProperties`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
