mzabaluev opened a new pull request, #9700: URL: https://github.com/apache/arrow-rs/pull/9700
# Which issue does this PR close? - Closes #9699. # What changes are included in this PR? Added `ColumnProperties` option `dictionary_fallback`, getting a `DictionaryFallback` enum value. Two behavior variants are provided (initially, the enum is non-exhaustive to allow more to be added later if necessary): - `OnPageSizeLimit` - the prior behavior and the default, triggers fallback on exceeding the dictionary page size limit. - `OnUnfavorableCompression` - a new behavior, includes the page size limit check and adds a check for encoded size not exceeding the plain data size. Implemented the new optional behavior in the encoder. # Are these changes tested? Added new tests exercising the `OnUnfavorableCompression` behavior. The existing tests exercise `OnPageSizeLimit`. # Are there any user-facing changes? Added API in parquet: * The `DictionaryFallback` enum * `ColumnProperties::dictionary_fallback`, `ColumnProperties::set_dictionary_fallback` * `WriterPropertiesBuilder::set_dictionary_fallback`, `WriterPropertiesBuilder::set_column_dictionary_fallback` * `WriterProperties::dictionary_fallback` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
