wgtmac commented on code in PR #40957: URL: https://github.com/apache/arrow/pull/40957#discussion_r1551960127
########## cpp/src/parquet/encoding.cc: ########## @@ -3988,4 +3988,21 @@ std::unique_ptr<Decoder> MakeDictDecoder(Type::type type_num, } } // namespace detail + +// Informed heavily by https://github.com/apache/parquet-format/blob/master/Encodings.md +// Should we also consider if we're in dictionary encoding mode? or assume no for the +// moment? +Encoding::type ChooseFallbackEncoding(Type::type data_type, + ParquetVersion::type parquet_version, + ParquetDataPageVersion datapage_version) { + if (data_type == Type::BOOLEAN && parquet_version != ParquetVersion::PARQUET_1_0 && + datapage_version == ParquetDataPageVersion::V2) { + return Encoding::RLE; + } else if (data_type == Type::BYTE_ARRAY) { Review Comment: @ClifHouck What about following what parquet-mr does? Although parquet-mr has bundled data page version and writer version, we can either do the following: - Enable v2 encodings if writer version is 2.0+ - Enable v2 encodings when writer version is 2.0+ and data page version is v2. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
