wgtmac commented on code in PR #35758:
URL: https://github.com/apache/arrow/pull/35758#discussion_r1205055621
##########
cpp/src/parquet/metadata.cc:
##########
@@ -1462,40 +1462,45 @@ class
ColumnChunkMetaDataBuilder::ColumnChunkMetaDataBuilderImpl {
column_chunk_->meta_data.__set_total_compressed_size(compressed_size);
std::vector<format::Encoding::type> thrift_encodings;
+ std::vector<format::PageEncodingStats> thrift_encoding_stats;
+ // Force Emplace encoding for RL/DL.
+ thrift_encodings.push_back(ToThrift(Encoding::RLE));
+ // Add dictionary page encoding stats
if (has_dictionary) {
-
thrift_encodings.push_back(ToThrift(properties_->dictionary_index_encoding()));
+ // For Parquet V1, Dictionary Data Page and Dictionary Index Page
encodings
+ // is PLAIN_DICTIONARY, but the actual DATA is PLAIN, so force adding
PLAIN.
Review Comment:
I was thinking if this is a wrong behavior which has existed for a long
time. Actually in the parquet file here, it only shows `PLAIN_DICTIONARY`, no
`PLAIN` in the encoding list.
https://github.com/apache/arrow/blob/6bd31f37ae66bd35594b077cb2f830be57e08acd/cpp/src/parquet/reader_test.cc#L788-L797
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
