Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/24579 )
Change subject: IMPALA-15139: Support DELTA_LENGTH_BYTE_ARRAY Parquet encoding ...................................................................... Patch Set 9: (3 comments) http://gerrit.cloudera.org:8080/#/c/24579/9/be/src/exec/parquet/parquet-column-readers.cc File be/src/exec/parquet/parquet-column-readers.cc: http://gerrit.cloudera.org:8080/#/c/24579/9/be/src/exec/parquet/parquet-column-readers.cc@1063 PS9, Line 1063: for (int i = 0; i < decoded; ++i) { : StringValue* val = reinterpret_cast<StringValue*>( : reinterpret_cast<uint8_t*>(out_vals) + stride * i); : if (!val->IsSmall()) { : DCHECK(!val->CanBeSmallified()); : col_chunk_reader_.keep_data_page_pool_ = true; : break; : } : } It seem better (potentially faster?) to do this within NextValues, e.g. by setting bool* or having a member for this within the decoder. http://gerrit.cloudera.org:8080/#/c/24579/9/testdata/parquet_delta_length_byte_array_encoding/parquet_files_generator.py File testdata/parquet_delta_length_byte_array_encoding/parquet_files_generator.py: http://gerrit.cloudera.org:8080/#/c/24579/9/testdata/parquet_delta_length_byte_array_encoding/parquet_files_generator.py@37 PS9, Line 37: NULL Are you sure that this is enough for to use RLE encoding? Not sure how Trino does it, but Impala uses rle instead of bitset only when it saves space, which is around 8/16 matching values. An alternative to creating bigger data here would be to create a larger table in trino interop tests, e.g. CTAS select * from functional_parquet.alltypes. Don't know how fast would it be with our internal mini Trino, so this would experimental. http://gerrit.cloudera.org:8080/#/c/24579/8/tests/query_test/test_parquet_delta_length_byte_array_encoding.py File tests/query_test/test_parquet_delta_length_byte_array_encoding.py: http://gerrit.cloudera.org:8080/#/c/24579/8/tests/query_test/test_parquet_delta_length_byte_array_encoding.py@24 PS8, Line 24: > Ack thx, maybe it is just personal taste, but it seems much clearer now -- To view, visit http://gerrit.cloudera.org:8080/24579 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9e4395fa30cd7fb2c89ef5a2b16b043d4de60b52 Gerrit-Change-Number: 24579 Gerrit-PatchSet: 9 Gerrit-Owner: Mihaly Szjatinya <[email protected]> Gerrit-Reviewer: Balazs Hevele <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Mihaly Szjatinya <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]> Gerrit-Comment-Date: Wed, 29 Jul 2026 09:54:13 +0000 Gerrit-HasComments: Yes
