Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/24566 )
Change subject: IMPALA-15157: Avoid decompressing a page that is fully skipped at a page boundary ...................................................................... IMPALA-15157: Avoid decompressing a page that is fully skipped at a page boundary During late materialization / row-level skipping, when the rows to skip end exactly on a Parquet page boundary (e.g. an entire row group is filtered out by row-level predicates), BaseScalarColumnReader:: SkipRowsInternal() would fall out of the page-skipping loop and then decompress that final page via ReadCurrentDataPage() only to then skip all of its values. This changes the loop guard from '>' to '>=' so a page whose values are all skipped is discarded as a whole compressed page (counted in NumPagesSkippedByLateMaterialization) without decompressing it. When the skip lands exactly on the page boundary we short-circuit and return before reading the next page header. Query results are unchanged; this only removes the unnecessary decompression and in-page value skipping. Testing: - Updated and extended parquet-late-materialization-unique-db.test to cover skipping to a page boundary (whole-page discard) and in-page value skipping. Assisted-by: Claude Opus 4.8 Change-Id: I18f0e69c267c9bf041ff9f6a3d677239ae43971a Reviewed-on: http://gerrit.cloudera.org:8080/24566 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M be/src/exec/parquet/parquet-column-readers.cc M be/src/exec/parquet/parquet-column-readers.h M testdata/workloads/functional-query/queries/QueryTest/parquet-late-materialization-unique-db.test 3 files changed, 39 insertions(+), 12 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/24566 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I18f0e69c267c9bf041ff9f6a3d677239ae43971a Gerrit-Change-Number: 24566 Gerrit-PatchSet: 6 Gerrit-Owner: Yifan Zhang <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Xuebin Su <[email protected]> Gerrit-Reviewer: Yifan Zhang <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
