Hello Xuebin Su, Zoltan Borok-Nagy, Michael Smith, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/24566
to look at the new patch set (#3).
Change subject: IMPALA-15157: Avoid decompressing a page that is fully skipped
at a page boundary
......................................................................
IMPALA-15157: Avoid decompressing a page that is fully skipped at a page
boundary
During late materialization / row-level skipping, when the rows to skip
end exactly on a Parquet page boundary (e.g. an entire row group is
filtered out by row-level predicates), BaseScalarColumnReader::
SkipRowsInternal() would fall out of the page-skipping loop and then
decompress that final page via ReadCurrentDataPage() only to then skip
all of its values.
This changes the loop guard from '>' to '>=' so a page whose values are
all skipped is discarded as a whole compressed page (counted in
NumPagesSkippedByLateMaterialization) without decompressing it. When the
skip lands exactly on the page boundary we short-circuit and return
before reading the next page header. Query results are unchanged; this
only removes the unnecessary decompression and in-page value skipping.
Testing:
- Updated and extended parquet-late-materialization-unique-db.test to
cover skipping to a page boundary (whole-page discard) and in-page
value skipping.
Change-Id: I18f0e69c267c9bf041ff9f6a3d677239ae43971a
Assisted-By: Claude Opus 4.8 <[email protected]>
---
M be/src/exec/parquet/parquet-column-readers.cc
M be/src/exec/parquet/parquet-column-readers.h
M
testdata/workloads/functional-query/queries/QueryTest/parquet-late-materialization-unique-db.test
3 files changed, 39 insertions(+), 12 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/66/24566/3
--
To view, visit http://gerrit.cloudera.org:8080/24566
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I18f0e69c267c9bf041ff9f6a3d677239ae43971a
Gerrit-Change-Number: 24566
Gerrit-PatchSet: 3
Gerrit-Owner: Yifan Zhang <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Michael Smith <[email protected]>
Gerrit-Reviewer: Xuebin Su <[email protected]>
Gerrit-Reviewer: Yifan Zhang <[email protected]>
Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
