Qifan Chen has posted comments on this change. ( http://gerrit.cloudera.org:8080/17289 )
Change subject: IMPALA-10532: TestOverlapMinMaxFilters.test_overlap_min_max_filters seems flaky ...................................................................... Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/17289/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/17289/2//COMMIT_MSG@10 PS2, Line 10: due to the query plan change (from 3-node scan to : 2-node scan) which splits the row groups among scan nodes differently. > It's not clear to me why no pages are filtered by the min-max filter. Could Here are some details if it is not too much :-) The bottom line is that only 2 nodes are involved and the split is such that in instance 1, the overlap ratio (0.749748) is higher than the threshold (0.5) which implies no min/max filtering at the page and row level. In instance 2, the overlap ration is 0 which implies no min/max filtering. Combined, no page level filtering is done and thus the total pages filtered is 0. The details is purposely left out in the commit message. Executor instance 1: 1732 E RowGroup Debug: Try to filter out a rowgroup via overlap predicate: MonotonicMillis=13987739, fid=1, SchemaNode=optional int64 l_orderkey [i:0 d:1 r:0], columnType=BIGINT, overlap ratio=0.749748, threshold=0.5, worthiness=0, enabled for page=0, enabled for row=0, data min=1, data max=3209607, content=BigIntMinMaxFilter(min=224167, max=2630562, always_false=false), always_true=false) 1778 E - NumRuntimeFilteredPages: 0 (0) 1779 E - NumRuntimeFilteredRowGroups: 0 (0) Executor instance 2 1919 E RowGroup Debug: Try to filter out a rowgroup via overlap predicate: MonotonicMillis=13987773, fid=1, SchemaNode=optional int64 l_orderkey [i:0 d:1 r:0], columnType=BIGINT, overlap ratio=0, threshold=0.5, worthiness=1, enabled for page=1, enabled for row=0, data min=3209632, data max=6000000, content=BigIntMinMaxFilter(min=224167, max=2630562, always_false=false), always_true=false) 1965 E - NumRuntimeFilteredPages: 0 (0) 1966 E - NumRuntimeFilteredRowGroups: 1 (1) -- To view, visit http://gerrit.cloudera.org:8080/17289 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I527de530f7db1ce959e7ef2ae3ced18677221c9f Gerrit-Change-Number: 17289 Gerrit-PatchSet: 2 Gerrit-Owner: Qifan Chen <qc...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com> Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com> Gerrit-Comment-Date: Mon, 12 Apr 2021 17:45:15 +0000 Gerrit-HasComments: Yes
