Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/17295 )
Change subject: IMPALA-10650: Bailout min/max filters in hash join builder early ...................................................................... Patch Set 27: (2 comments) Hi Qifan, I wonder if we can improve the minmax filter performance from the build side. I have the following questions and comments. http://gerrit.cloudera.org:8080/#/c/17295/27/be/src/exec/partitioned-hash-join-builder.cc File be/src/exec/partitioned-hash-join-builder.cc: http://gerrit.cloudera.org:8080/#/c/17295/27/be/src/exec/partitioned-hash-join-builder.cc@337 PS27, Line 337: for (const FilterContext& ctx : filter_ctxs_) { I wonder if we can speed this up by iterating ONLY the minmax filters. Maybe copy reference of the minmax filters into separate vector? This function seems to be called frequently on every PhjBuilder::AddBatch. I imagine if minmax filter is enabled, only half of filter_ctxs_ elements are actually minmax filter. We can also pop filter out of the vector once it deemed not useful, therefore speeding up the next iteration. http://gerrit.cloudera.org:8080/#/c/17295/27/be/src/exec/partitioned-hash-join-builder.cc@404 PS27, Line 404: PublishRuntimeFilters(num_build_rows); It seems to me that PublishRuntimeFilters is only called here in FinalizeBuild (I assume near the end of the build process). Since minmax filter can be quickly disabled after reading few early RowBatch, shall we consider to publish them as soon as possible? Say, immediately publish disabled minmax filter from PhjBuilder::DetermineUsefulnessForMinmaxFilters()? -- To view, visit http://gerrit.cloudera.org:8080/17295 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I193646e7acfdd3023f7c947d8107da58a1f41183 Gerrit-Change-Number: 17295 Gerrit-PatchSet: 27 Gerrit-Owner: Qifan Chen <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Qifan Chen <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Reviewer: Wenzhe Zhou <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]> Gerrit-Comment-Date: Tue, 18 May 2021 02:52:50 +0000 Gerrit-HasComments: Yes
