Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/19033 )
Change subject: IMPALA-11604 Planner changes for CPU usage ...................................................................... Patch Set 43: (3 comments) http://gerrit.cloudera.org:8080/#/c/19033/42/be/src/util/backend-gflag-util.cc File be/src/util/backend-gflag-util.cc: http://gerrit.cloudera.org:8080/#/c/19033/42/be/src/util/backend-gflag-util.cc@211 PS42, Line 211: um processing cost that a frag > Maybe add a TODO to set a range for this parameter to avoid unexpected beha Ack http://gerrit.cloudera.org:8080/#/c/19033/43/fe/src/main/java/org/apache/impala/planner/PlanFragment.java File fe/src/main/java/org/apache/impala/planner/PlanFragment.java: http://gerrit.cloudera.org:8080/#/c/19033/43/fe/src/main/java/org/apache/impala/planner/PlanFragment.java@273 PS43, Line 273: root.isBlockingNode This seems to be flawed for case of preaggregation streaming node. The implementation of isBlockingNode() in preaggregate streaming node returns false. But during execution, when the preaggragation managed to aggregate until the end (no rows passed through), CPU costing algorithm should really consider it as a blocking node. Doing so will make costing algorithm set higher instance count for query fragment right above it (which contains final aggregation node). I confirmed this by setting DISABLE_STREAMING_PREAGGREGATIONS=1 when running TPCDS Q78 and Q79. With streaming preaggregation disabled, the fragment containing final aggregation is scheduled with higher instance count. http://gerrit.cloudera.org:8080/#/c/19033/43/fe/src/main/java/org/apache/impala/planner/PlanFragment.java@692 PS43, Line 692: builder.append("work-size="); : builder.append(getTotalWorkSize()); This is now redundant with fragment's input cardinality and the sizing is based on min_processing_cost_per_thread instead of input row count. This can be removed. -- To view, visit http://gerrit.cloudera.org:8080/19033 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If32dc770dfffcdd0be2b5555a789a7720952c68a Gerrit-Change-Number: 19033 Gerrit-PatchSet: 43 Gerrit-Owner: Qifan Chen <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Kurt Deschler <[email protected]> Gerrit-Reviewer: Qifan Chen <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Reviewer: Wenzhe Zhou <[email protected]> Gerrit-Comment-Date: Tue, 07 Feb 2023 00:32:29 +0000 Gerrit-HasComments: Yes
