Qifan Chen has posted comments on this change. ( http://gerrit.cloudera.org:8080/19033 )
Change subject: IMPALA-11604 Planner changes for CPU usage ...................................................................... Patch Set 47: (8 comments) http://gerrit.cloudera.org:8080/#/c/19033/46//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/19033/46//COMMIT_MSG@138 PS46, Line 138: a list of ProcessingCost > The MergeJoin scenario you mention sounds possible. I believe such case can Yeah I agree that the feature in IV that can look at the connected fragments is a plus. However, the same idea can be implemented here too. Thanks a lot for check the MJ case, which is used to illustrate the importance of operator-driven based decision on DoP for blocking operator. Hopefully this can be implemented in step II, III and IV based implementation. http://gerrit.cloudera.org:8080/#/c/19033/46//COMMIT_MSG@258 PS46, Line 258: 03, F04]. : : A CoreRequirement > The sequential nature of build-probe is resolved through the CoreRequiremen The problem is with the sum((F05+F02), (F06+F01)) used for HJ04, as it assumes the HJ will build and probe as the same time. Looks like step IV should look into query node (same idea as collapsing into PC computation). http://gerrit.cloudera.org:8080/#/c/19033/47//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/19033/47//COMMIT_MSG@138 PS47, Line 138: A query fragment has a processingCosts_ field, a list of ProcessingCost : associated with the fragment. Each element of processingCosts_ may : solely come from a single PlanNode or sum of adjacent PlanNode costs. This para probably should be reworded as follows and moved after the para ending 149, as it is better to introduce the concept of blocking segments first and then define some properties for it. The two new data members are introduced for the query fragment as follows. 1. totalFragmentProcessingCost_ - the total fragment processing cost; 2. blockingSegmentProcessingCosts_ - a map of processing costs for each block segment, keyed on the root node of the blocking segment; Thus for a non-blocking segment, blockingSegmentProcessingCosts_ is empty. For a blocking segment, it has an entry in the map, summarizing the total processing cost for the blocking segment. http://gerrit.cloudera.org:8080/#/c/19033/47//COMMIT_MSG@142 PS47, Line 142: the nit a query http://gerrit.cloudera.org:8080/#/c/19033/47//COMMIT_MSG@149 PS47, Line 149: further in step IV. A fragment without any blocking nodes is called a non-blocking fragment. http://gerrit.cloudera.org:8080/#/c/19033/47//COMMIT_MSG@154 PS47, Line 154: 34550429, 2159270, 23752870, 1 There exist two blocking segments in F03: 1. 08(07) 2. 06(12(11)) Therefore we have PC(blocking segment 1) = 2159270 + 34548320 + 2109 PC(blocking segment 2) = 900 + 23751970 + 1 http://gerrit.cloudera.org:8080/#/c/19033/47//COMMIT_MSG@258 PS47, Line 258: (F05, F02), (F06, F01) These two do not follow the definition of a blocking subtree in that both root fragment 05 and 06 are not blocking. http://gerrit.cloudera.org:8080/#/c/19033/47//COMMIT_MSG@269 PS47, Line 269: successor not clear. You mean previous? -- To view, visit http://gerrit.cloudera.org:8080/19033 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If32dc770dfffcdd0be2b5555a789a7720952c68a Gerrit-Change-Number: 19033 Gerrit-PatchSet: 47 Gerrit-Owner: Qifan Chen <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Kurt Deschler <[email protected]> Gerrit-Reviewer: Qifan Chen <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Reviewer: Wenzhe Zhou <[email protected]> Gerrit-Comment-Date: Tue, 14 Feb 2023 16:33:35 +0000 Gerrit-HasComments: Yes
