Jason Fehr has posted comments on this change. ( http://gerrit.cloudera.org:8080/21762 )
Change subject: IMPALA-13333: Limit memory estimation if PlanNode can spill ...................................................................... Patch Set 31: (3 comments) http://gerrit.cloudera.org:8080/#/c/21762/31//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/21762/31//COMMIT_MSG@31 PS31, Line 31: MEM_ESTIMATE_SCALE_FOR_SPILLING_OPERATOR Please add documentation to https://impala.apache.org/docs/build/html/topics/impala_set.html about this query option, what it does, and it's default value. This could also be addressed in a follow-up change. http://gerrit.cloudera.org:8080/#/c/21762/31/fe/src/main/java/org/apache/impala/planner/ResourceProfile.java File fe/src/main/java/org/apache/impala/planner/ResourceProfile.java: http://gerrit.cloudera.org:8080/#/c/21762/31/fe/src/main/java/org/apache/impala/planner/ResourceProfile.java@36 PS31, Line 36: // both are set. It is, however, not guaranteed to be <= maxMemReservationBytes_ if Good catch on the mismatch between what the doc says and what the constructor actually does. http://gerrit.cloudera.org:8080/#/c/21762/31/fe/src/main/java/org/apache/impala/planner/ResourceProfileBuilder.java File fe/src/main/java/org/apache/impala/planner/ResourceProfileBuilder.java: http://gerrit.cloudera.org:8080/#/c/21762/31/fe/src/main/java/org/apache/impala/planner/ResourceProfileBuilder.java@110 PS31, Line 110: // in ResourceProfile constructor. However, their relationship with I don't like having the build logic separated between this class and the ResourceProfile constructor. However, it may be too late to correct if ResourceProfile instances are constructed in a lot of places without using ResourceProfileBuilder. The ResourceProfile constructor does not have a scope and is thus package-private. Thus, most places should be using ResourceProfileBuilder. For good measure, you could also remove the ResourceBuilder noReservation() and invalid() functions and instead put those functions in here so they return a ResourceBuilder that could be used like: ResourceBuilder.noReservation(memEstimateBytes).build() and ResourceBuilder.invalid().build() -- To view, visit http://gerrit.cloudera.org:8080/21762 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I290c4e889d4ab9e921e356f0f55a9c8b11d0854e Gerrit-Change-Number: 21762 Gerrit-PatchSet: 31 Gerrit-Owner: Riza Suminto <[email protected]> Gerrit-Reviewer: Abhishek Rawat <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Daniel Becker <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Jason Fehr <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Reviewer: Wenzhe Zhou <[email protected]> Gerrit-Reviewer: Yida Wu <[email protected]> Gerrit-Comment-Date: Tue, 11 Feb 2025 21:01:24 +0000 Gerrit-HasComments: Yes
