Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/17299 )
Change subject: IMPALA-10652: Optimize the checking of the size of incremental stats ...................................................................... Patch Set 4: (4 comments) http://gerrit.cloudera.org:8080/#/c/17299/4/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java File fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java: http://gerrit.cloudera.org:8080/#/c/17299/4/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java@461 PS4, Line 461: // The partition has incremental stats, : // and the partition is not calculated this time : if (part.hasIncrementalStats() && !partIds.contains(part.getId())) { : ++numOfAllIncStatsPartitions; : } In the commit message you have the following formula: incremental statistics size = Existing partition statistics + This time calculation partition stats - Repeated calculation partition stats You could also copy the formula here to make the code more verbose. If I understand correctly, this part is the "Existing partition statistics". Please add a comment about it. It'd be nice if the code and comments would cleanly express the new formula. http://gerrit.cloudera.org:8080/#/c/17299/4/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java@467 PS4, Line 467: numOfAllIncStatsPartitions += partitionSet_.getPartitions().size(); Please add comment that this is the "This time calculation partition stats" from the formula. http://gerrit.cloudera.org:8080/#/c/17299/4/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java@471 PS4, Line 471: stas nit: stats http://gerrit.cloudera.org:8080/#/c/17299/4/fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java File fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java: http://gerrit.cloudera.org:8080/#/c/17299/4/fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java@1787 PS4, Line 1787: checkComputeStatsStmt( : "compute incremental stats functional.alltypes partition(year=2010, month=10)"); : BackendConfig.INSTANCE.getBackendCfg().setInc_stats_size_limit_bytes(bytes); Please add a few more tests where you invoke compute incremental stats for a set of partitions. -- To view, visit http://gerrit.cloudera.org:8080/17299 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4f35ea936445015a3b8b8102b1891db29751b5ee Gerrit-Change-Number: 17299 Gerrit-PatchSet: 4 Gerrit-Owner: liuyao <[email protected]> Gerrit-Reviewer: Aman Sinha <[email protected]> Gerrit-Reviewer: Bikramjeet Vig <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Qifan Chen <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]> Gerrit-Reviewer: liuyao <[email protected]> Gerrit-Comment-Date: Tue, 27 Apr 2021 17:47:44 +0000 Gerrit-HasComments: Yes
