Sourabh Goyal has posted comments on this change. ( http://gerrit.cloudera.org:8080/18324 )
Change subject: IMPALA-11181: Improving performance of compaction checking ...................................................................... Patch Set 2: (5 comments) http://gerrit.cloudera.org:8080/#/c/18324/2/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java File fe/src/main/java/org/apache/impala/catalog/HdfsTable.java: http://gerrit.cloudera.org:8080/#/c/18324/2/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@297 PS2, Line 297: private long lastCompactionId_ = -1; Please add some more description on the use of this variable, how it is supposed to get updated etc http://gerrit.cloudera.org:8080/#/c/18324/2/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@974 PS2, Line 974: if (partition.getLastCompactionId() > 0) { nit: we can skip the check since the following condition is sufficient: lastCompactionId_ = Math.max(lastCompactionId_, partition.getLastCompactionId()); http://gerrit.cloudera.org:8080/#/c/18324/2/fe/src/main/java/org/apache/impala/catalog/local/DirectMetaProvider.java File fe/src/main/java/org/apache/impala/catalog/local/DirectMetaProvider.java: http://gerrit.cloudera.org:8080/#/c/18324/2/fe/src/main/java/org/apache/impala/catalog/local/DirectMetaProvider.java@485 PS2, Line 485: // Compaction id is not provided with DirectMetaProvider implementation Why are we changing the old behavior? http://gerrit.cloudera.org:8080/#/c/18324/2/fe/src/main/java/org/apache/impala/catalog/local/DirectMetaProvider.java@538 PS2, Line 538: if (metas.isEmpty()) { nit: We can skip this check since checkLatestCompaction() already checks metas size. http://gerrit.cloudera.org:8080/#/c/18324/2/fe/src/main/java/org/apache/impala/catalog/local/DirectMetaProvider.java@543 PS2, Line 543: List<PartitionRef> stalePartitions = MetastoreShim.checkLatestCompaction( nit: if we want to log the time that checkLatestCompaction() takes, it would be better to move the Stopwatch check in MetastoreShim.checkLatestCompaction() -- To view, visit http://gerrit.cloudera.org:8080/18324 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I32e30ec418ad09bef862e61163539a910c96c44c Gerrit-Change-Number: 18324 Gerrit-PatchSet: 2 Gerrit-Owner: Yu-Wen Lai <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Sourabh Goyal <[email protected]> Gerrit-Reviewer: Yu-Wen Lai <[email protected]> Gerrit-Comment-Date: Fri, 25 Mar 2022 00:16:01 +0000 Gerrit-HasComments: Yes
