Sahil Takiar has posted comments on this change. ( http://gerrit.cloudera.org:8080/12940 )
Change subject: IMPALA-7322: Add storage wait time to profile ...................................................................... Patch Set 3: (2 comments) http://gerrit.cloudera.org:8080/#/c/12940/8//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/12940/8//COMMIT_MSG@9 PS8, Line 9: Add metrics to record storage loading time in catalog for hdfs, kudu I think we need a clearer definition of "storage-load-time" spelled out in the commit message. We should be clear that it refers to the amount of time spent waiting loading metadata from the underlying storage layer (e.g. S3, HDFS, Kudu, HBase) and *not* does not include the amount of time spending loading data from HMS. http://gerrit.cloudera.org:8080/#/c/12940/3/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java File fe/src/main/java/org/apache/impala/catalog/HdfsTable.java: http://gerrit.cloudera.org:8080/#/c/12940/3/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@1237 PS3, Line 1237: final Timer.Context ctxStg = > Do not understand your comment. I think Bharath means that this code is measuring the amount of time it takes to load metadata from HMS, but what we want to measure is the amount of time it takes to load metadata from HDFS (e.g. from the NameNode). https://gerrit.cloudera.org/#/c/12950/ looks like it re-factors the code so all the filesystem loading is abstracted into a single method (I think loadFileMetadataForPartitions). -- To view, visit http://gerrit.cloudera.org:8080/12940 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6dde7e394b7c1c396d835ef6aa0a55930c0a8660 Gerrit-Change-Number: 12940 Gerrit-PatchSet: 3 Gerrit-Owner: Yongzhi Chen <yc...@cloudera.com> Gerrit-Reviewer: Anurag Mantripragada <anu...@cloudera.com> Gerrit-Reviewer: Bharath Vissapragada <bhara...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Sahil Takiar <stak...@cloudera.com> Gerrit-Reviewer: Vihang Karajgaonkar <vih...@cloudera.com> Gerrit-Reviewer: Yongzhi Chen <yc...@cloudera.com> Gerrit-Comment-Date: Tue, 28 May 2019 18:54:06 +0000 Gerrit-HasComments: Yes
