Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/14600 )
Change subject: IMPALA-9109: Add top-k metadata loading ranking on catalogd UI ...................................................................... Patch Set 4: (5 comments) Thanks for doing this! It will be very helpful in practise! I left some comments hoping we can expose more details. http://gerrit.cloudera.org:8080/#/c/14600/4//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/14600/4//COMMIT_MSG@16 PS4, Line 16: exisit nit: typo? http://gerrit.cloudera.org:8080/#/c/14600/4//COMMIT_MSG@18 PS4, Line 18: showed nit: shown http://gerrit.cloudera.org:8080/#/c/14600/4/common/thrift/JniCatalog.thrift File common/thrift/JniCatalog.thrift: http://gerrit.cloudera.org:8080/#/c/14600/4/common/thrift/JniCatalog.thrift@724 PS4, Line 724: 5: optional i64 table_loading_ns It'd be better to not just expose the median loading time. Since this is shown in a detail page, we can show more like the /rpcz page: Count, min / max, 75th_percentile, 95th_percentile, 98th_percentile, 99th_percentile, 999th_percentile. http://gerrit.cloudera.org:8080/#/c/14600/4/fe/src/main/java/org/apache/impala/catalog/Table.java File fe/src/main/java/org/apache/impala/catalog/Table.java: http://gerrit.cloudera.org:8080/#/c/14600/4/fe/src/main/java/org/apache/impala/catalog/Table.java@203 PS4, Line 203: getMedian() Can we add more metrics so in the future we don't need to touch this part again? I think the max time is also useful in sorting. For example, a partitioned table may take a long time in its first load and take short time in incremental loads later. Median time does not reflect this. Other metrics like 95th/99th percentile may also helpful. http://gerrit.cloudera.org:8080/#/c/14600/4/www/catalog.tmpl File www/catalog.tmpl: http://gerrit.cloudera.org:8080/#/c/14600/4/www/catalog.tmpl@169 PS4, Line 169: Metadata Loading Time (ms) We can show human readable time in this column, i.e. in the forms of 11m25s, 4s495ms, 15.178ms etc. Just like the query duration shown in the /queries page. Catalogd should still pass values in ms so DataTable can sort correctly. We just need to add a render function for this column. Here are two examples: https://github.com/apache/impala/commit/ea4715fd76d6dba0c3777146989c2bf020efabdd https://github.com/apache/impala/commit/725a47b3f275aa76db6a65d4e320f8dbaf9d6b28 -- To view, visit http://gerrit.cloudera.org:8080/14600 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9305a867d7053cde9acc42dae6e47ee440f1a8bf Gerrit-Change-Number: 14600 Gerrit-PatchSet: 4 Gerrit-Owner: Jiawei Wang <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Jiawei Wang <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Yongzhi Chen <[email protected]> Gerrit-Comment-Date: Mon, 04 Nov 2019 09:45:21 +0000 Gerrit-HasComments: Yes
