Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/8529 )
Change subject: [PREVIEW] IMPALA-4886: Expose table metrics in the catalog web UI. ...................................................................... Patch Set 1: (16 comments) http://gerrit.cloudera.org:8080/#/c/8529/1/be/src/catalog/catalog-server.cc File be/src/catalog/catalog-server.cc: http://gerrit.cloudera.org:8080/#/c/8529/1/be/src/catalog/catalog-server.cc@391 PS1, Line 391: DCHECK_EQ(catalog_usage_result.large_tables.size(), : catalog_usage_result.memory_estimates.size()); : DCHECK_EQ(catalog_usage_result.frequent_tables.size(), : catalog_usage_result.num_metadata_operations.size()); can be removed if using a struct instead of multiple arrays http://gerrit.cloudera.org:8080/#/c/8529/1/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java File fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java: http://gerrit.cloudera.org:8080/#/c/8529/1/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java@1091 PS1, Line 1091: REFRESH why not sync this name to RELOAD? http://gerrit.cloudera.org:8080/#/c/8529/1/fe/src/main/java/org/apache/impala/catalog/CatalogUsageMonitor.java File fe/src/main/java/org/apache/impala/catalog/CatalogUsageMonitor.java: http://gerrit.cloudera.org:8080/#/c/8529/1/fe/src/main/java/org/apache/impala/catalog/CatalogUsageMonitor.java@27 PS1, Line 27: Sigleton nit(spelling): Singleton http://gerrit.cloudera.org:8080/#/c/8529/1/fe/src/main/java/org/apache/impala/catalog/CatalogUsageMonitor.java@48 PS1, Line 48: true why always evict for this one? http://gerrit.cloudera.org:8080/#/c/8529/1/fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java File fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java: http://gerrit.cloudera.org:8080/#/c/8529/1/fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java@923 PS1, Line 923: HdfsTable.StorageStats stats, : Reference<Boolean> hasIncrementalStats this is a surprising api since it modifies the last two args and does a bit more than "toThrift". I'd keep the two parts (serializing to thrift and collecting stats) separate. http://gerrit.cloudera.org:8080/#/c/8529/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java File fe/src/main/java/org/apache/impala/catalog/HdfsTable.java: http://gerrit.cloudera.org:8080/#/c/8529/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@197 PS1, Line 197: al stats this seems to only be written, not read. can it be removed or will it be used for something? if its kept, then clarify what it means-- currently it looks like its flipped when some partition is incremental. http://gerrit.cloudera.org:8080/#/c/8529/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@220 PS1, Line 220: table or partition : // level. unclear what this is trying to say. is this: "aggregated table wide at the granularity of a partition"? http://gerrit.cloudera.org:8080/#/c/8529/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@256 PS1, Line 256: from nit: by http://gerrit.cloudera.org:8080/#/c/8529/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@1735 PS1, Line 1735: hasIncrementalStats.getRef() there's a lot going on here-- surprising to see that its tested given that its declared a few lines back. see the comment on toThrift in hdfsPartition. why not just get this bool from the partition? http://gerrit.cloudera.org:8080/#/c/8529/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@1737 PS1, Line 1737: hasIncrementalStats_ so incremental stats is a table-wide property and stored per partition? so some partitions can have incremental stats and others not? http://gerrit.cloudera.org:8080/#/c/8529/1/fe/src/main/java/org/apache/impala/util/TopNCache.java File fe/src/main/java/org/apache/impala/util/TopNCache.java: http://gerrit.cloudera.org:8080/#/c/8529/1/fe/src/main/java/org/apache/impala/util/TopNCache.java@35 PS1, Line 35: Always evict policy why is this needed? is it to reflect some sort of recency? http://gerrit.cloudera.org:8080/#/c/8529/1/fe/src/main/java/org/apache/impala/util/TopNCache.java@94 PS1, Line 94: . do you want to enforce a sort order here or at the caller? might be simpler here since you know the comparison function. http://gerrit.cloudera.org:8080/#/c/8529/1/www/catalog.tmpl File www/catalog.tmpl: http://gerrit.cloudera.org:8080/#/c/8529/1/www/catalog.tmpl@26 PS1, Line 26: Top-25 if parameterizing N, this would need to change as well. perhaps omit the N? http://gerrit.cloudera.org:8080/#/c/8529/1/www/catalog.tmpl@34 PS1, Line 34: several ws issues here. http://gerrit.cloudera.org:8080/#/c/8529/1/www/catalog.tmpl@65 PS1, Line 65: Top-25 same here http://gerrit.cloudera.org:8080/#/c/8529/1/www/catalog.tmpl@78 PS1, Line 78: {{#frequent_tables}} are these sorted (desc) by num operations? the screenshot for this one is not sorted. -- To view, visit http://gerrit.cloudera.org:8080/8529 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I37d407979e6d3b1a444b6b6265900b148facde9e Gerrit-Change-Number: 8529 Gerrit-PatchSet: 1 Gerrit-Owner: Dimitris Tsirogiannis <dtsirogian...@cloudera.com> Gerrit-Reviewer: Dimitris Tsirogiannis <dtsirogian...@cloudera.com> Gerrit-Reviewer: Philip Zeyliger <phi...@cloudera.com> Gerrit-Reviewer: Vuk Ercegovac <vercego...@cloudera.com> Gerrit-Comment-Date: Wed, 15 Nov 2017 19:37:09 +0000 Gerrit-HasComments: Yes
