Michael Smith has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/23547 )
Change subject: IMPALA-14502: Not tracking metrics in IncompleteTable ...................................................................... IMPALA-14502: Not tracking metrics in IncompleteTable Tables that are in unloaded state are represented as IncompleteTable. Table level metrics of them won't be used at all but occupy around 7KB of memory for each table. This is a significant amount comparing to the table name strings. This patch skips initializing these metrics for IncompleteTable to save memory usage. This reduces the initial memory requirement to launch catalogd. To avoid other codes unintentionally add new metrics to IncompleteTable, overrides all Table methods that use metrics_ to return simple results, e.g. IncompleteTable.getMedianTableLoadingTime() always returns 0. IncompleteTable.getMetrics() shouldn't be used. Added a Precondition check for this. Tests: - Verified in a heap dump file after loading 1.3M IncompleteTables that the heap usage reduces to 2GB and only few instances of com.codahale.metrics.Timer are created. Previously catalogd OOM in a heap size of 18GB when running global IM, and the number of com.codahale.metrics.Timer instances is similar to the number of IncompleteTables. - Passed CORE tests. Change-Id: If0fcfeab99bbfbefe618d0abf7f2482a0cc5ef9f Reviewed-on: http://gerrit.cloudera.org:8080/23547 Reviewed-by: Riza Suminto <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> Reviewed-by: Michael Smith <[email protected]> --- M fe/src/main/java/org/apache/impala/catalog/IncompleteTable.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M fe/src/main/java/org/apache/impala/catalog/events/RenameTableBarrierEvent.java 3 files changed, 49 insertions(+), 2 deletions(-) Approvals: Riza Suminto: Looks good to me, but someone else must approve Impala Public Jenkins: Verified Michael Smith: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/23547 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: If0fcfeab99bbfbefe618d0abf7f2482a0cc5ef9f Gerrit-Change-Number: 23547 Gerrit-PatchSet: 4 Gerrit-Owner: Quanlong Huang <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]>
