Hello Csaba Ringhofer, Impala Public Jenkins,
I'd like you to do a code review. Please visit
http://gerrit.cloudera.org:8080/19008
to review the following change.
Change subject: IMPALA-11580: Fix memory leak in legacy catalog mode when
applying incremental partition updates
......................................................................
IMPALA-11580: Fix memory leak in legacy catalog mode when applying incremental
partition updates
In the legacy catalog mode, catalogd propagates incremental metadata
updates at the partition level. While applying the updates, impalad
reuses the existing partition objects and moves them to a new HdfsTable
object. However, the partition objects are immutable, which means their
reference to the old table object remains unchanged. JVM GC cannot
collect the stale table objects since they still have active reference
from the partitions, which results in memory leak.
This patch fixes the issue by recreating a new partition object based on
the existing partition object with the new table field.
Tests:
- Verified locally that after applying the patch, I don’t see the
number of live HdfsTable objects keeps bumping.
Change-Id: Ie04ff243c6b82c1a06c489da74353f2d8afe423a
Reviewed-on: http://gerrit.cloudera.org:8080/18978
Tested-by: Impala Public Jenkins <[email protected]>
Reviewed-by: Csaba Ringhofer <[email protected]>
---
M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java
M fe/src/main/java/org/apache/impala/catalog/ImpaladCatalog.java
2 files changed, 11 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/08/19008/1
--
To view, visit http://gerrit.cloudera.org:8080/19008
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: branch-4.1.1
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ie04ff243c6b82c1a06c489da74353f2d8afe423a
Gerrit-Change-Number: 19008
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <[email protected]>
Gerrit-Reviewer: Csaba Ringhofer <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
