Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/24712 )
Change subject: IMPALA-13314: Cache HadoopCatalog instances and use holder pattern for singletons ...................................................................... IMPALA-13314: Cache HadoopCatalog instances and use holder pattern for singletons Previously, a new HadoopCatalog (and its FileIO/filesystem resources) was created on every table operation via getIcebergCatalog(). This caused excessive resource consumption when many Iceberg tables shared the same catalog location. Changes: - Introduce a Guava Cache with soft values in IcebergHadoopCatalog keyed by catalog location string. Tables sharing the same iceberg.catalog_location now reuse a single HadoopCatalog instance. Entries are evicted under memory pressure via soft references. - Replace synchronized lazy initialization with the initialization-on-demand holder idiom for IcebergHiveCatalog, IcebergHadoopTables, and IcebergCatalogUtil. The JVM guarantees thread-safe lazy initialization without synchronization overhead on every access. - Convert getIcebergCatalog() to a switch expression, removing the throws ImpalaRuntimeException declaration. Simplify loadTable() by dropping the now-unnecessary try-catch. - Remove unused getLastComputeStatsTimeMs() and collectFieldIdsWithNdvStats() from IcebergTable. - Fix logger in IcebergHadoopCatalog (was incorrectly using IcebergHadoopTables.class). - Make hadoopTables field final in IcebergHadoopTables. Testing: Ran existing Iceberg catalog tests. Assisted-by: Claude Opus 4.6 (Claude Code) Change-Id: Ibb3a6c8e4f1d2a9e7c5b0f8d3e6a4c2b1d0e9f7a Reviewed-on: http://gerrit.cloudera.org:8080/24712 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java M fe/src/main/java/org/apache/impala/catalog/iceberg/IcebergCatalogUtil.java M fe/src/main/java/org/apache/impala/catalog/iceberg/IcebergHadoopCatalog.java M fe/src/main/java/org/apache/impala/catalog/iceberg/IcebergHadoopTables.java M fe/src/main/java/org/apache/impala/catalog/iceberg/IcebergHiveCatalog.java M fe/src/main/java/org/apache/impala/util/IcebergUtil.java 6 files changed, 56 insertions(+), 70 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/24712 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ibb3a6c8e4f1d2a9e7c5b0f8d3e6a4c2b1d0e9f7a Gerrit-Change-Number: 24712 Gerrit-PatchSet: 7 Gerrit-Owner: Nandor Kollar <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Nandor Kollar <[email protected]> Gerrit-Reviewer: Peter Rozsa <[email protected]>
