Nandor Kollar has uploaded this change for review. ( http://gerrit.cloudera.org:8080/24614
Change subject: IMPALA-12587: Respect MAX_FS_WRITERS for Iceberg DELETE/UPDATE/MERGE ...................................................................... IMPALA-12587: Respect MAX_FS_WRITERS for Iceberg DELETE/UPDATE/MERGE Previously, MAX_FS_WRITERS only capped parallelism for HdfsTableSink (INSERT). Iceberg DELETE (IcebergBufferedDeleteSink), UPDATE and MERGE (MultiDataSink) ignored the limit, potentially creating too many small files. This patch: - Adds maxTableSinks_ to IcebergBufferedDeleteSink with a getNumInstances() method that caps writer instances. - Adds getNumInstances() to MultiDataSink that delegates to child sinks. - Introduces the InstanceCountProvidingSink interface to eliminate the instanceof cascade in PlanFragment.getNumInstances(). - Simplifies MultiDataSink construction with a varargs constructor. Change-Id: Ice7362cecb8b43fbff43b8ad82827e31e2b4eea3 Assisted-by: Claude Opus 4.6 (Claude Code) --- M fe/src/main/java/org/apache/impala/analysis/IcebergDeleteImpl.java M fe/src/main/java/org/apache/impala/analysis/IcebergMergeImpl.java M fe/src/main/java/org/apache/impala/analysis/IcebergUpdateImpl.java M fe/src/main/java/org/apache/impala/planner/HdfsTableSink.java M fe/src/main/java/org/apache/impala/planner/IcebergBufferedDeleteSink.java M fe/src/main/java/org/apache/impala/planner/IcebergMergeSink.java A fe/src/main/java/org/apache/impala/planner/InstanceCountProvidingSink.java M fe/src/main/java/org/apache/impala/planner/JoinBuildSink.java M fe/src/main/java/org/apache/impala/planner/MultiDataSink.java M fe/src/main/java/org/apache/impala/planner/PlanFragment.java A fe/src/test/java/org/apache/impala/planner/IcebergWriterLimitTest.java 11 files changed, 252 insertions(+), 42 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/24614/1 -- To view, visit http://gerrit.cloudera.org:8080/24614 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ice7362cecb8b43fbff43b8ad82827e31e2b4eea3 Gerrit-Change-Number: 24614 Gerrit-PatchSet: 1 Gerrit-Owner: Nandor Kollar <[email protected]>
