sqd commented on code in PR #15433:
URL: https://github.com/apache/iceberg/pull/15433#discussion_r3017991856
##########
docs/docs/flink-writes.md:
##########
@@ -483,11 +483,10 @@ We need the following information (DynamicRecord) for
every record:
| `Schema` | The schema of the record.
|
| `Spec` | The expected partitioning specification for the record.
|
| `RowData` | The actual row data to be written.
|
-| `DistributionMode` | The distribution mode for writing the record (currently
supports NONE or HASH). |
+| `DistributionMode` | The distribution mode for writing the record (NONE,
HASH or optional). When unspecified, the record won't be shuffled at all. |
Review Comment:
Fixed
##########
docs/docs/flink-writes.md:
##########
@@ -547,6 +546,28 @@ The Dynamic Iceberg Flink Sink is configured using the
Builder pattern. Here are
| `tableCreator(TableCreator creator)` | When DynamicIcebergSink creates new
Iceberg tables, allows overriding how tables are created - setting custom table
properties and location based on the table name. |
| `dropUnusedColumns(boolean enabled)` | When enabled, drops
all columns from the current table schema which are not contained in the input
schema (see the caveats above on dropping columns). |
+### Distribution Modes
+
+The `DistributionMode` set on each `DynamicRecord` controls how that record is
routed from the processor to the writer:
+
+| Mode | Behavior |
+|---------------|----------|
+| `NONE` | Records are distributed across writer subtasks in a
round-robin fashion (or by equality fields if set). |
+| `HASH` | Records are distributed by partition key (partitioned
tables) or equality fields (unpartitioned tables). Ensures that records for the
same partition are handled by the same writer subtask. |
+| (unspecified) | Forward mode: bypasses distribution entirely and sends
records directly via a forward edge (see below). |
Review Comment:
Fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
