herbherbherb opened a new pull request, #15316:
URL: https://github.com/apache/iceberg/pull/15316
## Summary
- Add `CommittableMetadata` framework: a composition-based extension point
allowing downstream to
attach custom metadata to committables flowing through the sink pipeline
- Widen access modifiers on sink classes to enable downstream connector
implementations to compose
with (reference, wrap, delegate to) Iceberg's existing sink infrastructure
Resolves #15315.
## Changes
### New files (3, in both v2.0 and v2.1)
- `CommittableMetadata.java` — `Serializable` marker interface for custom
metadata
- `CommittableMetadataSerializer.java` — serializer interface for metadata
- `CommittableMetadataRegistry.java` — global registry (downstream calls
`register()` before
sink creation)
### Modified files (in both v2.0 and v2.1)
- `IcebergCommittable` — public class, public accessors, optional `@Nullable
metadata` field,
backward-compatible constructor chaining
- `IcebergCommittableSerializer` — writes boolean metadata flag + delegates
serialization
- `IcebergCommitter` — public class + constructor
- `IcebergSinkWriter` — public class + constructor
- `IcebergWriteAggregator` — public class + constructor
- `IcebergSinkBuilder` — public interface
- `IcebergSink` — protected constructor, protected getters, protected
`Builder()`
- `CachingTableSupplier` — public class + constructor
- `RowDataTaskWriterFactory` — protected getters for
spec/format/outputFileFactory
- `SinkUtil` — public `checkAndGetEqualityFieldIds()`
### Backward compatibility
- All existing constructors still work (new `IcebergCommittable` constructor
chains with
`metadata = null`)
- Serialization is backward-compatible (boolean flag distinguishes metadata
presence)
- No behavioral changes to any existing code path
## Test plan
- [ ] Existing `TestIcebergCommitter` passes
- [ ] Existing `TestIcebergSink*` tests pass
- [ ] Serialization backward compatibility: old serialized data (no metadata
flag) still
deserializes correctly
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
