Re: [PR] Flink: Add extensibility support to IcebergSink for downstream composition [iceberg]
herbherbherb closed pull request #15316: Flink: Add extensibility support to IcebergSink for downstream composition URL: https://github.com/apache/iceberg/pull/15316 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Flink: Add extensibility support to IcebergSink for downstream composition [iceberg]
herbherbherb commented on PR #15316: URL: https://github.com/apache/iceberg/pull/15316#issuecomment-4128677956 > There are currently multiple changes for the IcebergSink. I don't think we would like to expose the internals of the Sink as it would bind us to these contracts which we would like to change in the near future. > > If you think this still would be a valuable addition, please prepare a proposal and start a dev list discussion around this. > > Thanks, Peter Thank you Peter for the feedback. I'll close this PR. I've broken the approach into three smaller, focused PRs that add plugin hooks rather than exposing sink internals or changing access modifiers. Each adds a single `@FunctionalInterface` callback set via the builder and no visibility changes to existing classes: 1. OutputFileFactoryProvider — custom `OutputFileFactory` creation (https://github.com/apache/iceberg/issues/15763 / https://github.com/apache/iceberg/pull/15764) 2. PostCommitHook — callback after successful Iceberg commit (https://github.com/apache/iceberg/issues/15768 / https://github.com/apache/iceberg/pull/15769) 3. CommitGate — deferred commits with `ListState` buffering (https://github.com/apache/iceberg/issues/15770 / https://github.com/apache/iceberg/pull/15771) All three are backward compatible with no behavioral change when the hooks are not set. Happy to start a dev list discussion if you'd prefer that before reviewing the individual PRs, thank you. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Flink: Add extensibility support to IcebergSink for downstream composition [iceberg]
pvary commented on PR #15316: URL: https://github.com/apache/iceberg/pull/15316#issuecomment-4069625607 There are currently multiple changes for the IcebergSink. I don't think we would like to expose the internals of the Sink as it would bind us to these contracts which we would like to change in the near future. If you think this still would be a valuable addition, please prepare a proposal and start a dev list discussion around this. Thanks, Peter -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Flink: Add extensibility support to IcebergSink for downstream composition [iceberg]
herbherbherb commented on PR #15316: URL: https://github.com/apache/iceberg/pull/15316#issuecomment-4068853318 This PR still needs a review from the community please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Flink: Add extensibility support to IcebergSink for downstream composition [iceberg]
github-actions[bot] commented on PR #15316: URL: https://github.com/apache/iceberg/pull/15316#issuecomment-4064308859 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Flink: Add extensibility support to IcebergSink for downstream composition [iceberg]
herbherbherb commented on PR #15316: URL: https://github.com/apache/iceberg/pull/15316#issuecomment-3900030194 > The general approach is first make your change to the current version (2.1), then once that is merged, cherry pick to the other supported versions (separate PRs for each version). ack, thank you -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Flink: Add extensibility support to IcebergSink for downstream composition [iceberg]
bryanck commented on PR #15316: URL: https://github.com/apache/iceberg/pull/15316#issuecomment-3900027992 The general approach is first make your change to the current version (2.1), then once that is merged, cherry pick to the other supported versions (separate PRs for each version). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
