clairemcginty commented on PR #39645: URL: https://github.com/apache/beam/pull/39645#issuecomment-5218629601
> > there isn't currently a way to do accomplish the same behavior using IcebergIO > > I think the missing piece is a `withTableProperties(...)` option in IcebergIO. My recommendation based on effort level: > > 1. Simply recommend using Managed.ICEBERG, which enumerates more options like `table_properties` and `write_properties` > 2. Add `withTableProperties(...)` to IcebergIO for parity > > Frankly I think `executionProperties` makes sense, but I wouldn't change the `writeProperties` naming, mostly because it aligns with the naming in Spark and Flink > Sounds good! > > there isn't currently a way to do accomplish the same behavior using IcebergIO > > I think the missing piece is a `withTableProperties(...)` option in IcebergIO. My recommendation based on effort level: > > 1. Simply recommend using Managed.ICEBERG, which enumerates more options like `table_properties` and `write_properties` > 2. Add `withTableProperties(...)` to IcebergIO for parity > > Frankly I think `executionProperties` makes sense, but I wouldn't change the `writeProperties` naming, mostly because it aligns with the naming in Spark and Flink > Makes sense! Updated the PR to revert the changes and just document the `.withWriteProperties` option. I also kept the ManagedIO test because I found it helpful to have an example of all these config properties working together, but can revert that too if you find it unnecessary! > P.S. Looks like the wrong Ahmed got tagged by mistake 😅 Ahhh so sorry, I autocompleted too hastily 😅 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
