jackye1995 commented on PR #6742: URL: https://github.com/apache/iceberg/pull/6742#issuecomment-1416812241
Nice, I anticipated similar concerns as in that thread, that's why I'd like to just put it up and see how the community reacts to this. I think the conversation there was around the fact that `registerTable` is used for a recovery use case, so there is no requirement for atomic operation and the original metadata location might be broken. What Theo is trying to achieve (based on my understanding) is a use case of continuous registration, where a user sends notification of the latest metadata location, and then the metadata location is updated for an existing table in Glue. This is trying to help with migration use cases of for example `HiveCatalog` or `HadoopCatalog` to `GlueCatalog`. In this use case, atmoic switch of metadata location is a requirement, compared to a drop + register case. And it can clearly be achieved through calling `glue.updateTable` with all the right information. I think we would like to know if that is something worth adding to the Iceberg OSS, or it needs to just remain as a custom logic. I think it has benefit in OSS, as people naturally think of `registerTable` in this use case, and as we have so many catalog offerings, it's worth supporting cross-catalog migration explicitly. Any thoughts? @RussellSpitzer @yabola @flyrain @szehon-ho @rdblue -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
