funcbro commented on issue #14606: URL: https://github.com/apache/iceberg/issues/14606#issuecomment-4505167329
I took some more time digging into the `rewriteTablePath` source. Currently there does not seem be any way to migrate the manifests, if the original source-path is not available. Independent if loading the table via the catalog or manually via `StaticTableOperations`, the issue is still the same: Already when loading a table Iceberg tries to read all metadata and manifests of the table, which can only fail, because the refereneces are simply not valid anymore. This only seems to be fixable in the load-table operation itself, **before** rewriteTablePath is actually executed - or in a lazy-load operation within rewriteTablePath, which allows adapting the paths while processing. Now seeing progress in https://github.com/apache/iceberg/issues/13141 I wonder, if rewriteTablePath will be fixed/enabled when moving to spec v4? Obviously moving to relative-paths will solve some of the mentioned issues, but still we will need at least an option to migrate pre-v4-spec tables, where we will again talk about rewriteTablePath i guess? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
