dramaticlly commented on issue #13719: URL: https://github.com/apache/iceberg/issues/13719#issuecomment-3252112788
We had a constructive discussion about this issue on iceberg community sync [Youtube recording](https://www.youtube.com/watch?v=LB5wwL2TEgI) and here's the takeaway 1. With access to target table or ability to read last copied metadata.json for target table is prerequisite to provide incremental path rewrite. Currently we have a gap as existing rewrite table path Spark action solely on information available on source table. 2. When target table is not available, rewrite historical manifests are needed in order to have correct manifest size in target table after path rewrite, some but not all engines require correct manifest size in order to perform read on the iceberg table. User opt-in might be needed instead of making this a default operation. 3. Other alternatives like repair-manifest https://github.com/apache/iceberg/pull/10784 is also discussed, as it might be easier to directly repair the manifests of target table. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
