pvary commented on issue #1617: URL: https://github.com/apache/iceberg/issues/1617#issuecomment-878961343
Thanks @jackye1995 for highlighting the properties which we need to handle! I have not know about the `write.object-storage.path` before. Do I understand correctly that this is a `LocationProvider` specific parameter for `ObjectStoreLocationProvider` which is more or less the same than the `write.object-storage.path` for the `DefaultLocationProvider`? If so it highlights how tricky could it be if we want to make sure that every `LocationProvider` specific parameter is handled correctly when we are updating the locations in the metadata json. Maybe we should delegate this path generation to the `LocationProvider` altogether, or something similar. > All updates to the 4 locations on the top can be done in a UpdateProperties + UpdateLocation transaction The process you have described above sounds useful to me if we have a place where both of the replicas are accessible, like a central node where we can initiate the copy of the data and then call UpdateProperties + UpdateLocation, so the replica could have it's own metadata json which holds the correct info. In our cases for the on-prem to cloud migration it is possible that the file copy is done by a different method and then we need to read and update the metadata json file, in which the locations are not accessible. So it would be good if we can run UpdateProperties + UpdateLocation on an "invalid" table. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
