amogh-jahagirdar commented on issue #15108: URL: https://github.com/apache/iceberg/issues/15108#issuecomment-3785686712
Metadata.JSONs are not required to contain the complete history of all partition specs, but they should contain the partition specs that are part of the active table state (otherwise I'd consider that corrupted table state). Maintenance like expire snapshots can remove older specs if they're no longer used in any active manifests in the table after the expiration (but again those are not active). e.g. as part of expiration we'll detect that there are no more manifests with spec 1, we'll just remove it. It looks like iceberg-go is tracking off some assumptions on the spec ID always being there and indexing, but that's not a correct way of tracking things imo. I think it should be tracking from a map<spec id, spec>. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
