amogh-jahagirdar commented on issue #10487: URL: https://github.com/apache/iceberg/issues/10487#issuecomment-2164155791
Thanks for reporting @madeirak . I think this is similar to https://github.com/apache/iceberg/issues/10234 but this particular issue also brings up a nuance in the "unreferenced partition spec" case. One problem imo is we should not allow dropping columns which are part of historical partition specs. Now in your case it's actually still not referenced since no data was actually written (just the metadata updates) so this is another case to think about. Looking at the PartitionSpec/TableMetadata builder code path, what happens is when the schema is updated, all of the historical partition specs and sort orders need to be updated for the new schema. When you attempt to add the column back it'll have a new id and that id won't align with what was on the historical partition spec; this means updating the historical spec which still referenced the old column will fail. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
