Hi everyone, I think we’ve reached the point where it’s time to finalize and adopt the changes for Iceberg v3. We’ve been working toward this for the last few months and have now implemented the v3 features in the Java library to reduce the risk of needing changes or hitting problems (row lineage support in Spark 3.5 just went in!). We’ve also incorporated some clarifications and minor changes back into the spec from what we’ve learned.
At this point, I’m confident that the spec is reasonable and correct. Thank you to everyone working on these reference implementations! The next step is to discuss any outstanding items or concerns about moving forward, and then to have a vote thread to adopt the spec. I’ll start off with a couple of items: One potential concern is that the upstream Variant spec hasn’t yet been finalized by the Parquet community, but we’ve built a full, independent implementation in Iceberg to validate the spec. I think the Parquet community is primarily waiting on getting the PRs in to have a Java reference implementation, so the risk of changes to the Variant spec is small. There’s also an on-going vote to add encryption keys in support of full table encryption that I think we want to get in. Any other items we may want to clear up? Ryan
