-0 This PR includes changes to add a v4 column for statistics files, but it does this in an odd way. First, as Steven pointed out, it changes from grouping v2 and v3 together to grouping v1 and v2 together. All 3 versions appear to be identical.
Next, the intent of adding the v4 column is to disallow the optional `key-metadata` field, but that would mean that statistics files from v3 cannot be carried forward into a v4 table without relocating the key to the table of encryption keys. That's doable, but it doesn't seem necessary to combine updating key management with updating the table spec version. Why not allow existing keys to reside where they are but deprecate the field? When writing this metadata as v4, I doubt that the upgrade will change the encryption scheme, so whatever is translating the metadata would still store the key from the statistics table unencrypted, just in a different place. I think that there is also a mismatch between the intent of the keys table and how key-metadata is used for stats files today. Right now, there is a key-metadata per stats file, but the keys table is intended to have one or two keys that are reused, not a key per encrypted file referenced from the table metadata. Rather than require per-file keys to be moved, I think we should just let the existing per-file keys remain where they are and add an optional key-id for a better way of managing the keys instead. On Tue, Aug 11, 2026 at 11:31 AM Steven Wu <[email protected]> wrote: > +1 (binding) > > On Tue, Aug 11, 2026 at 11:25 AM Alexander Bailey <[email protected]> > wrote: > >> +1 >> >> Thanks, >> Xander >> >> On Tue, 11 Aug 2026 at 19:01, Anurag Mantripragada < >> [email protected]> wrote: >> >>> +1 (non-binding) >>> >>> On Tue, Aug 11, 2026 at 8:57 AM Russell Spitzer < >>> [email protected]> wrote: >>> >>>> +1 >>>> >>>> On Tue, Aug 11, 2026 at 1:42 AM Gidon Gershinsky <[email protected]> >>>> wrote: >>>> >>>>> +1 >>>>> >>>>> Cheers, Gidon >>>>> >>>>> >>>>> On Mon, Aug 10, 2026 at 10:36 PM Gábor Kaszab <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi All, >>>>>> >>>>>> The current spec says that the encryption key for table statistics >>>>>> should be stored as raw key-metadata. However, since the table statistics >>>>>> metadata is stored within the unencrypted table metadata, it's not secure >>>>>> to follow the spec here. >>>>>> >>>>>> The proposal is two fold: >>>>>> 1) Deprecate the 'key-metadata' field in table statistics (Note, >>>>>> partition statistics doesn't have this field in the spec) >>>>>> 2) Add encryption 'key-id' field to table and partition statistics. >>>>>> This points to an encrypted encryption key stored in Table >>>>>> metadata's 'encryption-keys' (that in turn points to the KEK in the same >>>>>> list). This behaviour is similar to how we do the same for manifest list >>>>>> encryption keys. >>>>>> >>>>>> PR: https://github.com/apache/iceberg/pull/17533 >>>>>> >>>>>> Best Regards, >>>>>> Gabor Kaszab >>>>>> >>>>>
