Hi everyone I have been working on a more generic approach to compression and similar encoding/decoding tasks
I have put my current thinking on this in https://wiki.postgresql.org/wiki/Pg_encoding_context This is meant to provide generic support functionality for things like compression dictionaries for zstd and lz4, Avro schemas, huffman tables and vector quanttization base data to name a few I appreciate any feedback on the approach --- Hannu On Wed, Sep 30, 2026 at 8:20 AM Michael Paquier <[email protected]> wrote: > > On Mon, Sep 28, 2026 at 10:39:42PM -0700, Nikhil Kumar Veldanda wrote: > > v7 attached > > I was just looking at v7-0001 that wants to add the ToastCompressionId > to toast_external_data, and this feels half-baked due to the > inconsistency this brings with extsize and VARATT_EXTINFO_GET_EXTSIZE. > > Couldn't we do better here by normalizing more data from the existing > fields? Another could be the is_compressed state which is guessed > from a comparison between the raw size and the compressed size, > perhaps? > > Regarding v7-0002 and v7-0003, I am doubting the wisdom of tackling > the last-compression-bit issue for this release. The OID8 code has > already changed a lot of code, and maybe we should be conservative in > terms of the amount of the changes we do in this area for a single > release. By that, I mean to catch up on the refactoring pieces on > this thread once some dust has settled on HEAD and tackle this issue > around the time v21 opens up (Aka I'm preparing myself for these > dozens of agents to complain about the shape of the code already > committed, we'll see how it goes). > -- > Michael
