Hi! If I understand Andres' message correctly - the proposition is to make use of compression dictionaries automatic, possibly just setting a parameter when the table is created, something like CREATE TABLE t ( ..., t JSONB USE DICTIONARY); The question is then how to create such dictionaries automatically and extend them while data is being added to the table. Because it is not something unusual when after a time circumstances change and a rather small table is started to be loaded with huge amounts of data.
I prefer extending a dictionary over re-creating it because while dictionary is recreated we leave users two choices - to wait until dictionary creation is over or to use the old version (say, kept as as a snapshot while a new one is created). Keeping many versions simultaneously does not make sense and would extend DB size. Also, compressing small data with a large dictionary (the case for one-for-many tables dictionary), I think, would add some considerable overhead to the INSERT/UPDATE commands, so the most reasonable choice is a per-table dictionary. Am I right? Any ideas on how to create and extend such dictionaries automatically? On Thu, Feb 9, 2023 at 2:01 PM Andres Freund <and...@anarazel.de> wrote: > Hi, > > On February 9, 2023 2:50:57 AM PST, Aleksander Alekseev < > aleksan...@timescale.com> wrote: > >Hi Andres, > > > >> > So to clarify, are we talking about tuple-level compression? Or > >> > perhaps page-level compression? > >> > >> Tuple level. > > > >> although my own patch proposed attribute-level compression, not > >> tuple-level one, it is arguably closer to tuple-level approach than > >> page-level one > > > >Just wanted to make sure that by tuple-level we mean the same thing. > > > >When saying tuple-level do you mean that the entire tuple should be > >compressed as one large binary (i.e. similarly to page-level > >compression but more granularly), or every single attribute should be > >compressed separately (similarly to how TOAST does this)? > > Good point - should have been clearer. I meant attribute wise compression. > Like we do today, except that we would use a dictionary to increase > compression rates. > > Andres > -- > Sent from my Android device with K-9 Mail. Please excuse my brevity. > -- Regards, Nikita Malakhov Postgres Professional https://postgrespro.ru/
