Hi hackers! We need community feedback on previously discussed topic [1]. There are some long-live issues in Postgres related to the TOAST mechanics, like [2]. Some time ago we already proposed a set of patches with an API allowing to plug in different TOAST implementations into a live database. The patch set introduced a lot of code and was quite crude in some places, so after several implementations we decided to try to implement it in the production environment for further check-up.
The main idea behind pluggable TOAST is make it possible to easily plug in and use different implementations of large values storage, preserving existing mechanics to keep backward compatibilitну provide easy Postgres-way give users alternative mechanics for storing large column values in a more effective way - we already have custom and very effective (up to tens and even hundreds of times faster) TOAST implementations for bytea and JSONb data types. As we see it - Pluggable TOAST proposes 1) changes in TOAST pointer itself, extending it to store custom data - current limitations of TOAST pointer were discussed in [1] and [4]; 2) API which allows calls of custom TOAST implementations for certain table columns and (a topic for discussion) certain datatypes. Custom TOAST could be also used in a not so trivial way - for example, limited columnar storage could be easily implemented and plugged in without heavy core modifications of implementation of Pluggable Storage (Table Access Methods), preserving existing data and database structure, be upgraded, replicated and so on. Any thoughts and proposals are welcome. [1] Pluggable TOAST https://www.postgresql.org/message-id/flat/224711f9-83b7-a307-b17f-4457ab73aa0a%40sigaev.ru [2] Infinite loop while acquiring new TOAST Oid https://www.postgresql.org/message-id/flat/CAN-LCVPRvRzxeUdYdDCZ7UwZQs1NmZpqBUCd%3D%2BRdMPFTyt-bRQ%40mail.gmail.com [3] JSONB Compression dictionaries https://www.postgresql.org/message-id/flat/CAJ7c6TOtAB0z1UrksvGTStNE-herK-43bj22%3D5xVBg7S4vr5rQ%40mail.gmail.com [4] Extending the TOAST pointer https://www.postgresql.org/message-id/flat/CAN-LCVMq2X%3Dfhx7KLxfeDyb3P%2BBXuCkHC0g%3D9GF%2BJD4izfVa0Q%40mail.gmail.com -- Regards, Nikita Malakhov Postgres Professional The Russian Postgres Company https://postgrespro.ru/
