Hi hackers! I've prepared a patch with a 64-bit TOAST Value ID. It is a kind of prototype and needs some further work, but it is already working and ready to play with.
I've introduced 64-bit value ID field in varatt_external, but to keep it compatible with older bases 64-bit value is stored as a structure of two 32-bit fields and value ID moved to be the last in the varatt_external structure. Also, I've introduced different ID assignment function - instead of using global OID assignment function I use individual counters for each TOAST table, automatically cached and after server restart when new value is inserted into TOAST table maximum used value is searched and used to assign the next one. >Andres Freund: >I think we'll need to do something about the width of varatt_external to make >the conversion to 64bit toast oids viable - and if we do, I don't think >there's a decent argument for staying with 4 byte toast OIDs. I think the >varatt_external equivalent would end up being smaller in just about all cases. >And as you said earlier, the increased overhead inside the toast table / index >is not relevant compared to the size of toasted datums. There is some small overhead due to indexing 64-bit values. Also, for smaller tables we can use 32-bit ID instead of 64-bit, but then we would have a problem when we reach the limit of 2^32. Pg_upgrade seems to be not a very difficult case if we go keeping old TOAST tables using 32-bit values, Please have a look. I'd be grateful for some further directions. GIT branch with this feature, rebased onto current master: https://github.com/postgrespro/postgres/tree/toast_64bit On Wed, Dec 7, 2022 at 12:00 AM Nikita Malakhov <huku...@gmail.com> wrote: > Hi hackers! > > Here's some update on the subject - I've made a branch based on master from > 28/11 and introduced 64-bit TOAST value ID there. It is not complete now > but > is already working and has some features: > - extended structure for TOAST pointer (varatt_long_external) with 64-bit > TOAST value ID; > - individual ID counters for each TOAST table, being cached for faster > access > and lookup of maximum TOAST ID in use after server restart. > https://github.com/postgrespro/postgres/tree/toast_64bit > > -- > Regards, > Nikita Malakhov > Postgres Professional > https://postgrespro.ru/ > -- Regards, Nikita Malakhov Postgres Professional https://postgrespro.ru/
0001_64_bit_toast_valueid_v1.patch
Description: Binary data