Add support for external on-disk TOAST pointers as oid8 A TOAST table whose chunk_id column is oid8 is now fed a 64-bit object ID thanks to the introduction of a new vartag_external called VARTAG_ONDISK_OID8, coupled with a new varatt_external_oid8. The only difference between the existing varatt_external_oid and the new varatt_external_oid8 is an extra 4 bytes to store 8 bytes worth an object ID, spread across two uint32 to force alignement as memcmp may be used for TOAST pointer comparison (line varatt_external_oid).
There is a one-one mapping between the vartag_external and the type of a chunk_id: - An oid8 chunk_id implies the use of VARTAG_ONDISK_OID8. - An oid chunk_id implies the use of VARTAG_ONDISK_OID. There are a couple of changes worth noting: - In toast_tuple_init(), for the comparison of old and new external values with a memcmp over VARSIZE_EXTERNAL(old_value). Now that multiple types of external on-disk TOAST pointers are possible, we need to add a check based on the vartag used by an old or new value. One of the tests added by c68cba09dd7f cover this case. - In toast_tuple_find_biggest_attribute(), the initial "biggest_size" depends on the type of chunk_id. This needs a change in RelationData to track the chunk_id type; this cached data is filled on demand. - In detoast.h, toast_external_info_get() and toast_external_data act as a translation layer to retrieve all the data associated to an Oid or an Oid8 TOAST pointer, without the need to worry about the vartag dealt with. The introduction of this function and structure has reduced a lot of code churn that existed in the initial versions of this patch, as per a suggestion from Bharath Rupireddy. - amcheck is updated to check the vartag <-> chunk_id type mapping requirements, handling both vartags depending on the underlying type of chunk_id on the TOAST table. Bump catalog version, due to at least the change of VARTAG_SIZE(). (This counts for the bump forgotten in a625fc570c22..) Author: Michael Paquier <[email protected]> Reviewed-by: Greg Burd <[email protected]> Reviewed-by: Bharath Rupireddy <[email protected]> Reviewed-by: Yugo Nagata <[email protected]> Discussion: https://postgr.es/m/[email protected] Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/6e58d6356fcb58f06bfcebb5bfa808660e9531d2 Modified Files -------------- contrib/amcheck/verify_heapam.c | 85 ++++++--- doc/src/sgml/storage.sgml | 12 +- src/backend/access/common/detoast.c | 71 ++++--- src/backend/access/common/toast_compression.c | 8 +- src/backend/access/common/toast_internals.c | 238 ++++++++++++++---------- src/backend/access/heap/heaptoast.c | 2 +- src/backend/access/table/toast_helper.c | 22 ++- src/backend/replication/logical/reorderbuffer.c | 14 +- src/backend/utils/adt/varlena.c | 6 +- src/backend/utils/cache/relcache.c | 37 ++++ src/include/access/detoast.h | 48 +++++ src/include/access/heaptoast.h | 22 ++- src/include/catalog/catversion.h | 2 +- src/include/utils/rel.h | 9 + src/include/utils/relcache.h | 1 + src/include/varatt.h | 78 +++++--- src/tools/pgindent/typedefs.list | 2 + 17 files changed, 448 insertions(+), 209 deletions(-)
