Hi hackers, Jan
I would like to propose a design and prototype implementation for a new
TOAST format, which we are calling **Direct TOAST**.
Not a big change, just leaving out the index part.
Hopefully this gives TOAST another 25-30 years :)
I have been talking about the need of this at least since having "Fixing
the TOAST" section in my
https://www.pgcon.org/2023/schedule/session/469-postgresql-is-nowhere-near-ready-3-new-directions-to-develop-postgresql/index.html
talk
### A few Caveats:
- This patch is NOT for code review, just to verify the concept
- This patch came to be because I was tuning some pgvector stuff and the
TOAST overhead completely drawned out any tuning effects. So I asked Jetski
to implement Direct Toast
- this implements the simplest possible Direct Toast variant (I have
several more in mind :) )
- the code is fully AI-generated, I have not even looked at it so not meant
for any kind of *code* review (actually I took a very quick peek and saw
that some things are weird)
#### The good parts:
- The patch cleanly aopplies to REL18_STABLE, passes all tests and seems
generally stable
- The patch is under 1200 lines and likely can be shortened from that.
- The expected speedups are indeed there
- non-indexed TOP-N vector queries are 5% - 35% faster (see below)
- a million-rows heavily toasted table `COPY table TO '/dev/null' WITH
(FORMAT BINARY)`
- takes 31 sec with Direct Toast and 67 sec with Plain/old Toast
(with everything in memory, 5GB of shared buffers)
- `SELECT sum(length(t::text)) FROM toasttable AS t` is 22 sec with
direct and 36 sec with plain
- The OID exhaustion at 4B is obviously not there - you can not run out of
tuple ids
As expected, bigger gains show up with larger tables where TOAST index is
of significant size
I plan to run also the pessimal case test where the TOAST index does not
fit even in Linux disk cache and then almost each row may force a disk
access
## Goals
The main goals of this proposal are to significantly improve TOAST write
and read performance, and to address the issue of OID counter exhaustion in
tables with billions of toasted entries.
### The Problem with Plain TOAST
Currently, Plain TOAST relies on logical OIDs (`chunk_id`) to link parent
tuples to toast chunks.
- **On Write**: Generating a unique OID requires scanning the index
(`GetNewOidWithIndex`), and inserting chunks requires inserting entries
into the unique index, causing index write amplification.
- **On Read**: Retrieving a toasted value always requires a B-Tree index
scan on the toast table.
- **OID Exhaustion**: Every toasted value consumes an OID from the shared
32-bit global counter. For systems inserting billions of toasted entries,
this leads to rapid OID wraparound.
### High-Level Design of Direct TOAST
Direct TOAST optimizes this by a new toast pointer type `VARTAG_DIRECT
storing physical Tuple IDs (TIDs) instead of logical OIDs where possible:
1. **Toast Pointer (`VARTAG_DIRECT`)**: The parent relation stores the TID
of the *final* chunk of the toasted value directly in its toast pointer.
2. **Multi-Page Chunks (`tid[]`)**: For toasted values spanning multiple
pages, the final chunk stores an array of TIDs (`chunk_tids` of type
`tid[]`) pointing to all preceding chunks. Any page referenced in the array
can recursively contain its own `tid[]` array, allowing hierarchical tree
structures.
3. **Zero-OID Chunks**: Direct TOAST chunks are written with `chunk_id =
InvalidOid (0)`.
4. **Index Skip on Write**: We completely skip inserting direct TOAST
chunks into the toast table's unique index during writes.
5. **Partial Index**: The toast table's primary key index is created as a
partial index: `UNIQUE INDEX ... WHERE chunk_id <> 0`. This excludes direct
chunks from the index, allowing `REINDEX` to rebuild the index from the
heap without unique constraint violations on duplicate `(0, seq)` entries.
### Advantages
- **Write Performance**: Bypassing OID generation and index inserts yields
significant write path speedups and reduces WAL volume.
- **Read Performance**: Single-page toasted values are retrieved directly
via TID in a single heap fetch, completely bypassing the index. Multi-page
values bypass index lookups for all but the root chunk.
- **No OID Exhaustion**: By using `chunk_id = 0` for direct chunks, we
completely stop consuming OIDs for these entries.
#### Speedup of TOP-N vector queries
For the folowing test query
SELECT * FROM embtabof<DIM> ORDER BY embedding <-> (SELECT embedding from
e384 limit 1) LIMIT N;
The results for plain and direct toast are as follows
+------+---------+---------------+---------+---------+----------+
| Dim | Rows | Table Size | LIM 10 | LIM 100 | LIM 1000 |
+------+---------+---------------+---------+---------+----------+
| 768 | 598,875 | 2,516,844,544 | 1,883 | 1,885 | 1,898 |
| | Direct Toast --> | 1,413 | 1,418 | 1,427 |
| | Speedup --> | 25% | 25% | 25% |
| | | | | | |
| 1536 | 272,750 | 2,276,147,200 | 1,605 | 1,611 | 1,620 |
| | Direct Toast --> | 1,381 | 1,378 | 1,375 |
| | | | 14% | 14% | 15% |
| | | | | | |
| 5376 | 110,762 | 2,284,707,840 | 949 | 1,002 | 1,015 |
| | Direct Toast --> | 893 | 890 | 887 |
| | | | 6% | 11% | 13% |
+------+---------+---------------+---------+---------+----------+
(Top row is Plain Toast)
### Current Implementation Status
I have a working prototype with code written by Jetski against the
`REL_18_STABLE` branch.
- Read/Write paths are fully implemented and integrated.
- Delete path handles recursive deletion of TIDs during parent tuple
cleanup.
- Added GUC `toast_flavour` and table-level storage parameter
`toast_flavour = 'direct' | 'plain'` to control the format.
- All regression tests compile and pass.
### Limitations & Trade-offs
- **No Manual `CLUSTER` on Toast Tables**: Because the toast index is
partial, running `CLUSTER pg_toast.pg_toast_xxx` directly on the toast
table is not supported (fails with `cannot cluster on partial index`).
However, `CLUSTER` on the parent table works normally as it rebuilds the
toast table by swapping files.
- **TID Dependency**: If toast rows are moved (e.g. by `VACUUM FULL`), the
TIDs change. The prototype relies on PostgreSQL's internal table rebuilding
mechanisms to update these pointers during such operations.
### Future plans
There are many more things that can be done once we move to new structure
- substring replacment (would allow replacing full large Large Object
functionality)
- flexible compression - adding info for compression on the TOAST side is
no not limited by the two available bits in toast pointer
- partially updated structured types (JSON, BSON, ...)
### Initially I would love to get discussion going on this design,
particularly regarding:
- The partial index approach to allow duplicate `chunk_id = 0` in the heap.
- The use of `tid[]` arrays for multi-page storage.
- Any concerns about the TID dependency.
- what other changes are needed in addition to disabling direct VACUUM FULL
on TOAST table ?
Regards,
Hannu
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index 62651787742..94988e46d85 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -19,6 +19,7 @@
#include "access/toast_internals.h"
#include "common/int.h"
#include "common/pg_lzcompress.h"
+#include "utils/array.h"
#include "utils/expandeddatum.h"
#include "utils/rel.h"
@@ -29,6 +30,15 @@ static struct varlena *toast_fetch_datum_slice(struct varlena *attr,
static struct varlena *toast_decompress_datum(struct varlena *attr);
static struct varlena *toast_decompress_datum_slice(struct varlena *attr, int32 slicelength);
+static struct varlena *toast_fetch_datum_direct(struct varlena *attr);
+static struct varlena *toast_fetch_datum_slice_direct(struct varlena *attr,
+ int32 sliceoffset,
+ int32 slicelength);
+static void toast_fetch_datum_direct_slice_recursive(Relation toastrel, ItemPointer tid,
+ struct varlena *result, int32 *logical_offset,
+ int32 sliceoffset, int32 slicelength,
+ TupleTableSlot *slot);
+
/* ----------
* detoast_external_attr -
*
@@ -53,6 +63,10 @@ detoast_external_attr(struct varlena *attr)
*/
result = toast_fetch_datum(attr);
}
+ else if (VARATT_IS_EXTERNAL_DIRECT(attr))
+ {
+ result = toast_fetch_datum_direct(attr);
+ }
else if (VARATT_IS_EXTERNAL_INDIRECT(attr))
{
/*
@@ -130,6 +144,18 @@ detoast_attr(struct varlena *attr)
pfree(tmp);
}
}
+ else if (VARATT_IS_EXTERNAL_DIRECT(attr))
+ {
+ attr = toast_fetch_datum_direct(attr);
+ /* If it's compressed, decompress it */
+ if (VARATT_IS_COMPRESSED(attr))
+ {
+ struct varlena *tmp = attr;
+
+ attr = toast_decompress_datum(tmp);
+ pfree(tmp);
+ }
+ }
else if (VARATT_IS_EXTERNAL_INDIRECT(attr))
{
/*
@@ -264,6 +290,29 @@ detoast_attr_slice(struct varlena *attr,
else
preslice = toast_fetch_datum(attr);
}
+ else if (VARATT_IS_EXTERNAL_DIRECT(attr))
+ {
+ struct varatt_direct toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER_DIRECT(toast_pointer, attr);
+
+ /* fast path for non-compressed external datums */
+ if (!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
+ return toast_fetch_datum_slice_direct(attr, sliceoffset, slicelength);
+
+ if (slicelimit >= 0)
+ {
+ int32 max_size = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
+
+ if (VARATT_EXTERNAL_GET_COMPRESS_METHOD(toast_pointer) ==
+ TOAST_PGLZ_COMPRESSION_ID)
+ max_size = pglz_maximum_compressed_size(slicelimit, max_size);
+
+ preslice = toast_fetch_datum_slice_direct(attr, 0, max_size);
+ }
+ else
+ preslice = toast_fetch_datum_direct(attr);
+ }
else if (VARATT_IS_EXTERNAL_INDIRECT(attr))
{
struct varatt_indirect redirect;
@@ -555,6 +604,13 @@ toast_raw_datum_size(Datum value)
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
result = toast_pointer.va_rawsize;
}
+ else if (VARATT_IS_EXTERNAL_DIRECT(attr))
+ {
+ struct varatt_direct toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER_DIRECT(toast_pointer, attr);
+ result = toast_pointer.va_rawsize;
+ }
else if (VARATT_IS_EXTERNAL_INDIRECT(attr))
{
struct varatt_indirect toast_pointer;
@@ -615,6 +671,13 @@ toast_datum_size(Datum value)
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
result = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
}
+ else if (VARATT_IS_EXTERNAL_DIRECT(attr))
+ {
+ struct varatt_direct toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER_DIRECT(toast_pointer, attr);
+ result = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
+ }
else if (VARATT_IS_EXTERNAL_INDIRECT(attr))
{
struct varatt_indirect toast_pointer;
@@ -644,3 +707,277 @@ toast_datum_size(Datum value)
}
return result;
}
+
+/*
+ * Reconstruct a Datum from direct TOAST pointers.
+ */
+static struct varlena *
+toast_fetch_datum_direct(struct varlena *attr)
+{
+ struct varatt_direct toast_pointer;
+ int32 attrsize;
+ struct varlena *result;
+ Relation toastrel;
+ TupleTableSlot *slot;
+ int32 logical_offset = 0;
+
+ if (!VARATT_IS_EXTERNAL_DIRECT(attr))
+ elog(ERROR, "toast_fetch_datum_direct shouldn't be called for non-direct datums");
+
+ VARATT_EXTERNAL_GET_POINTER_DIRECT(toast_pointer, attr);
+
+ attrsize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
+
+
+
+ result = (struct varlena *) palloc(attrsize + VARHDRSZ);
+
+ if (VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
+ SET_VARSIZE_COMPRESSED(result, attrsize + VARHDRSZ);
+ else
+ SET_VARSIZE(result, attrsize + VARHDRSZ);
+
+ if (attrsize == 0)
+ return result;
+
+ toastrel = table_open(toast_pointer.va_toastrelid, AccessShareLock);
+ slot = table_slot_create(toastrel, NULL);
+
+ toast_fetch_datum_direct_slice_recursive(toastrel, &toast_pointer.va_tid,
+ result, &logical_offset,
+ 0, attrsize, slot);
+
+ ExecDropSingleTupleTableSlot(slot);
+ table_close(toastrel, AccessShareLock);
+
+ return result;
+}
+
+static struct varlena *
+toast_fetch_datum_slice_direct(struct varlena *attr, int32 sliceoffset,
+ int32 slicelength)
+{
+ struct varatt_direct toast_pointer;
+ int32 attrsize;
+ struct varlena *result;
+ Relation toastrel;
+ TupleTableSlot *slot;
+ int32 logical_offset = 0;
+
+ if (!VARATT_IS_EXTERNAL_DIRECT(attr))
+ elog(ERROR, "toast_fetch_datum_slice_direct shouldn't be called for non-direct datums");
+
+ VARATT_EXTERNAL_GET_POINTER_DIRECT(toast_pointer, attr);
+
+ attrsize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
+
+ if (sliceoffset >= attrsize)
+ {
+ sliceoffset = 0;
+ slicelength = 0;
+ }
+
+ if (((sliceoffset + slicelength) > attrsize) || slicelength < 0)
+ slicelength = attrsize - sliceoffset;
+
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ if (VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
+ SET_VARSIZE_COMPRESSED(result, slicelength + VARHDRSZ);
+ else
+ SET_VARSIZE(result, slicelength + VARHDRSZ);
+
+ if (slicelength == 0)
+ return result;
+
+ toastrel = table_open(toast_pointer.va_toastrelid, AccessShareLock);
+ slot = table_slot_create(toastrel, NULL);
+
+ toast_fetch_datum_direct_slice_recursive(toastrel, &toast_pointer.va_tid,
+ result, &logical_offset,
+ sliceoffset, slicelength, slot);
+
+ ExecDropSingleTupleTableSlot(slot);
+ table_close(toastrel, AccessShareLock);
+
+ return result;
+}
+
+static void
+toast_fetch_datum_direct_slice_recursive(Relation toastrel, ItemPointer tid,
+ struct varlena *result, int32 *logical_offset,
+ int32 sliceoffset, int32 slicelength,
+ TupleTableSlot *slot)
+{
+ Snapshot snapshot = get_toast_snapshot();
+ Datum data_datum;
+ Datum tids_datum;
+ bool is_null_data;
+ bool is_null_tids;
+
+ if (!table_tuple_fetch_row_version(toastrel, tid, snapshot, slot))
+ {
+ elog(ERROR, "failed to fetch toast tuple by TID");
+ }
+
+ data_datum = slot_getattr(slot, 3, &is_null_data);
+ tids_datum = slot_getattr(slot, 4, &is_null_tids);
+
+ if (!is_null_tids)
+ {
+ ArrayType *arr = DatumGetArrayTypePCopy(tids_datum);
+ Datum *elems;
+ bool *nulls;
+ int nelems;
+ bool self_in_array = false;
+ int self_index = -1;
+ int32 chunk_size = 0;
+ char *chunk_data = NULL;
+ struct varlena *data_val = NULL;
+ int i;
+ int32 chunk_start;
+ int32 chunk_end;
+ int32 req_start;
+ int32 req_end;
+ int32 copy_start;
+ int32 copy_end;
+ int32 copy_len;
+ int32 src_offset;
+ int32 dest_offset;
+
+ deconstruct_array_builtin(arr, TIDOID, &elems, &nulls, &nelems);
+
+ if (!is_null_data)
+ {
+ data_val = PG_DETOAST_DATUM_COPY(data_datum);
+ chunk_size = VARSIZE_ANY_EXHDR(data_val);
+ chunk_data = VARDATA_ANY(data_val);
+ }
+
+ ExecClearTuple(slot);
+
+ for (i = 0; i < nelems; i++)
+ {
+ ItemPointer elem_tid = (ItemPointer) DatumGetPointer(elems[i]);
+ if (ItemPointerEquals(elem_tid, tid))
+ {
+ self_in_array = true;
+ self_index = i;
+ break;
+ }
+ }
+
+ for (i = 0; i < nelems; i++)
+ {
+ if (self_in_array && i == self_index)
+ {
+ /* Copy own data */
+ if (chunk_size > 0)
+ {
+ chunk_start = *logical_offset;
+ chunk_end = chunk_start + chunk_size;
+ *logical_offset = chunk_end;
+
+ req_start = sliceoffset;
+ req_end = sliceoffset + slicelength;
+
+ if (chunk_end > req_start && chunk_start < req_end)
+ {
+ copy_start = Max(chunk_start, req_start);
+ copy_end = Min(chunk_end, req_end);
+ copy_len = copy_end - copy_start;
+
+ if (copy_len > 0)
+ {
+ src_offset = copy_start - chunk_start;
+ dest_offset = copy_start - req_start;
+ memcpy(VARDATA(result) + dest_offset, chunk_data + src_offset, copy_len);
+ }
+ }
+ }
+ }
+ else
+ {
+ ItemPointer elem_tid = (ItemPointer) DatumGetPointer(elems[i]);
+ toast_fetch_datum_direct_slice_recursive(toastrel, elem_tid,
+ result, logical_offset,
+ sliceoffset, slicelength,
+ slot);
+ }
+ }
+
+ if (!self_in_array)
+ {
+ /* Copy own data last */
+ if (chunk_size > 0)
+ {
+ chunk_start = *logical_offset;
+ chunk_end = chunk_start + chunk_size;
+ *logical_offset = chunk_end;
+
+ req_start = sliceoffset;
+ req_end = sliceoffset + slicelength;
+
+ if (chunk_end > req_start && chunk_start < req_end)
+ {
+ copy_start = Max(chunk_start, req_start);
+ copy_end = Min(chunk_end, req_end);
+ copy_len = copy_end - copy_start;
+
+ if (copy_len > 0)
+ {
+ src_offset = copy_start - chunk_start;
+ dest_offset = copy_start - req_start;
+ memcpy(VARDATA(result) + dest_offset, chunk_data + src_offset, copy_len);
+ }
+ }
+ }
+ }
+
+ pfree(elems);
+ pfree(nulls);
+ pfree(arr);
+ if (data_val)
+ pfree(data_val);
+ }
+ else
+ {
+ /* No tid array. Just copy data chunk */
+ if (!is_null_data)
+ {
+ struct varlena *data_val = PG_DETOAST_DATUM(data_datum);
+ int32 chunk_size = VARSIZE_ANY_EXHDR(data_val);
+ int32 chunk_start;
+ int32 chunk_end;
+ int32 req_start;
+ int32 req_end;
+ int32 copy_start;
+ int32 copy_end;
+ int32 copy_len;
+ int32 src_offset;
+ int32 dest_offset;
+
+ chunk_start = *logical_offset;
+ chunk_end = chunk_start + chunk_size;
+ *logical_offset = chunk_end;
+
+ req_start = sliceoffset;
+ req_end = sliceoffset + slicelength;
+
+ if (chunk_end > req_start && chunk_start < req_end)
+ {
+ copy_start = Max(chunk_start, req_start);
+ copy_end = Min(chunk_end, req_end);
+ copy_len = copy_end - copy_start;
+
+ if (copy_len > 0)
+ {
+ src_offset = copy_start - chunk_start;
+ dest_offset = copy_start - req_start;
+ memcpy(VARDATA(result) + dest_offset, VARDATA_ANY(data_val) + src_offset, copy_len);
+ }
+ }
+ }
+ ExecClearTuple(slot);
+ }
+}
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 969d1028cae..eab6f213744 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -343,6 +343,7 @@ fill_val(CompactAttribute *att,
*infomask |= HEAP_HASEXTERNAL;
/* no alignment, since it's short by definition */
data_length = VARSIZE_EXTERNAL(val);
+
memcpy(data, val, data_length);
}
}
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index 50747c16396..abca97b2368 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -498,6 +498,13 @@ static relopt_enum_elt_def StdRdOptIndexCleanupValues[] =
{(const char *) NULL} /* list terminator */
};
+static relopt_enum_elt_def toastFlavourOptValues[] =
+{
+ {"plain", TOAST_FLAVOUR_PLAIN},
+ {"direct", TOAST_FLAVOUR_DIRECT},
+ {(const char *) NULL} /* list terminator */
+};
+
/* values from GistOptBufferingMode */
static relopt_enum_elt_def gistBufferingOptValues[] =
{
@@ -529,6 +536,17 @@ static relopt_enum enumRelOpts[] =
STDRD_OPTION_VACUUM_INDEX_CLEANUP_AUTO,
gettext_noop("Valid values are \"on\", \"off\", and \"auto\".")
},
+ {
+ {
+ "toast_flavour",
+ "Sets the TOAST flavour for this table",
+ RELOPT_KIND_HEAP,
+ ShareUpdateExclusiveLock
+ },
+ toastFlavourOptValues,
+ TOAST_FLAVOUR_NOT_SET,
+ gettext_noop("Valid values are \"plain\" and \"direct\".")
+ },
{
{
"buffering",
@@ -1912,6 +1930,8 @@ default_reloptions(Datum reloptions, bool validate, relopt_kind kind)
offsetof(StdRdOptions, parallel_workers)},
{"vacuum_index_cleanup", RELOPT_TYPE_ENUM,
offsetof(StdRdOptions, vacuum_index_cleanup)},
+ {"toast_flavour", RELOPT_TYPE_ENUM,
+ offsetof(StdRdOptions, toast_flavour)},
{"vacuum_truncate", RELOPT_TYPE_BOOL,
offsetof(StdRdOptions, vacuum_truncate), offsetof(StdRdOptions, vacuum_truncate_set)},
{"vacuum_max_eager_freeze_failure_rate", RELOPT_TYPE_REAL,
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index b03196fdf3e..9ae85cf5953 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -22,12 +22,19 @@
#include "access/xact.h"
#include "catalog/catalog.h"
#include "miscadmin.h"
+#include "utils/array.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
+int toast_flavour = TOAST_FLAVOUR_PLAIN;
+
static bool toastrel_valueid_exists(Relation toastrel, Oid valueid);
static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
+static Datum toast_save_datum_direct(Relation rel, Datum value,
+ struct varlena *oldexternal, int options);
+static void toast_delete_datum_direct(Relation rel, Datum value, bool is_speculative);
+static void toast_delete_datum_direct_recursive(Relation toastrel, ItemPointer tid, bool is_speculative);
/* ----------
* toast_compress_datum -
@@ -123,8 +130,8 @@ toast_save_datum(Relation rel, Datum value,
Relation *toastidxs;
HeapTuple toasttup;
TupleDesc toasttupDesc;
- Datum t_values[3];
- bool t_isnull[3];
+ Datum t_values[4];
+ bool t_isnull[4];
CommandId mycid = GetCurrentCommandId(true);
struct varlena *result;
struct varatt_external toast_pointer;
@@ -144,6 +151,9 @@ toast_save_datum(Relation rel, Datum value,
int num_indexes;
int validIndex;
+ if (RelationGetToastFlavour(rel) == TOAST_FLAVOUR_DIRECT)
+ return toast_save_datum_direct(rel, value, oldexternal, options);
+
Assert(!VARATT_IS_EXTERNAL(value));
/*
@@ -294,9 +304,11 @@ toast_save_datum(Relation rel, Datum value,
*/
t_values[0] = ObjectIdGetDatum(toast_pointer.va_valueid);
t_values[2] = PointerGetDatum(&chunk_data);
+ t_values[3] = PointerGetDatum(NULL);
t_isnull[0] = false;
t_isnull[1] = false;
t_isnull[2] = false;
+ t_isnull[3] = true;
/*
* Split up the item into chunks
@@ -394,6 +406,12 @@ toast_delete_datum(Relation rel, Datum value, bool is_speculative)
int num_indexes;
int validIndex;
+ if (VARATT_IS_EXTERNAL_DIRECT(attr))
+ {
+ toast_delete_datum_direct(rel, value, is_speculative);
+ return;
+ }
+
if (!VARATT_IS_EXTERNAL_ONDISK(attr))
return;
@@ -654,3 +672,237 @@ get_toast_snapshot(void)
return &SnapshotToastData;
}
+
+/*
+ * Direct TOAST write path.
+ * Stores the TID of the last chunk directly in the varlena header.
+ * If multi-chunk, the last chunk contains an array of TIDs of previous chunks.
+ */
+static Datum
+toast_save_datum_direct(Relation rel, Datum value,
+ struct varlena *oldexternal, int options)
+{
+ Relation toastrel;
+ HeapTuple toasttup;
+ TupleDesc toasttupDesc;
+ Datum t_values[4];
+ bool t_isnull[4];
+ CommandId mycid = GetCurrentCommandId(true);
+ struct varlena *result;
+ struct varatt_direct toast_pointer;
+ union
+ {
+ struct varlena hdr;
+ char data[TOAST_MAX_CHUNK_SIZE + VARHDRSZ];
+ int32 align_it;
+ } chunk_data = {0};
+ int32 chunk_size;
+ int32 chunk_seq = 0;
+ char *data_p;
+ int32 data_todo;
+ Pointer dval = DatumGetPointer(value);
+ Oid chunk_id;
+ int total_chunks;
+ ItemPointerData *tids;
+ int i;
+
+ Assert(!VARATT_IS_EXTERNAL(value));
+
+ toastrel = table_open(rel->rd_rel->reltoastrelid, RowExclusiveLock);
+ toasttupDesc = toastrel->rd_att;
+
+
+
+ if (VARATT_IS_SHORT(dval))
+ {
+ data_p = VARDATA_SHORT(dval);
+ data_todo = VARSIZE_SHORT(dval) - VARHDRSZ_SHORT;
+ toast_pointer.va_rawsize = data_todo + VARHDRSZ;
+ toast_pointer.va_extinfo = data_todo;
+ }
+ else if (VARATT_IS_COMPRESSED(dval))
+ {
+ data_p = VARDATA(dval);
+ data_todo = VARSIZE(dval) - VARHDRSZ;
+ toast_pointer.va_rawsize = VARDATA_COMPRESSED_GET_EXTSIZE(dval) + VARHDRSZ;
+ VARATT_EXTERNAL_SET_SIZE_AND_COMPRESS_METHOD(toast_pointer, data_todo,
+ VARDATA_COMPRESSED_GET_COMPRESS_METHOD(dval));
+ }
+ else
+ {
+ data_p = VARDATA(dval);
+ data_todo = VARSIZE(dval) - VARHDRSZ;
+ toast_pointer.va_rawsize = VARSIZE(dval);
+ toast_pointer.va_extinfo = data_todo;
+ }
+
+
+
+ if (OidIsValid(rel->rd_toastoid))
+ toast_pointer.va_toastrelid = rel->rd_toastoid;
+ else
+ toast_pointer.va_toastrelid = RelationGetRelid(toastrel);
+
+ chunk_id = InvalidOid;
+
+ total_chunks = ((data_todo - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
+ tids = palloc(sizeof(ItemPointerData) * total_chunks);
+
+ /* Insert chunks 0 to N-2 */
+ for (i = 0; i < total_chunks - 1; i++)
+ {
+ CHECK_FOR_INTERRUPTS();
+ chunk_size = Min(TOAST_MAX_CHUNK_SIZE, data_todo);
+
+ t_values[0] = ObjectIdGetDatum(chunk_id);
+ t_values[1] = Int32GetDatum(chunk_seq++);
+ SET_VARSIZE(&chunk_data, chunk_size + VARHDRSZ);
+ memcpy(VARDATA(&chunk_data), data_p, chunk_size);
+ t_values[2] = PointerGetDatum(&chunk_data);
+ t_values[3] = PointerGetDatum(NULL);
+
+ t_isnull[0] = false;
+ t_isnull[1] = false;
+ t_isnull[2] = false;
+ t_isnull[3] = true;
+
+ toasttup = heap_form_tuple(toasttupDesc, t_values, t_isnull);
+ heap_insert(toastrel, toasttup, mycid, options, NULL);
+
+ tids[i] = toasttup->t_self;
+
+
+
+ heap_freetuple(toasttup);
+ data_todo -= chunk_size;
+ data_p += chunk_size;
+ }
+
+ /* Insert final chunk containing previous TIDs */
+ CHECK_FOR_INTERRUPTS();
+ chunk_size = data_todo;
+ Assert(chunk_size <= TOAST_MAX_CHUNK_SIZE);
+
+ t_values[0] = ObjectIdGetDatum(chunk_id);
+ t_values[1] = Int32GetDatum(chunk_seq);
+ SET_VARSIZE(&chunk_data, chunk_size + VARHDRSZ);
+ memcpy(VARDATA(&chunk_data), data_p, chunk_size);
+ t_values[2] = PointerGetDatum(&chunk_data);
+
+ if (total_chunks > 1)
+ {
+ Datum *tids_datums;
+ ArrayType *tid_array;
+
+ tids_datums = palloc(sizeof(Datum) * (total_chunks - 1));
+ for (i = 0; i < total_chunks - 1; i++)
+ {
+ tids_datums[i] = PointerGetDatum(&tids[i]);
+ }
+ tid_array = construct_array(tids_datums, total_chunks - 1, TIDOID, 6, false, 's');
+ t_values[3] = PointerGetDatum(tid_array);
+ t_isnull[3] = false;
+ }
+ else
+ {
+ t_values[3] = PointerGetDatum(NULL);
+ t_isnull[3] = true;
+ }
+
+ t_isnull[0] = false;
+ t_isnull[1] = false;
+ t_isnull[2] = false;
+
+ toasttup = heap_form_tuple(toasttupDesc, t_values, t_isnull);
+ heap_insert(toastrel, toasttup, mycid, options, NULL);
+
+ toast_pointer.va_tid = toasttup->t_self;
+
+ heap_freetuple(toasttup);
+ table_close(toastrel, NoLock);
+
+ result = (struct varlena *) palloc(DIRECT_POINTER_SIZE);
+ SET_VARTAG_EXTERNAL(result, VARTAG_DIRECT);
+ memcpy(VARDATA_EXTERNAL(result), &toast_pointer, sizeof(toast_pointer));
+
+
+
+ return PointerGetDatum(result);
+}
+
+/*
+ * Direct TOAST delete path.
+ */
+static void
+toast_delete_datum_direct(Relation rel, Datum value, bool is_speculative)
+{
+ struct varlena *attr = (struct varlena *) DatumGetPointer(value);
+ struct varatt_direct toast_pointer;
+ Relation toastrel;
+
+ VARATT_EXTERNAL_GET_POINTER_DIRECT(toast_pointer, attr);
+
+ toastrel = table_open(toast_pointer.va_toastrelid, RowExclusiveLock);
+
+ toast_delete_datum_direct_recursive(toastrel, &toast_pointer.va_tid, is_speculative);
+
+ table_close(toastrel, RowExclusiveLock);
+}
+
+/*
+ * Recursively delete direct TOAST tuples.
+ */
+static void
+toast_delete_datum_direct_recursive(Relation toastrel, ItemPointer tid, bool is_speculative)
+{
+ TupleTableSlot *slot;
+ Snapshot snapshot = get_toast_snapshot();
+ Datum tids_datum;
+ bool is_null_tids;
+
+ slot = table_slot_create(toastrel, NULL);
+
+ if (!table_tuple_fetch_row_version(toastrel, tid, snapshot, slot))
+ {
+ ExecDropSingleTupleTableSlot(slot);
+ return;
+ }
+
+ tids_datum = slot_getattr(slot, 4, &is_null_tids);
+
+ if (!is_null_tids)
+ {
+ ArrayType *arr = DatumGetArrayTypePCopy(tids_datum);
+ Datum *elems;
+ bool *nulls;
+ int nelems;
+ int i;
+
+ deconstruct_array_builtin(arr, TIDOID, &elems, &nulls, &nelems);
+
+ ExecClearTuple(slot);
+
+ for (i = 0; i < nelems; i++)
+ {
+ ItemPointer elem_tid = (ItemPointer) DatumGetPointer(elems[i]);
+ if (!ItemPointerEquals(elem_tid, tid))
+ {
+ toast_delete_datum_direct_recursive(toastrel, elem_tid, is_speculative);
+ }
+ }
+ pfree(elems);
+ pfree(nulls);
+ pfree(arr);
+ }
+ else
+ {
+ ExecClearTuple(slot);
+ }
+
+ ExecDropSingleTupleTableSlot(slot);
+
+ if (is_speculative)
+ heap_abort_speculative(toastrel, tid);
+ else
+ simple_heap_delete(toastrel, tid);
+}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index b60fab0a4d2..119a67ca635 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -71,10 +71,11 @@ toast_tuple_init(ToastTupleContext *ttc)
* we have to delete it later.
*/
if (att->attlen == -1 && !ttc->ttc_oldisnull[i] &&
- VARATT_IS_EXTERNAL_ONDISK(old_value))
+ (VARATT_IS_EXTERNAL_ONDISK(old_value) || VARATT_IS_EXTERNAL_DIRECT(old_value)))
{
if (ttc->ttc_isnull[i] ||
- !VARATT_IS_EXTERNAL_ONDISK(new_value) ||
+ !(VARATT_IS_EXTERNAL_ONDISK(new_value) || VARATT_IS_EXTERNAL_DIRECT(new_value)) ||
+ VARTAG_EXTERNAL(old_value) != VARTAG_EXTERNAL(new_value) ||
memcmp(old_value, new_value,
VARSIZE_EXTERNAL(old_value)) != 0)
{
@@ -330,7 +331,7 @@ toast_delete_external(Relation rel, const Datum *values, const bool *isnull,
if (isnull[i])
continue;
- else if (VARATT_IS_EXTERNAL_ONDISK(value))
+ else if (VARATT_IS_EXTERNAL_ONDISK(value) || VARATT_IS_EXTERNAL_DIRECT(value))
toast_delete_datum(rel, value, is_speculative);
}
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 874a8fc89ad..5e06d8779be 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -32,6 +32,7 @@
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
+#include "utils/lsyscache.h"
static void CheckAndCreateToastTable(Oid relOid, Datum reloptions,
LOCKMODE lockmode, bool check,
@@ -201,7 +202,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
"pg_toast_%u_index", relOid);
/* this is pretty painful... need a tuple descriptor */
- tupdesc = CreateTemplateTupleDesc(3);
+ tupdesc = CreateTemplateTupleDesc(4);
TupleDescInitEntry(tupdesc, (AttrNumber) 1,
"chunk_id",
OIDOID,
@@ -214,6 +215,10 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
"chunk_data",
BYTEAOID,
-1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4,
+ "chunk_tids",
+ TIDARRAYOID,
+ -1, 0);
/*
* Ensure that the toast table doesn't itself get toasted, or we'll be
@@ -223,11 +228,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 0)->attstorage = TYPSTORAGE_PLAIN;
TupleDescAttr(tupdesc, 1)->attstorage = TYPSTORAGE_PLAIN;
TupleDescAttr(tupdesc, 2)->attstorage = TYPSTORAGE_PLAIN;
+ TupleDescAttr(tupdesc, 3)->attstorage = TYPSTORAGE_PLAIN;
/* Toast field should not be compressed */
TupleDescAttr(tupdesc, 0)->attcompression = InvalidCompressionMethod;
TupleDescAttr(tupdesc, 1)->attcompression = InvalidCompressionMethod;
TupleDescAttr(tupdesc, 2)->attcompression = InvalidCompressionMethod;
+ TupleDescAttr(tupdesc, 3)->attcompression = InvalidCompressionMethod;
/*
* Toast tables for regular relations go in pg_toast; those for temp
@@ -292,7 +299,15 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
indexInfo->ii_IndexAttrNumbers[1] = 2;
indexInfo->ii_Expressions = NIL;
indexInfo->ii_ExpressionsState = NIL;
- indexInfo->ii_Predicate = NIL;
+ {
+ Expr *pred_expr = make_opclause(608, BOOLOID, false,
+ (Expr *) makeVar(1, 1, OIDOID, -1, InvalidOid, 0),
+ (Expr *) makeConst(OIDOID, -1, InvalidOid, sizeof(Oid), ObjectIdGetDatum(InvalidOid), false, true),
+ InvalidOid, InvalidOid);
+ OpExpr *op_expr = (OpExpr *) pred_expr;
+ op_expr->opfuncid = get_opcode(608);
+ indexInfo->ii_Predicate = list_make1(pred_expr);
+ }
indexInfo->ii_PredicateState = NULL;
indexInfo->ii_ExclusionOps = NULL;
indexInfo->ii_ExclusionProcs = NULL;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 6b82a23435e..5cd0b914201 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -33,6 +33,7 @@
#include "access/gin.h"
#include "access/slru.h"
#include "access/toast_compression.h"
+#include "access/toast_internals.h"
#include "access/twophase.h"
#include "access/xlog_internal.h"
#include "access/xlogprefetcher.h"
@@ -284,6 +285,12 @@ static const struct config_enum_entry xmloption_options[] = {
StaticAssertDecl(lengthof(xmloption_options) == (XMLOPTION_CONTENT + 2),
"array length mismatch");
+static const struct config_enum_entry toast_flavour_options[] = {
+ {"plain", TOAST_FLAVOUR_PLAIN, false},
+ {"direct", TOAST_FLAVOUR_DIRECT, false},
+ {NULL, 0, false}
+};
+
/*
* Although only "on", "off", and "safe_encoding" are documented, we
* accept all the likely variants of "on" and "off".
@@ -5067,6 +5074,17 @@ struct config_enum ConfigureNamesEnum[] =
NULL, NULL, NULL
},
+ {
+ {"toast_flavour", PGC_USERSET, CLIENT_CONN_STATEMENT,
+ gettext_noop("Sets the TOAST flavour to use for new writes."),
+ NULL
+ },
+ &toast_flavour,
+ TOAST_FLAVOUR_PLAIN,
+ toast_flavour_options,
+ NULL, NULL, NULL
+ },
+
{
{"default_transaction_isolation", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the transaction isolation level of each new transaction."),
diff --git a/src/bin/pg_basebackup/Makefile b/src/bin/pg_basebackup/Makefile
index a9557c0789f..8bdbe7550cc 100644
--- a/src/bin/pg_basebackup/Makefile
+++ b/src/bin/pg_basebackup/Makefile
@@ -39,7 +39,7 @@ BBOBJS = \
pg_basebackup.o \
astreamer_inject.o
-all: pg_basebackup pg_createsubscriber pg_receivewal pg_recvlogical
+all: pg_basebackup pg_createsubscriber pg_recvlogical
pg_basebackup: $(BBOBJS) $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
$(CC) $(CFLAGS) $(BBOBJS) $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
@@ -56,7 +56,6 @@ pg_recvlogical: pg_recvlogical.o $(OBJS) | submake-libpq submake-libpgport subma
install: all installdirs
$(INSTALL_PROGRAM) pg_basebackup$(X) '$(DESTDIR)$(bindir)/pg_basebackup$(X)'
$(INSTALL_PROGRAM) pg_createsubscriber$(X) '$(DESTDIR)$(bindir)/pg_createsubscriber$(X)'
- $(INSTALL_PROGRAM) pg_receivewal$(X) '$(DESTDIR)$(bindir)/pg_receivewal$(X)'
$(INSTALL_PROGRAM) pg_recvlogical$(X) '$(DESTDIR)$(bindir)/pg_recvlogical$(X)'
installdirs:
@@ -65,12 +64,11 @@ installdirs:
uninstall:
rm -f '$(DESTDIR)$(bindir)/pg_basebackup$(X)'
rm -f '$(DESTDIR)$(bindir)/pg_createsubscriber$(X)'
- rm -f '$(DESTDIR)$(bindir)/pg_receivewal$(X)'
rm -f '$(DESTDIR)$(bindir)/pg_recvlogical$(X)'
clean distclean:
- rm -f pg_basebackup$(X) pg_createsubscriber$(X) pg_receivewal$(X) pg_recvlogical$(X) \
- $(BBOBJS) pg_createsubscriber.o pg_receivewal.o pg_recvlogical.o \
+ rm -f pg_basebackup$(X) pg_createsubscriber$(X) pg_recvlogical$(X) \
+ $(BBOBJS) pg_createsubscriber.o pg_recvlogical.o \
$(OBJS)
rm -rf tmp_check
diff --git a/src/bin/pg_basebackup/pg_receivewal.c b/src/bin/pg_basebackup/pg_receivewal.c
index e816cf58101..4035aef1e3f 100644
--- a/src/bin/pg_basebackup/pg_receivewal.c
+++ b/src/bin/pg_basebackup/pg_receivewal.c
@@ -566,10 +566,8 @@ StreamLog(void)
Assert(stream.startpos != InvalidXLogRecPtr &&
stream.timeline != 0);
- /*
- * Always start streaming at the beginning of a segment
- */
- stream.startpos -= XLogSegmentOffset(stream.startpos, WalSegSz);
+ if (gcs_bucket == NULL)
+ stream.startpos -= XLogSegmentOffset(stream.startpos, WalSegSz);
/*
* Start the replication
diff --git a/src/bin/pg_basebackup/receivelog.c b/src/bin/pg_basebackup/receivelog.c
index c9c7bbadc9a..31b772241b6 100644
--- a/src/bin/pg_basebackup/receivelog.c
+++ b/src/bin/pg_basebackup/receivelog.c
@@ -150,6 +150,32 @@ open_walfile(StreamCtl *stream, XLogRecPtr startpoint)
}
if (size != 0)
{
+ if (stream->walmethod->allow_intermediate_size)
+ {
+ /* Intermediate size allowed. Open for write with pad_to_size = 0 to resume */
+ f = stream->walmethod->ops->open_for_write(stream->walmethod, walfile_name, stream->partial_suffix, 0);
+ if (f == NULL)
+ {
+ pg_log_error("could not open existing write-ahead log file \"%s\": %s",
+ fn, GetLastWalMethodError(stream->walmethod));
+ pg_free(fn);
+ return false;
+ }
+
+ /* fsync file in case of a previous crash */
+ if (stream->walmethod->ops->sync(f) != 0)
+ {
+ pg_log_error("could not fsync existing write-ahead log file \"%s\": %s",
+ fn, GetLastWalMethodError(stream->walmethod));
+ stream->walmethod->ops->close(f, CLOSE_UNLINK);
+ exit(1);
+ }
+
+ walfile = f;
+ pg_free(fn);
+ return true;
+ }
+
/* if write didn't set errno, assume problem is no disk space */
if (errno == 0)
errno = ENOSPC;
@@ -1091,7 +1117,7 @@ ProcessXLogDataMsg(PGconn *conn, StreamCtl *stream, char *copybuf, int len,
if (walfile == NULL)
{
/* No file open yet */
- if (xlogoff != 0)
+ if (xlogoff != 0 && !stream->walmethod->allow_intermediate_size)
{
pg_log_error("received write-ahead log record for offset %u with no file open",
xlogoff);
diff --git a/src/bin/pg_basebackup/walmethods.c b/src/bin/pg_basebackup/walmethods.c
index eaaabc5f374..af328291871 100644
--- a/src/bin/pg_basebackup/walmethods.c
+++ b/src/bin/pg_basebackup/walmethods.c
@@ -649,6 +649,7 @@ CreateWalDirectoryMethod(const char *basedir,
wwmethod->base.compression_algorithm = compression_algorithm;
wwmethod->base.compression_level = compression_level;
wwmethod->base.sync = sync;
+ wwmethod->base.allow_intermediate_size = false;
clear_error(&wwmethod->base);
wwmethod->basedir = pg_strdup(basedir);
@@ -1366,6 +1367,7 @@ CreateWalTarMethod(const char *tarbase,
wwmethod->base.compression_algorithm = compression_algorithm;
wwmethod->base.compression_level = compression_level;
wwmethod->base.sync = sync;
+ wwmethod->base.allow_intermediate_size = false;
clear_error(&wwmethod->base);
wwmethod->tarfilename = pg_malloc0(strlen(tarbase) + strlen(suffix) + 1);
diff --git a/src/bin/pg_basebackup/walmethods.h b/src/bin/pg_basebackup/walmethods.h
index f7a6dc18439..495a0e95f9c 100644
--- a/src/bin/pg_basebackup/walmethods.h
+++ b/src/bin/pg_basebackup/walmethods.h
@@ -106,6 +106,7 @@ struct WalWriteMethod
pg_compress_algorithm compression_algorithm;
int compression_level;
bool sync;
+ bool allow_intermediate_size;
const char *lasterrstring; /* if set, takes precedence over lasterrno */
int lasterrno;
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index e603a2276c3..34af70bd95f 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -27,8 +27,18 @@ do { \
memcpy(&(toast_pointer), VARDATA_EXTERNAL(attre), sizeof(toast_pointer)); \
} while (0)
+#define VARATT_EXTERNAL_GET_POINTER_DIRECT(toast_pointer, attr) \
+do { \
+ varattrib_1b_e *attre = (varattrib_1b_e *) (attr); \
+ Assert(VARATT_IS_EXTERNAL(attre)); \
+ Assert(VARTAG_EXTERNAL(attre) == VARTAG_DIRECT); \
+ Assert(VARSIZE_EXTERNAL(attre) == sizeof(toast_pointer) + VARHDRSZ_EXTERNAL); \
+ memcpy(&(toast_pointer), VARDATA_EXTERNAL(attre), sizeof(toast_pointer)); \
+} while (0)
+
/* Size of an EXTERNAL datum that contains a standard TOAST pointer */
#define TOAST_POINTER_SIZE (VARHDRSZ_EXTERNAL + sizeof(varatt_external))
+#define DIRECT_POINTER_SIZE (VARHDRSZ_EXTERNAL + sizeof(varatt_direct))
/* Size of an EXTERNAL datum that contains an indirection pointer */
#define INDIRECT_POINTER_SIZE (VARHDRSZ_EXTERNAL + sizeof(varatt_indirect))
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 06ae8583c1e..902a900d356 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -16,6 +16,15 @@
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
+#include "utils/rel.h"
+
+#define RelationGetToastFlavour(relation) \
+ ((relation)->rd_options ? \
+ (((StdRdOptions *) (relation)->rd_options)->toast_flavour != TOAST_FLAVOUR_NOT_SET ? \
+ ((StdRdOptions *) (relation)->rd_options)->toast_flavour : (ToastFlavour) toast_flavour) \
+ : (ToastFlavour) toast_flavour)
+
+extern PGDLLIMPORT int toast_flavour;
/*
* The information at the start of the compressed toast data.
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 09c794239ca..0696956e027 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -337,6 +337,13 @@ typedef enum StdRdOptIndexCleanup
STDRD_OPTION_VACUUM_INDEX_CLEANUP_ON,
} StdRdOptIndexCleanup;
+typedef enum ToastFlavour
+{
+ TOAST_FLAVOUR_NOT_SET = -1,
+ TOAST_FLAVOUR_PLAIN = 0,
+ TOAST_FLAVOUR_DIRECT = 1
+} ToastFlavour;
+
typedef struct StdRdOptions
{
int32 vl_len_; /* varlena header (do not touch directly!) */
@@ -354,6 +361,7 @@ typedef struct StdRdOptions
* to freeze. 0 if disabled, -1 if unspecified.
*/
double vacuum_max_eager_freeze_failure_rate;
+ ToastFlavour toast_flavour;
} StdRdOptions;
#define HEAP_MIN_FILLFACTOR 10
diff --git a/src/include/varatt.h b/src/include/varatt.h
index 2e8564d4998..64c4f44ccad 100644
--- a/src/include/varatt.h
+++ b/src/include/varatt.h
@@ -15,6 +15,8 @@
#ifndef VARATT_H
#define VARATT_H
+#include "storage/itemptr.h"
+
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
@@ -38,6 +40,15 @@ typedef struct varatt_external
Oid va_toastrelid; /* RelID of TOAST table containing it */
} varatt_external;
+typedef struct varatt_direct
+{
+ int32 va_rawsize; /* Original data size (includes header) */
+ uint32 va_extinfo; /* External saved size (without header) and
+ * compression method */
+ ItemPointerData va_tid; /* TID of first toast tuple */
+ Oid va_toastrelid; /* RelID of TOAST table containing it */
+} varatt_direct;
+
/*
* These macros define the "saved size" portion of va_extinfo. Its remaining
* two high-order bits identify the compression method.
@@ -86,7 +97,8 @@ typedef enum vartag_external
VARTAG_INDIRECT = 1,
VARTAG_EXPANDED_RO = 2,
VARTAG_EXPANDED_RW = 3,
- VARTAG_ONDISK = 18
+ VARTAG_ONDISK = 18,
+ VARTAG_DIRECT = 19
} vartag_external;
/* this test relies on the specific tag values above */
@@ -97,6 +109,7 @@ typedef enum vartag_external
((tag) == VARTAG_INDIRECT ? sizeof(varatt_indirect) : \
VARTAG_IS_EXPANDED(tag) ? sizeof(varatt_expanded) : \
(tag) == VARTAG_ONDISK ? sizeof(varatt_external) : \
+ (tag) == VARTAG_DIRECT ? sizeof(varatt_direct) : \
(AssertMacro(false), 0))
/*
@@ -289,6 +302,8 @@ typedef struct
#define VARATT_IS_EXTERNAL(PTR) VARATT_IS_1B_E(PTR)
#define VARATT_IS_EXTERNAL_ONDISK(PTR) \
(VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_ONDISK)
+#define VARATT_IS_EXTERNAL_DIRECT(PTR) \
+ (VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_DIRECT)
#define VARATT_IS_EXTERNAL_INDIRECT(PTR) \
(VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_INDIRECT)
#define VARATT_IS_EXTERNAL_EXPANDED_RO(PTR) \
diff --git a/src/test/regress/expected/cluster.out b/src/test/regress/expected/cluster.out
index 4d40a6809ab..ad7fdc12524 100644
--- a/src/test/regress/expected/cluster.out
+++ b/src/test/regress/expected/cluster.out
@@ -308,6 +308,7 @@ WHERE pg_class.oid=indexrelid
-- Verify that toast tables are clusterable
CLUSTER pg_toast.pg_toast_826 USING pg_toast_826_index;
+ERROR: cannot cluster on partial index "pg_toast_826_index"
-- Verify that clustering all tables does in fact cluster the right ones
CREATE USER regress_clstr_user;
CREATE TABLE clstr_1 (a INT PRIMARY KEY);
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 04a16527f69..750e5961ed6 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -5148,9 +5148,10 @@ TOAST table "pg_toast.pg_toast_2619"
chunk_id | oid
chunk_seq | integer
chunk_data | bytea
+ chunk_tids | tid[]
Owning table: "pg_catalog.pg_statistic"
Indexes:
- "pg_toast_2619_index" PRIMARY KEY, btree (chunk_id, chunk_seq)
+ "pg_toast_2619_index" PRIMARY KEY, btree (chunk_id, chunk_seq) WHERE chunk_id <> '0'::oid
-- check printing info about access methods
\dA
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index e1e0c540194..b484075c3b0 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -102,7 +102,7 @@ test: publication subscription
# Another group of parallel tests
# select_views depends on create_view
# ----------
-test: select_views portals_p2 foreign_key cluster dependency guc bitmapops combocid tsearch tsdicts foreign_data window xmlmap functional_deps advisory_lock indirect_toast equivclass
+test: select_views portals_p2 foreign_key cluster dependency guc bitmapops combocid tsearch tsdicts foreign_data window xmlmap functional_deps advisory_lock indirect_toast direct_toast equivclass
# ----------
# Another group of parallel tests (JSON related)
