Hi Robert
> But I don't quite understand the point of this
> response: it seems like you're just restating what the design does
> without really justifying it. The question here isn't whether a 3-byte
> header can describe a length up to 16MB; I think we all know our
> powers of two well enough to agree on the answer to that question. The
> question is whether it's a good use of 3 bytes, and I don't think it
> is.
My initial decision to include a 3‑byte length field was driven by two goals:
1. Avoid introducing separate callbacks for each algorithm.
2. Provide a single, algorithm-agnostic mechanism for handling
metadata length.
After re-evaluating based on your feedback, I agree that the fixed
overhead of a 3-byte length field outweighs its benefit; per-algorithm
callbacks deliver the same functionality while saving three bytes per
datum.
> I did consider the fact that future compression algorithms might want
> to use variable-length headers; but I couldn't see a reason why we
> shouldn't let each of those compression algorithms decide for
> themselves how to encode whatever information they need. If a
> compression algorithm needs a variable-length header, then it just
> needs to make that header self-describing. Worst case scenario, it can
> make the first byte of that variable-length header a length byte, and
> then go from there; but it's probably possible to be even smarter and
> use less than a full byte. Say for example we store a dictionary ID
> that in concept is a 32-bit quantity but we use a variable-length
> integer representation for it. It's easy to see that we shouldn't ever
> need more than 3 bits for that so a full length byte is overkill and,
> in fact, would undermine the value of a variable-length representation
> rather severely. (I suspect it's a bad idea anyway, but it's a worse
> idea if you burn a full byte on a length header.)
>
I agree. Each compression algorithm can decide its own metadata size
overhead. Callbacks can provide this information as well rather than
storing in fixed length bytes(3 bytes). The revised patch introduces a
"toast_cmpid_meta_size(const varatt_cmp_extended *hdr)", which
calculates the metadata size.
> But there's an even larger question here too, which is why we're
> having some kind of discussion about generalized metadata when the
> current project seemingly only requires a 4-byte dictionary OID. If
> you have some other use of this space in mind, I don't think you've
> told us what it is. If you don't, then I'm not sure why we're
> designing around an up-to-16MB variable-length quantity when what we
> have before us is a 4-byte fixed-length quantity.
This project only requires 4 bytes of fixed-size metadata to store the
dictionary ID.
Updated design for extending varattrib_4b compression
1. extensible header
/*
* varatt_cmp_extended: an optional per‐datum header for extended
compression method.
* Only used when va_tcinfo's top two bits are "11".
*/
typedef struct varatt_cmp_extended
{
uint8 cmp_alg;
char cmp_meta[FLEXIBLE_ARRAY_MEMBER]; /*
algorithm‐specific metadata */
} varatt_cmp_extended;
2. Algorithm registry and metadata size dispatch
static inline uint32
unsupported_meta_size(const varatt_cmp_extended *hdr)
{
elog(ERROR, "toast_cmpid_meta_size called for unsupported
compression algorithm");
return 0; /* unreachable */
}
/* no metadata for plain-ZSTD */
static inline uint32
zstd_nodict_meta_size(const varatt_cmp_extended *hdr)
{
return 0;
}
static inline uint32
zstd_dict_meta_size(const varatt_cmp_extended *hdr)
{
return sizeof(Oid);
}
/*
* TOAST compression methods enumeration.
*
* NAME : algorithm identifier
* VALUE : enum value
* META-SIZE-FN : Calculates algorithm metadata size.
*/
#define TOAST_COMPRESSION_LIST \
X(PGLZ, 0, unsupported_meta_size) \
X(LZ4, 1, unsupported_meta_size) \
X(ZSTD_NODICT, 2, zstd_nodict_meta_size) \
X(ZSTD_DICT, 3, zstd_dict_meta_size) \
X(INVALID, 4, unsupported_meta_size) /* sentinel */
/* Compression algorithm identifiers */
typedef enum ToastCompressionId
{
#define X(name,val,fn) TOAST_##name##_COMPRESSION_ID = (val),
TOAST_COMPRESSION_LIST
#undef X
} ToastCompressionId;
/* lookup table to check if compression method uses extended format */
static const bool toast_cmpid_extended[] = {
#define X(name,val,fn) \
/* PGLZ, LZ4 don't use extended format */ \
[TOAST_##name##_COMPRESSION_ID] = \
((val) != TOAST_PGLZ_COMPRESSION_ID && \
(val) != TOAST_LZ4_COMPRESSION_ID && \
(val) != TOAST_INVALID_COMPRESSION_ID),
TOAST_COMPRESSION_LIST
#undef X
};
#define TOAST_CMPID_EXTENDED(alg) (toast_cmpid_extended[alg])
/*
* Prototype for a per-datum metadata-size callback:
* given a pointer to the extended header, return
* how many metadata bytes follow it.
*/
typedef uint32 (*ToastMetaSizeFn) (const varatt_cmp_extended *hdr);
/* Callback table—indexed by ToastCompressionId */
static const ToastMetaSizeFn toast_meta_size_fns[] = {
#define X(name,val,fn) [TOAST_##name##_COMPRESSION_ID] = fn,
TOAST_COMPRESSION_LIST
#undef X
};
/* Calculates algorithm metadata size */
static inline uint32
toast_cmpid_meta_size(const varatt_cmp_extended *hdr)
{
Assert(hdr != NULL);
return toast_meta_size_fns[hdr->cmp_alg] (hdr);
}
Each compression algorithm provides a static callback that returns the
size of its metadata, given a pointer to the varatt_cmp_extended
header. Algorithms with fixed-size metadata return a constant, while
algorithms with variable-length metadata are responsible for defining
and parsing their own internal headers to compute the metadata size.
3. Resulting on-disk layouts for zstd
ZSTD (nodict) — datum on‑disk layout
+----------------------------------+
| va_header (uint32) |
+----------------------------------+
| va_tcinfo (uint32) | ← top two bits = 11 (extended)
+----------------------------------+
| cmp_alg (uint8) | ← (ZSTD_NODICT)
+----------------------------------+
| compressed bytes … | ← ZSTD frame
+----------------------------------+
ZSTD(dict) — datum on‑disk layout
+----------------------------------+
| va_header (uint32) |
+----------------------------------+
| va_tcinfo (uint32) | ← top two bits = 11 (extended)
+----------------------------------+
| cmp_alg (uint8) | ← (ZSTD_DICT)
+----------------------------------+
| dict_id (uint32) | ← dictionary OID
+----------------------------------+
| compressed bytes … | ← ZSTD frame
+----------------------------------+
I hope this updated design addresses your concerns. I would appreciate
any further feedback you may have. Thanks again for your guidance—it's
been very helpful.
v20-0001-varattrib_4b-design-proposal-to-make-it-extended.patch:
varattrib_4b extensibility – adds varatt_cmp_extended, metadata size
dispatch and useful macros; behaviour unchanged.
v20-0002-zstd-nodict-compression.patch: Plain ZSTD (non dict) support.
--
Nikhil Veldanda
From dc6172c6945354634063b03215dc6798ae22cc2f Mon Sep 17 00:00:00 2001
From: Nikhil Kumar Veldanda <nikhilkv@amazon.com>
Date: Sat, 3 May 2025 02:14:02 +0000
Subject: [PATCH v20 2/2] zstd nodict compression
---
contrib/amcheck/verify_heapam.c | 1 +
src/backend/access/common/detoast.c | 12 +-
src/backend/access/common/reloptions.c | 14 +-
src/backend/access/common/toast_compression.c | 171 +++++++++++++++++-
src/backend/access/common/toast_internals.c | 4 +
src/backend/utils/adt/varlena.c | 3 +
src/backend/utils/misc/guc_tables.c | 3 +
src/backend/utils/misc/postgresql.conf.sample | 2 +-
src/bin/psql/describe.c | 5 +-
src/bin/psql/tab-complete.in.c | 4 +-
src/include/access/toast_compression.h | 36 +++-
src/include/access/toast_internals.h | 3 +-
src/include/utils/attoptcache.h | 1 +
src/test/regress/expected/compression.out | 5 +-
src/test/regress/expected/compression_1.out | 3 +
.../expected/compression_zstd_nodict.out | 152 ++++++++++++++++
.../expected/compression_zstd_nodict_1.out | 103 +++++++++++
src/test/regress/parallel_schedule | 2 +-
src/test/regress/sql/compression.sql | 1 +
.../regress/sql/compression_zstd_nodict.sql | 82 +++++++++
20 files changed, 581 insertions(+), 26 deletions(-)
create mode 100644 src/test/regress/expected/compression_zstd_nodict.out
create mode 100644 src/test/regress/expected/compression_zstd_nodict_1.out
create mode 100644 src/test/regress/sql/compression_zstd_nodict.sql
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index d7c2ac6951a..111bb308341 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -1792,6 +1792,7 @@ check_tuple_attribute(HeapCheckContext *ctx)
/* List of all valid compression method IDs */
case TOAST_PGLZ_COMPRESSION_ID:
case TOAST_LZ4_COMPRESSION_ID:
+ case TOAST_ZSTD_NODICT_COMPRESSION_ID:
valid = true;
break;
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index 01419d1c65f..451230023ec 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -246,10 +246,10 @@ detoast_attr_slice(struct varlena *attr,
* Determine maximum amount of compressed data needed for a prefix
* of a given length (after decompression).
*
- * At least for now, if it's LZ4 data, we'll have to fetch the
- * whole thing, because there doesn't seem to be an API call to
- * determine how much compressed data we need to be sure of being
- * able to decompress the required slice.
+ * At least for now, if it's LZ4 or Zstandard data, we'll have to
+ * fetch the whole thing, because there doesn't seem to be an API
+ * call to determine how much compressed data we need to be sure
+ * of being able to decompress the required slice.
*/
if (VARATT_EXTERNAL_GET_COMPRESS_METHOD(toast_pointer) ==
TOAST_PGLZ_COMPRESSION_ID)
@@ -485,6 +485,8 @@ toast_decompress_datum(struct varlena *attr)
return pglz_decompress_datum(attr);
case TOAST_LZ4_COMPRESSION_ID:
return lz4_decompress_datum(attr);
+ case TOAST_ZSTD_NODICT_COMPRESSION_ID:
+ return zstd_nodict_decompress_datum(attr);
default:
elog(ERROR, "invalid compression method id %d", cmid);
return NULL; /* keep compiler quiet */
@@ -528,6 +530,8 @@ toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
return pglz_decompress_datum_slice(attr, slicelength);
case TOAST_LZ4_COMPRESSION_ID:
return lz4_decompress_datum_slice(attr, slicelength);
+ case TOAST_ZSTD_NODICT_COMPRESSION_ID:
+ return zstd_nodict_decompress_datum_slice(attr, slicelength);
default:
elog(ERROR, "invalid compression method id %d", cmid);
return NULL; /* keep compiler quiet */
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index 46c1dce222d..1267668a242 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -24,6 +24,7 @@
#include "access/nbtree.h"
#include "access/reloptions.h"
#include "access/spgist_private.h"
+#include "access/toast_compression.h"
#include "catalog/pg_type.h"
#include "commands/defrem.h"
#include "commands/tablespace.h"
@@ -381,7 +382,15 @@ static relopt_int intRelOpts[] =
},
-1, 0, 1024
},
-
+ {
+ {
+ "zstd_level",
+ "Set column's ZSTD compression level",
+ RELOPT_KIND_ATTRIBUTE,
+ ShareUpdateExclusiveLock
+ },
+ DEFAULT_ZSTD_LEVEL, MIN_ZSTD_LEVEL, MAX_ZSTD_LEVEL
+ },
/* list terminator */
{{NULL}}
};
@@ -2097,7 +2106,8 @@ attribute_reloptions(Datum reloptions, bool validate)
{
static const relopt_parse_elt tab[] = {
{"n_distinct", RELOPT_TYPE_REAL, offsetof(AttributeOpts, n_distinct)},
- {"n_distinct_inherited", RELOPT_TYPE_REAL, offsetof(AttributeOpts, n_distinct_inherited)}
+ {"n_distinct_inherited", RELOPT_TYPE_REAL, offsetof(AttributeOpts, n_distinct_inherited)},
+ {"zstd_level", RELOPT_TYPE_INT, offsetof(AttributeOpts, zstd_level)},
};
return (bytea *) build_reloptions(reloptions, validate,
diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c
index 5e5d42d80ef..02823d1c435 100644
--- a/src/backend/access/common/toast_compression.c
+++ b/src/backend/access/common/toast_compression.c
@@ -17,6 +17,10 @@
#include <lz4.h>
#endif
+#ifdef USE_ZSTD
+#include <zstd.h>
+#endif
+
#include "access/detoast.h"
#include "access/toast_compression.h"
#include "common/pg_lzcompress.h"
@@ -26,11 +30,19 @@
/* GUC */
int default_toast_compression = TOAST_PGLZ_COMPRESSION;
-#define NO_LZ4_SUPPORT() \
+#ifdef USE_ZSTD
+#define ZSTD_CHECK_ERROR(zstd_ret, msg) \
+ do { \
+ if (ZSTD_isError(zstd_ret)) \
+ ereport(ERROR, (errmsg("%s: %s", (msg), ZSTD_getErrorName(zstd_ret)))); \
+ } while (0)
+#endif
+
+#define COMPRESSION_METHOD_NOT_SUPPORTED(method) \
ereport(ERROR, \
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED), \
- errmsg("compression method lz4 not supported"), \
- errdetail("This functionality requires the server to be built with lz4 support.")))
+ errmsg("compression method %s not supported", method), \
+ errdetail("This functionality requires the server to be built with %s support.", method)))
/*
* Compress a varlena using PGLZ.
@@ -140,7 +152,7 @@ struct varlena *
lz4_compress_datum(const struct varlena *value)
{
#ifndef USE_LZ4
- NO_LZ4_SUPPORT();
+ COMPRESSION_METHOD_NOT_SUPPORTED("lz4");
return NULL; /* keep compiler quiet */
#else
int32 valsize;
@@ -183,7 +195,7 @@ struct varlena *
lz4_decompress_datum(const struct varlena *value)
{
#ifndef USE_LZ4
- NO_LZ4_SUPPORT();
+ COMPRESSION_METHOD_NOT_SUPPORTED("lz4");
return NULL; /* keep compiler quiet */
#else
int32 rawsize;
@@ -216,7 +228,7 @@ struct varlena *
lz4_decompress_datum_slice(const struct varlena *value, int32 slicelength)
{
#ifndef USE_LZ4
- NO_LZ4_SUPPORT();
+ COMPRESSION_METHOD_NOT_SUPPORTED("lz4");
return NULL; /* keep compiler quiet */
#else
int32 rawsize;
@@ -246,6 +258,129 @@ lz4_decompress_datum_slice(const struct varlena *value, int32 slicelength)
#endif
}
+/* Compress datum using ZSTD with optional dictionary (using cdict) */
+struct varlena *
+zstd_nodict_compress_datum(const struct varlena *value, CompressionInfo cmp)
+{
+#ifdef USE_ZSTD
+ uint32 valsize = VARSIZE_ANY_EXHDR(value);
+ size_t max_size = ZSTD_compressBound(valsize);
+ struct varlena *compressed;
+ void *dest;
+ size_t cmp_size;
+
+ /* Allocate space for the compressed varlena (header + data) */
+ compressed = (struct varlena *) palloc(max_size + VARATT_4BCE_HDRSZ(cmp.cmp_ext));
+ dest = (char *) compressed + VARATT_4BCE_HDRSZ(cmp.cmp_ext);
+
+ cmp_size = ZSTD_compress(dest,
+ max_size,
+ VARDATA_ANY(value),
+ valsize,
+ cmp.zstd_level);
+
+ if (ZSTD_isError(cmp_size))
+ {
+ pfree(compressed);
+ ZSTD_CHECK_ERROR(cmp_size, "ZSTD compression failed");
+ }
+
+ /*
+ * If compression did not reduce size, return NULL so that the
+ * uncompressed data is stored
+ */
+ if (cmp_size > valsize)
+ {
+ pfree(compressed);
+ return NULL;
+ }
+
+ /* Set the compressed size in the varlena header */
+ SET_VARSIZE_COMPRESSED(compressed, cmp_size + VARATT_4BCE_HDRSZ(cmp.cmp_ext));
+ return compressed;
+
+#else
+ COMPRESSION_METHOD_NOT_SUPPORTED("zstd_nodict");
+ return NULL;
+#endif
+}
+
+/* Decompression routine */
+struct varlena *
+zstd_nodict_decompress_datum(const struct varlena *value)
+{
+#ifdef USE_ZSTD
+ uint32 actual_size_exhdr = VARDATA_COMPRESSED_GET_EXTSIZE(value);
+ uint32 cmp_size_exhdr = VARATT_4BCE_PAYLOAD_SIZE(value);
+ struct varlena *result;
+ size_t uncmp_size;
+
+ /* Allocate space for the uncompressed data */
+ result = (struct varlena *) palloc(actual_size_exhdr + VARHDRSZ);
+
+ uncmp_size = ZSTD_decompress(VARDATA(result),
+ actual_size_exhdr,
+ VARATT_4BCE_PAYLOAD_PTR(value),
+ cmp_size_exhdr);
+
+ if (ZSTD_isError(uncmp_size))
+ {
+ pfree(result);
+ ZSTD_CHECK_ERROR(uncmp_size, "ZSTD decompression failed");
+ }
+
+ /* Set final size in the varlena header */
+ SET_VARSIZE(result, uncmp_size + VARHDRSZ);
+ return result;
+
+#else
+ COMPRESSION_METHOD_NOT_SUPPORTED("zstd_nodict");
+ return NULL;
+#endif
+}
+
+/* Decompress a slice of the datum using the streaming API and optional dictionary */
+struct varlena *
+zstd_nodict_decompress_datum_slice(const struct varlena *value, int32 slicelength)
+{
+#ifdef USE_ZSTD
+ struct varlena *result;
+ ZSTD_inBuffer inBuf;
+ ZSTD_outBuffer outBuf;
+ size_t ret;
+ ZSTD_DCtx *ZstdDecompressionCtx = ZSTD_createDCtx();
+
+ inBuf.src = VARATT_4BCE_PAYLOAD_PTR(value);
+ inBuf.size = VARATT_4BCE_PAYLOAD_SIZE(value);
+ inBuf.pos = 0;
+
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+ outBuf.dst = (char *) result + VARHDRSZ;
+ outBuf.size = slicelength;
+ outBuf.pos = 0;
+
+ /* Common decompression loop */
+ while (inBuf.pos < inBuf.size && outBuf.pos < outBuf.size)
+ {
+ ret = ZSTD_decompressStream(ZstdDecompressionCtx, &outBuf, &inBuf);
+ if (ZSTD_isError(ret))
+ {
+ pfree(result);
+ ZSTD_freeDCtx(ZstdDecompressionCtx);
+ ZSTD_CHECK_ERROR(ret, "zstd decompression failed");
+ }
+ }
+
+ Assert(outBuf.size == slicelength && outBuf.pos == slicelength);
+ SET_VARSIZE(result, outBuf.pos + VARHDRSZ);
+ ZSTD_freeDCtx(ZstdDecompressionCtx);
+ return result;
+#else
+ COMPRESSION_METHOD_NOT_SUPPORTED("zstd_nodict");
+ return NULL;
+#endif
+}
+
/*
* Extract compression ID from a varlena.
*
@@ -293,10 +428,17 @@ CompressionNameToMethod(const char *compression)
else if (strcmp(compression, "lz4") == 0)
{
#ifndef USE_LZ4
- NO_LZ4_SUPPORT();
+ COMPRESSION_METHOD_NOT_SUPPORTED("lz4");
#endif
return TOAST_LZ4_COMPRESSION;
}
+ else if (strcmp(compression, "zstd_nodict") == 0)
+ {
+#ifndef USE_ZSTD
+ COMPRESSION_METHOD_NOT_SUPPORTED("zstd_nodict");
+#endif
+ return TOAST_ZSTD_NODICT_COMPRESSION;
+ }
return InvalidCompressionMethod;
}
@@ -313,6 +455,8 @@ GetCompressionMethodName(char method)
return "pglz";
case TOAST_LZ4_COMPRESSION:
return "lz4";
+ case TOAST_ZSTD_NODICT_COMPRESSION:
+ return "zstd_nodict";
default:
elog(ERROR, "invalid compression method %c", method);
return NULL; /* keep compiler quiet */
@@ -326,6 +470,7 @@ setup_compression_info(char cmethod, Form_pg_attribute att)
/* initialize from the attribute’s default settings */
info.cmethod = cmethod;
+ info.zstd_level = DEFAULT_ZSTD_LEVEL;
info.cmp_ext = NULL;
/* If the compression method is not valid, use the current default */
@@ -337,6 +482,18 @@ setup_compression_info(char cmethod, Form_pg_attribute att)
case TOAST_PGLZ_COMPRESSION:
case TOAST_LZ4_COMPRESSION:
break;
+ case TOAST_ZSTD_NODICT_COMPRESSION:
+ {
+ AttributeOpts *aopt = get_attribute_options(att->attrelid, att->attnum);
+
+ if (aopt != NULL)
+ info.zstd_level = aopt->zstd_level;
+
+ info.cmp_ext = palloc(sizeof(varatt_cmp_extended));
+
+ VARATT_4BCE_SET_HDR(info.cmp_ext, TOAST_ZSTD_NODICT_COMPRESSION_ID);
+ }
+ break;
default:
elog(ERROR, "invalid compression method %c", info.cmethod);
}
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 83b537d51bf..5521d78bb48 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -68,6 +68,10 @@ toast_compress_datum(Datum value, CompressionInfo cmp)
tmp = lz4_compress_datum((const struct varlena *) value);
cmid = TOAST_LZ4_COMPRESSION_ID;
break;
+ case TOAST_ZSTD_NODICT_COMPRESSION:
+ tmp = zstd_nodict_compress_datum((const struct varlena *) value, cmp);
+ cmid = TOAST_ZSTD_NODICT_COMPRESSION_ID;
+ break;
default:
elog(ERROR, "invalid compression method %c", cmp.cmethod);
}
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index 3e4d5568bde..5b9151c7e16 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -5301,6 +5301,9 @@ pg_column_compression(PG_FUNCTION_ARGS)
case TOAST_LZ4_COMPRESSION_ID:
result = "lz4";
break;
+ case TOAST_ZSTD_NODICT_COMPRESSION_ID:
+ result = "zstd_nodict";
+ break;
default:
elog(ERROR, "invalid compression method id %d", cmid);
}
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 2f8cbd86759..948454e2093 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -460,6 +460,9 @@ static const struct config_enum_entry default_toast_compression_options[] = {
{"pglz", TOAST_PGLZ_COMPRESSION, false},
#ifdef USE_LZ4
{"lz4", TOAST_LZ4_COMPRESSION, false},
+#endif
+#ifdef USE_ZSTD
+ {"zstd_nodict", TOAST_ZSTD_NODICT_COMPRESSION, false},
#endif
{NULL, 0, false}
};
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 34826d01380..f2d2ca39514 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -756,7 +756,7 @@ autovacuum_worker_slots = 16 # autovacuum worker slots to allocate
#row_security = on
#default_table_access_method = 'heap'
#default_tablespace = '' # a tablespace name, '' uses the default
-#default_toast_compression = 'pglz' # 'pglz' or 'lz4'
+#default_toast_compression = 'pglz' # 'pglz' or 'lz4' or 'zstd_nodict'
#temp_tablespaces = '' # a list of tablespace names, '' uses
# only default tablespace
#check_function_bodies = on
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 1d08268393e..3831a7fab03 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -2171,8 +2171,9 @@ describeOneTableDetails(const char *schemaname,
/* these strings are literal in our syntax, so not translated. */
printTableAddCell(&cont, (compression[0] == 'p' ? "pglz" :
(compression[0] == 'l' ? "lz4" :
- (compression[0] == '\0' ? "" :
- "???"))),
+ (compression[0] == 'n' ? "zstd_nodict" :
+ (compression[0] == '\0' ? "" :
+ "???")))),
false, false);
}
diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c
index c916b9299a8..2441acf41ce 100644
--- a/src/bin/psql/tab-complete.in.c
+++ b/src/bin/psql/tab-complete.in.c
@@ -2875,11 +2875,11 @@ match_previous_words(int pattern_id,
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
- COMPLETE_WITH("n_distinct", "n_distinct_inherited");
+ COMPLETE_WITH("n_distinct", "n_distinct_inherited", "zstd_level");
/* ALTER TABLE ALTER [COLUMN] <foo> SET COMPRESSION */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION"))
- COMPLETE_WITH("DEFAULT", "PGLZ", "LZ4");
+ COMPLETE_WITH("DEFAULT", "PGLZ", "LZ4", "ZSTD_NODICT");
/* ALTER TABLE ALTER [COLUMN] <foo> SET EXPRESSION */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "EXPRESSION") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "EXPRESSION"))
diff --git a/src/include/access/toast_compression.h b/src/include/access/toast_compression.h
index 1aef65cde99..94d56c7a4ef 100644
--- a/src/include/access/toast_compression.h
+++ b/src/include/access/toast_compression.h
@@ -16,6 +16,10 @@
#include "varatt.h"
#include "catalog/pg_attribute.h"
+#ifdef USE_ZSTD
+#include <zstd.h>
+#endif
+
/*
* GUC support.
*
@@ -36,6 +40,13 @@ unsupported_meta_size(const varatt_cmp_extended *hdr)
return 0; /* unreachable */
}
+/* no metadata for plain-ZSTD */
+static inline uint32
+zstd_nodict_meta_size(const varatt_cmp_extended *hdr)
+{
+ return 0;
+}
+
/*
* TOAST compression methods enumeration.
*
@@ -43,10 +54,11 @@ unsupported_meta_size(const varatt_cmp_extended *hdr)
* VALUE : enum value
* META-SIZE-FN : Calculates algorithm metadata size.
*/
-#define TOAST_COMPRESSION_LIST \
- X(PGLZ, 0, unsupported_meta_size) \
- X(LZ4, 1, unsupported_meta_size) \
- X(INVALID, 2, unsupported_meta_size) /* sentinel */
+#define TOAST_COMPRESSION_LIST \
+ X(PGLZ, 0, unsupported_meta_size) \
+ X(LZ4, 1, unsupported_meta_size) \
+ X(ZSTD_NODICT, 2, zstd_nodict_meta_size) \
+ X(INVALID, 3, unsupported_meta_size) /* sentinel */
/* Compression algorithm identifiers */
typedef enum ToastCompressionId
@@ -96,6 +108,7 @@ toast_cmpid_meta_size(const varatt_cmp_extended *hdr)
typedef struct CompressionInfo
{
char cmethod;
+ int zstd_level;
varatt_cmp_extended *cmp_ext; /* non-NULL only if uses extended
* compression methods */
} CompressionInfo;
@@ -107,10 +120,20 @@ typedef struct CompressionInfo
*/
#define TOAST_PGLZ_COMPRESSION 'p'
#define TOAST_LZ4_COMPRESSION 'l'
+#define TOAST_ZSTD_NODICT_COMPRESSION 'n'
#define InvalidCompressionMethod '\0'
#define CompressionMethodIsValid(cm) ((cm) != InvalidCompressionMethod)
+#ifdef USE_ZSTD
+#define DEFAULT_ZSTD_LEVEL ZSTD_CLEVEL_DEFAULT
+#define MIN_ZSTD_LEVEL (int)-ZSTD_BLOCKSIZE_MAX
+#define MAX_ZSTD_LEVEL 22
+#else
+#define DEFAULT_ZSTD_LEVEL 0
+#define MIN_ZSTD_LEVEL 0
+#define MAX_ZSTD_LEVEL 0
+#endif
/* pglz compression/decompression routines */
extern struct varlena *pglz_compress_datum(const struct varlena *value);
@@ -124,6 +147,11 @@ extern struct varlena *lz4_decompress_datum(const struct varlena *value);
extern struct varlena *lz4_decompress_datum_slice(const struct varlena *value,
int32 slicelength);
+/* zstd nodict compression/decompression routines */
+extern struct varlena *zstd_nodict_compress_datum(const struct varlena *value, CompressionInfo cmp);
+extern struct varlena *zstd_nodict_decompress_datum(const struct varlena *value);
+extern struct varlena *zstd_nodict_decompress_datum_slice(const struct varlena *value, int32 slicelength);
+
/* other stuff */
extern ToastCompressionId toast_get_compression_id(struct varlena *attr);
extern char CompressionNameToMethod(const char *compression);
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index f4a4829ad17..b1859ef202b 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -35,7 +35,8 @@ typedef struct toast_compress_header
do { \
Assert((len) > 0 && (len) <= VARLENA_EXTSIZE_MASK); \
Assert((cm_method) == TOAST_PGLZ_COMPRESSION_ID || \
- (cm_method) == TOAST_LZ4_COMPRESSION_ID); \
+ (cm_method) == TOAST_LZ4_COMPRESSION_ID || \
+ (cm_method) == TOAST_ZSTD_NODICT_COMPRESSION_ID); \
if (!TOAST_CMPID_EXTENDED((cm_method))) \
{ \
((toast_compress_header *)(ptr))->tcinfo = \
diff --git a/src/include/utils/attoptcache.h b/src/include/utils/attoptcache.h
index f684a772af5..51d65ebd646 100644
--- a/src/include/utils/attoptcache.h
+++ b/src/include/utils/attoptcache.h
@@ -21,6 +21,7 @@ typedef struct AttributeOpts
int32 vl_len_; /* varlena header (do not touch directly!) */
float8 n_distinct;
float8 n_distinct_inherited;
+ int zstd_level;
} AttributeOpts;
extern AttributeOpts *get_attribute_options(Oid attrelid, int attnum);
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 4dd9ee7200d..c7e108a0f52 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -238,10 +238,11 @@ NOTICE: merging multiple inherited definitions of column "f1"
-- test default_toast_compression GUC
SET default_toast_compression = '';
ERROR: invalid value for parameter "default_toast_compression": ""
-HINT: Available values: pglz, lz4.
+HINT: Available values: pglz, lz4, zstd_nodict.
SET default_toast_compression = 'I do not exist compression';
ERROR: invalid value for parameter "default_toast_compression": "I do not exist compression"
-HINT: Available values: pglz, lz4.
+HINT: Available values: pglz, lz4, zstd_nodict.
+SET default_toast_compression = 'zstd_nodict';
SET default_toast_compression = 'lz4';
SET default_toast_compression = 'pglz';
-- test alter compression method
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 7bd7642b4b9..5b10d8c5259 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -233,6 +233,9 @@ HINT: Available values: pglz.
SET default_toast_compression = 'I do not exist compression';
ERROR: invalid value for parameter "default_toast_compression": "I do not exist compression"
HINT: Available values: pglz.
+SET default_toast_compression = 'zstd_nodict';
+ERROR: invalid value for parameter "default_toast_compression": "zstd_nodict"
+HINT: Available values: pglz.
SET default_toast_compression = 'lz4';
ERROR: invalid value for parameter "default_toast_compression": "lz4"
HINT: Available values: pglz.
diff --git a/src/test/regress/expected/compression_zstd_nodict.out b/src/test/regress/expected/compression_zstd_nodict.out
new file mode 100644
index 00000000000..d80814e0492
--- /dev/null
+++ b/src/test/regress/expected/compression_zstd_nodict.out
@@ -0,0 +1,152 @@
+\set HIDE_TOAST_COMPRESSION false
+-- Ensure stable results regardless of the installation's default.
+SET default_toast_compression = 'pglz';
+----------------------------------------------------------------
+-- 1. Create Test Table with Zstd Compression
+----------------------------------------------------------------
+DROP TABLE IF EXISTS cmdata_zstd_nodict CASCADE;
+NOTICE: table "cmdata_zstd_nodict" does not exist, skipping
+CREATE TABLE cmdata_zstd_nodict (
+ f1 TEXT COMPRESSION zstd_nodict
+);
+----------------------------------------------------------------
+-- 2. Insert Data Rows
+----------------------------------------------------------------
+DO $$
+BEGIN
+ FOR i IN 1..15 LOOP
+ INSERT INTO cmdata_zstd_nodict (f1) VALUES (repeat('1234567890', 1004));
+ END LOOP;
+END $$;
+----------------------------------------------------------------
+-- 3. Verify Table Structure and Compression Settings
+----------------------------------------------------------------
+-- Table Structure for cmdata_zstd
+\d+ cmdata_zstd_nodict;
+ Table "public.cmdata_zstd_nodict"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | zstd_nodict | |
+
+-- Compression Settings for f1 Column
+SELECT pg_column_compression(f1) AS compression_method,
+ count(*) AS row_count
+FROM cmdata_zstd_nodict
+GROUP BY pg_column_compression(f1);
+ compression_method | row_count
+--------------------+-----------
+ zstd_nodict | 15
+(1 row)
+
+----------------------------------------------------------------
+-- 4. Decompression Tests
+----------------------------------------------------------------
+-- Decompression Slice Test (Extracting Substrings)
+SELECT SUBSTR(f1, 200, 50) AS data_slice
+FROM cmdata_zstd_nodict;
+ data_slice
+----------------------------------------------------
+ 01234567890123456789012345678901234567890123456789
+ 01234567890123456789012345678901234567890123456789
+ 01234567890123456789012345678901234567890123456789
+ 01234567890123456789012345678901234567890123456789
+ 01234567890123456789012345678901234567890123456789
+ 01234567890123456789012345678901234567890123456789
+ 01234567890123456789012345678901234567890123456789
+ 01234567890123456789012345678901234567890123456789
+ 01234567890123456789012345678901234567890123456789
+ 01234567890123456789012345678901234567890123456789
+ 01234567890123456789012345678901234567890123456789
+ 01234567890123456789012345678901234567890123456789
+ 01234567890123456789012345678901234567890123456789
+ 01234567890123456789012345678901234567890123456789
+ 01234567890123456789012345678901234567890123456789
+(15 rows)
+
+----------------------------------------------------------------
+-- 5. Test Table Creation with LIKE INCLUDING COMPRESSION
+----------------------------------------------------------------
+DROP TABLE IF EXISTS cmdata_zstd_nodict_2;
+NOTICE: table "cmdata_zstd_nodict_2" does not exist, skipping
+CREATE TABLE cmdata_zstd_nodict_2 (LIKE cmdata_zstd_nodict INCLUDING COMPRESSION);
+-- Table Structure for cmdata_zstd_2
+\d+ cmdata_zstd_nodict_2;
+ Table "public.cmdata_zstd_nodict_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | zstd_nodict | |
+
+DROP TABLE cmdata_zstd_nodict_2;
+----------------------------------------------------------------
+-- 6. Materialized View Compression Test
+----------------------------------------------------------------
+DROP MATERIALIZED VIEW IF EXISTS compressmv_zstd_nodict;
+NOTICE: materialized view "compressmv_zstd_nodict" does not exist, skipping
+CREATE MATERIALIZED VIEW compressmv_zstd_nodict AS
+ SELECT f1 FROM cmdata_zstd_nodict;
+-- Materialized View Structure for compressmv_zstd
+\d+ compressmv_zstd_nodict;
+ Materialized view "public.compressmv_zstd_nodict"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | | |
+View definition:
+ SELECT f1
+ FROM cmdata_zstd_nodict;
+
+-- Materialized View Compression Check
+SELECT pg_column_compression(f1) AS mv_compression
+FROM compressmv_zstd_nodict;
+ mv_compression
+----------------
+ zstd_nodict
+ zstd_nodict
+ zstd_nodict
+ zstd_nodict
+ zstd_nodict
+ zstd_nodict
+ zstd_nodict
+ zstd_nodict
+ zstd_nodict
+ zstd_nodict
+ zstd_nodict
+ zstd_nodict
+ zstd_nodict
+ zstd_nodict
+ zstd_nodict
+(15 rows)
+
+----------------------------------------------------------------
+-- 7. Additional Updates and Round-Trip Tests
+----------------------------------------------------------------
+-- Update some rows to check if the dictionary remains effective after modifications.
+UPDATE cmdata_zstd_nodict
+SET f1 = f1 || ' UPDATED';
+-- Verification of Updated Rows
+SELECT SUBSTR(f1, LENGTH(f1) - 7 + 1, 7) AS preview
+FROM cmdata_zstd_nodict;
+ preview
+---------
+ UPDATED
+ UPDATED
+ UPDATED
+ UPDATED
+ UPDATED
+ UPDATED
+ UPDATED
+ UPDATED
+ UPDATED
+ UPDATED
+ UPDATED
+ UPDATED
+ UPDATED
+ UPDATED
+ UPDATED
+(15 rows)
+
+----------------------------------------------------------------
+-- 8. Clean Up
+----------------------------------------------------------------
+DROP MATERIALIZED VIEW compressmv_zstd_nodict;
+DROP TABLE cmdata_zstd_nodict;
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/test/regress/expected/compression_zstd_nodict_1.out b/src/test/regress/expected/compression_zstd_nodict_1.out
new file mode 100644
index 00000000000..161d11fcae2
--- /dev/null
+++ b/src/test/regress/expected/compression_zstd_nodict_1.out
@@ -0,0 +1,103 @@
+\set HIDE_TOAST_COMPRESSION false
+-- Ensure stable results regardless of the installation's default.
+SET default_toast_compression = 'pglz';
+----------------------------------------------------------------
+-- 1. Create Test Table with Zstd Compression
+----------------------------------------------------------------
+DROP TABLE IF EXISTS cmdata_zstd_nodict CASCADE;
+NOTICE: table "cmdata_zstd_nodict" does not exist, skipping
+CREATE TABLE cmdata_zstd_nodict (
+ f1 TEXT COMPRESSION zstd_nodict
+);
+ERROR: compression method zstd_nodict not supported
+DETAIL: This functionality requires the server to be built with zstd_nodict support.
+----------------------------------------------------------------
+-- 2. Insert Data Rows
+----------------------------------------------------------------
+DO $$
+BEGIN
+ FOR i IN 1..15 LOOP
+ INSERT INTO cmdata_zstd_nodict (f1) VALUES (repeat('1234567890', 1004));
+ END LOOP;
+END $$;
+ERROR: relation "cmdata_zstd_nodict" does not exist
+LINE 1: INSERT INTO cmdata_zstd_nodict (f1) VALUES (repeat('12345678...
+ ^
+QUERY: INSERT INTO cmdata_zstd_nodict (f1) VALUES (repeat('1234567890', 1004))
+CONTEXT: PL/pgSQL function inline_code_block line 4 at SQL statement
+----------------------------------------------------------------
+-- 3. Verify Table Structure and Compression Settings
+----------------------------------------------------------------
+-- Table Structure for cmdata_zstd
+\d+ cmdata_zstd_nodict;
+-- Compression Settings for f1 Column
+SELECT pg_column_compression(f1) AS compression_method,
+ count(*) AS row_count
+FROM cmdata_zstd_nodict
+GROUP BY pg_column_compression(f1);
+ERROR: relation "cmdata_zstd_nodict" does not exist
+LINE 3: FROM cmdata_zstd_nodict
+ ^
+----------------------------------------------------------------
+-- 4. Decompression Tests
+----------------------------------------------------------------
+-- Decompression Slice Test (Extracting Substrings)
+SELECT SUBSTR(f1, 200, 50) AS data_slice
+FROM cmdata_zstd_nodict;
+ERROR: relation "cmdata_zstd_nodict" does not exist
+LINE 2: FROM cmdata_zstd_nodict;
+ ^
+----------------------------------------------------------------
+-- 5. Test Table Creation with LIKE INCLUDING COMPRESSION
+----------------------------------------------------------------
+DROP TABLE IF EXISTS cmdata_zstd_nodict_2;
+NOTICE: table "cmdata_zstd_nodict_2" does not exist, skipping
+CREATE TABLE cmdata_zstd_nodict_2 (LIKE cmdata_zstd_nodict INCLUDING COMPRESSION);
+ERROR: relation "cmdata_zstd_nodict" does not exist
+LINE 1: CREATE TABLE cmdata_zstd_nodict_2 (LIKE cmdata_zstd_nodict I...
+ ^
+-- Table Structure for cmdata_zstd_2
+\d+ cmdata_zstd_nodict_2;
+DROP TABLE cmdata_zstd_nodict_2;
+ERROR: table "cmdata_zstd_nodict_2" does not exist
+----------------------------------------------------------------
+-- 6. Materialized View Compression Test
+----------------------------------------------------------------
+DROP MATERIALIZED VIEW IF EXISTS compressmv_zstd_nodict;
+NOTICE: materialized view "compressmv_zstd_nodict" does not exist, skipping
+CREATE MATERIALIZED VIEW compressmv_zstd_nodict AS
+ SELECT f1 FROM cmdata_zstd_nodict;
+ERROR: relation "cmdata_zstd_nodict" does not exist
+LINE 2: SELECT f1 FROM cmdata_zstd_nodict;
+ ^
+-- Materialized View Structure for compressmv_zstd
+\d+ compressmv_zstd_nodict;
+-- Materialized View Compression Check
+SELECT pg_column_compression(f1) AS mv_compression
+FROM compressmv_zstd_nodict;
+ERROR: relation "compressmv_zstd_nodict" does not exist
+LINE 2: FROM compressmv_zstd_nodict;
+ ^
+----------------------------------------------------------------
+-- 7. Additional Updates and Round-Trip Tests
+----------------------------------------------------------------
+-- Update some rows to check if the dictionary remains effective after modifications.
+UPDATE cmdata_zstd_nodict
+SET f1 = f1 || ' UPDATED';
+ERROR: relation "cmdata_zstd_nodict" does not exist
+LINE 1: UPDATE cmdata_zstd_nodict
+ ^
+-- Verification of Updated Rows
+SELECT SUBSTR(f1, LENGTH(f1) - 7 + 1, 7) AS preview
+FROM cmdata_zstd_nodict;
+ERROR: relation "cmdata_zstd_nodict" does not exist
+LINE 2: FROM cmdata_zstd_nodict;
+ ^
+----------------------------------------------------------------
+-- 8. Clean Up
+----------------------------------------------------------------
+DROP MATERIALIZED VIEW compressmv_zstd_nodict;
+ERROR: materialized view "compressmv_zstd_nodict" does not exist
+DROP TABLE cmdata_zstd_nodict;
+ERROR: table "cmdata_zstd_nodict" does not exist
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index a424be2a6bf..75cea22c418 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -123,7 +123,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# The stats test resets stats, so nothing else needing stats access can be in
# this group.
# ----------
-test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression memoize stats predicate numa
+test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression compression_zstd_nodict memoize stats predicate numa
# event_trigger depends on create_am and cannot run concurrently with
# any test that runs DDL
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 490595fcfb2..27979eb7997 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -102,6 +102,7 @@ CREATE TABLE cminh() INHERITS (cmdata, cmdata3);
-- test default_toast_compression GUC
SET default_toast_compression = '';
SET default_toast_compression = 'I do not exist compression';
+SET default_toast_compression = 'zstd_nodict';
SET default_toast_compression = 'lz4';
SET default_toast_compression = 'pglz';
diff --git a/src/test/regress/sql/compression_zstd_nodict.sql b/src/test/regress/sql/compression_zstd_nodict.sql
new file mode 100644
index 00000000000..e1f18004cc9
--- /dev/null
+++ b/src/test/regress/sql/compression_zstd_nodict.sql
@@ -0,0 +1,82 @@
+\set HIDE_TOAST_COMPRESSION false
+
+-- Ensure stable results regardless of the installation's default.
+SET default_toast_compression = 'pglz';
+
+----------------------------------------------------------------
+-- 1. Create Test Table with Zstd Compression
+----------------------------------------------------------------
+DROP TABLE IF EXISTS cmdata_zstd_nodict CASCADE;
+CREATE TABLE cmdata_zstd_nodict (
+ f1 TEXT COMPRESSION zstd_nodict
+);
+
+----------------------------------------------------------------
+-- 2. Insert Data Rows
+----------------------------------------------------------------
+DO $$
+BEGIN
+ FOR i IN 1..15 LOOP
+ INSERT INTO cmdata_zstd_nodict (f1) VALUES (repeat('1234567890', 1004));
+ END LOOP;
+END $$;
+
+----------------------------------------------------------------
+-- 3. Verify Table Structure and Compression Settings
+----------------------------------------------------------------
+-- Table Structure for cmdata_zstd
+\d+ cmdata_zstd_nodict;
+
+-- Compression Settings for f1 Column
+SELECT pg_column_compression(f1) AS compression_method,
+ count(*) AS row_count
+FROM cmdata_zstd_nodict
+GROUP BY pg_column_compression(f1);
+
+----------------------------------------------------------------
+-- 4. Decompression Tests
+----------------------------------------------------------------
+-- Decompression Slice Test (Extracting Substrings)
+SELECT SUBSTR(f1, 200, 50) AS data_slice
+FROM cmdata_zstd_nodict;
+
+----------------------------------------------------------------
+-- 5. Test Table Creation with LIKE INCLUDING COMPRESSION
+----------------------------------------------------------------
+DROP TABLE IF EXISTS cmdata_zstd_nodict_2;
+CREATE TABLE cmdata_zstd_nodict_2 (LIKE cmdata_zstd_nodict INCLUDING COMPRESSION);
+-- Table Structure for cmdata_zstd_2
+\d+ cmdata_zstd_nodict_2;
+DROP TABLE cmdata_zstd_nodict_2;
+
+----------------------------------------------------------------
+-- 6. Materialized View Compression Test
+----------------------------------------------------------------
+DROP MATERIALIZED VIEW IF EXISTS compressmv_zstd_nodict;
+CREATE MATERIALIZED VIEW compressmv_zstd_nodict AS
+ SELECT f1 FROM cmdata_zstd_nodict;
+
+-- Materialized View Structure for compressmv_zstd
+\d+ compressmv_zstd_nodict;
+
+-- Materialized View Compression Check
+SELECT pg_column_compression(f1) AS mv_compression
+FROM compressmv_zstd_nodict;
+
+----------------------------------------------------------------
+-- 7. Additional Updates and Round-Trip Tests
+----------------------------------------------------------------
+-- Update some rows to check if the dictionary remains effective after modifications.
+UPDATE cmdata_zstd_nodict
+SET f1 = f1 || ' UPDATED';
+
+-- Verification of Updated Rows
+SELECT SUBSTR(f1, LENGTH(f1) - 7 + 1, 7) AS preview
+FROM cmdata_zstd_nodict;
+----------------------------------------------------------------
+-- 8. Clean Up
+----------------------------------------------------------------
+DROP MATERIALIZED VIEW compressmv_zstd_nodict;
+DROP TABLE cmdata_zstd_nodict;
+
+\set HIDE_TOAST_COMPRESSION true
--
2.47.1
From 3043cfc51dc345683b5f2aac6b0c431680a476b6 Mon Sep 17 00:00:00 2001
From: Nikhil Kumar Veldanda <nikhilkv@amazon.com>
Date: Sat, 3 May 2025 01:57:15 +0000
Subject: [PATCH v20 1/2] varattrib_4b design proposal to make it extended to
support multiple compression algorithms.
---
contrib/amcheck/verify_heapam.c | 2 +-
src/backend/access/brin/brin_tuple.c | 4 +-
src/backend/access/common/detoast.c | 6 +-
src/backend/access/common/indextuple.c | 5 +-
src/backend/access/common/toast_compression.c | 38 ++++++++-
src/backend/access/common/toast_internals.c | 18 ++--
src/backend/access/table/toast_helper.c | 4 +-
src/include/access/toast_compression.h | 85 ++++++++++++++++---
src/include/access/toast_internals.h | 38 ++++++---
src/include/varatt.h | 75 +++++++++++++++-
src/tools/pgindent/typedefs.list | 2 +
11 files changed, 232 insertions(+), 45 deletions(-)
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index aa9cccd1da4..d7c2ac6951a 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -1786,7 +1786,7 @@ check_tuple_attribute(HeapCheckContext *ctx)
bool valid = false;
/* Compressed attributes should have a valid compression method */
- cmid = TOAST_COMPRESS_METHOD(&toast_pointer);
+ cmid = toast_get_compression_id(attr);
switch (cmid)
{
/* List of all valid compression method IDs */
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 861f397e6db..9c1e22e98c6 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -223,6 +223,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
{
Datum cvalue;
char compression;
+ CompressionInfo cmp;
Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
keyno);
@@ -237,7 +238,8 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
else
compression = InvalidCompressionMethod;
- cvalue = toast_compress_datum(value, compression);
+ cmp = setup_compression_info(compression, att);
+ cvalue = toast_compress_datum(value, cmp);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index 62651787742..01419d1c65f 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -478,7 +478,7 @@ toast_decompress_datum(struct varlena *attr)
* Fetch the compression method id stored in the compression header and
* decompress the data using the appropriate decompression routine.
*/
- cmid = TOAST_COMPRESS_METHOD(attr);
+ cmid = VARDATA_COMPRESSED_GET_COMPRESS_METHOD(attr);
switch (cmid)
{
case TOAST_PGLZ_COMPRESSION_ID:
@@ -514,14 +514,14 @@ toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
* have been seen to give wrong results if passed an output size that is
* more than the data's true decompressed size.
*/
- if ((uint32) slicelength >= TOAST_COMPRESS_EXTSIZE(attr))
+ if ((uint32) slicelength >= VARDATA_COMPRESSED_GET_EXTSIZE(attr))
return toast_decompress_datum(attr);
/*
* Fetch the compression method id stored in the compression header and
* decompress the data slice using the appropriate decompression routine.
*/
- cmid = TOAST_COMPRESS_METHOD(attr);
+ cmid = VARDATA_COMPRESSED_GET_COMPRESS_METHOD(attr);
switch (cmid)
{
case TOAST_PGLZ_COMPRESSION_ID:
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 1986b943a28..0386f5a1491 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -123,9 +123,10 @@ index_form_tuple_context(TupleDesc tupleDescriptor,
att->attstorage == TYPSTORAGE_MAIN))
{
Datum cvalue;
+ CompressionInfo cmp;
- cvalue = toast_compress_datum(untoasted_values[i],
- att->attcompression);
+ cmp = setup_compression_info(att->attcompression, att);
+ cvalue = toast_compress_datum(untoasted_values[i], cmp);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c
index 21f2f4af97e..5e5d42d80ef 100644
--- a/src/backend/access/common/toast_compression.c
+++ b/src/backend/access/common/toast_compression.c
@@ -21,6 +21,7 @@
#include "access/toast_compression.h"
#include "common/pg_lzcompress.h"
#include "varatt.h"
+#include "utils/attoptcache.h"
/* GUC */
int default_toast_compression = TOAST_PGLZ_COMPRESSION;
@@ -266,7 +267,10 @@ toast_get_compression_id(struct varlena *attr)
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- if (VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
+ if (VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer)
+ && VARATT_EXTERNAL_GET_COMPRESS_METHOD(toast_pointer) == VARATT_4BCE_MASK)
+ cmid = VARDATA_COMPRESSED_GET_COMPRESS_METHOD(detoast_external_attr(attr));
+ else
cmid = VARATT_EXTERNAL_GET_COMPRESS_METHOD(toast_pointer);
}
else if (VARATT_IS_COMPRESSED(attr))
@@ -314,3 +318,35 @@ GetCompressionMethodName(char method)
return NULL; /* keep compiler quiet */
}
}
+
+CompressionInfo
+setup_compression_info(char cmethod, Form_pg_attribute att)
+{
+ CompressionInfo info;
+
+ /* initialize from the attribute’s default settings */
+ info.cmethod = cmethod;
+ info.cmp_ext = NULL;
+
+ /* If the compression method is not valid, use the current default */
+ if (!CompressionMethodIsValid(cmethod))
+ info.cmethod = default_toast_compression;
+
+ switch (info.cmethod)
+ {
+ case TOAST_PGLZ_COMPRESSION:
+ case TOAST_LZ4_COMPRESSION:
+ break;
+ default:
+ elog(ERROR, "invalid compression method %c", info.cmethod);
+ }
+
+ return info;
+}
+
+void
+free_compression_info(CompressionInfo *info)
+{
+ if (info->cmp_ext != NULL)
+ pfree(info->cmp_ext);
+}
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 7d8be8346ce..83b537d51bf 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -43,25 +43,22 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value, char cmethod)
+toast_compress_datum(Datum value, CompressionInfo cmp)
{
struct varlena *tmp = NULL;
int32 valsize;
ToastCompressionId cmid = TOAST_INVALID_COMPRESSION_ID;
+ varatt_cmp_extended *cmp_ext = cmp.cmp_ext;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- /* If the compression method is not valid, use the current default */
- if (!CompressionMethodIsValid(cmethod))
- cmethod = default_toast_compression;
-
/*
* Call appropriate compression routine for the compression method.
*/
- switch (cmethod)
+ switch (cmp.cmethod)
{
case TOAST_PGLZ_COMPRESSION:
tmp = pglz_compress_datum((const struct varlena *) value);
@@ -72,11 +69,14 @@ toast_compress_datum(Datum value, char cmethod)
cmid = TOAST_LZ4_COMPRESSION_ID;
break;
default:
- elog(ERROR, "invalid compression method %c", cmethod);
+ elog(ERROR, "invalid compression method %c", cmp.cmethod);
}
if (tmp == NULL)
+ {
+ free_compression_info(&cmp);
return PointerGetDatum(NULL);
+ }
/*
* We recheck the actual size even if compression reports success, because
@@ -92,13 +92,15 @@ toast_compress_datum(Datum value, char cmethod)
{
/* successful compression */
Assert(cmid != TOAST_INVALID_COMPRESSION_ID);
- TOAST_COMPRESS_SET_SIZE_AND_COMPRESS_METHOD(tmp, valsize, cmid);
+ TOAST_COMPRESS_SET_SIZE_AND_COMPRESS_METHOD_INFO(tmp, valsize, cmid, cmp_ext);
+ free_compression_info(&cmp);
return PointerGetDatum(tmp);
}
else
{
/* incompressible data */
pfree(tmp);
+ free_compression_info(&cmp);
return PointerGetDatum(NULL);
}
}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index b60fab0a4d2..ba5af5db404 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -229,8 +229,10 @@ toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
Datum *value = &ttc->ttc_values[attribute];
Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ Form_pg_attribute att = TupleDescAttr(ttc->ttc_rel->rd_att, attribute);
+ CompressionInfo cmp = setup_compression_info(attr->tai_compression, att);
- new_value = toast_compress_datum(*value, attr->tai_compression);
+ new_value = toast_compress_datum(*value, cmp);
if (DatumGetPointer(new_value) != NULL)
{
diff --git a/src/include/access/toast_compression.h b/src/include/access/toast_compression.h
index 13c4612ceed..1aef65cde99 100644
--- a/src/include/access/toast_compression.h
+++ b/src/include/access/toast_compression.h
@@ -13,6 +13,9 @@
#ifndef TOAST_COMPRESSION_H
#define TOAST_COMPRESSION_H
+#include "varatt.h"
+#include "catalog/pg_attribute.h"
+
/*
* GUC support.
*
@@ -23,24 +26,80 @@
extern PGDLLIMPORT int default_toast_compression;
/*
- * Built-in compression method ID. The toast compression header will store
- * this in the first 2 bits of the raw length. These built-in compression
- * method IDs are directly mapped to the built-in compression methods.
+ * Stub errors if someone tries to query metadata size
+ * for an algorithm that doesn’t support it.
+ */
+static inline uint32
+unsupported_meta_size(const varatt_cmp_extended *hdr)
+{
+ elog(ERROR, "toast_cmpid_meta_size called for unsupported compression algorithm");
+ return 0; /* unreachable */
+}
+
+/*
+ * TOAST compression methods enumeration.
*
- * Don't use these values for anything other than understanding the meaning
- * of the raw bits from a varlena; in particular, if the goal is to identify
- * a compression method, use the constants TOAST_PGLZ_COMPRESSION, etc.
- * below. We might someday support more than 4 compression methods, but
- * we can never have more than 4 values in this enum, because there are
- * only 2 bits available in the places where this is stored.
+ * NAME : algorithm identifier
+ * VALUE : enum value
+ * META-SIZE-FN : Calculates algorithm metadata size.
*/
+#define TOAST_COMPRESSION_LIST \
+ X(PGLZ, 0, unsupported_meta_size) \
+ X(LZ4, 1, unsupported_meta_size) \
+ X(INVALID, 2, unsupported_meta_size) /* sentinel */
+
+/* Compression algorithm identifiers */
typedef enum ToastCompressionId
{
- TOAST_PGLZ_COMPRESSION_ID = 0,
- TOAST_LZ4_COMPRESSION_ID = 1,
- TOAST_INVALID_COMPRESSION_ID = 2,
+#define X(name,val,fn) TOAST_##name##_COMPRESSION_ID = (val),
+ TOAST_COMPRESSION_LIST
+#undef X
} ToastCompressionId;
+/* lookup table to check if compression method uses extended format */
+static const bool toast_cmpid_extended[] = {
+#define X(name,val,fn) \
+ /* PGLZ, LZ4 don't use extended format */ \
+ [TOAST_##name##_COMPRESSION_ID] = \
+ ((val) != TOAST_PGLZ_COMPRESSION_ID && \
+ (val) != TOAST_LZ4_COMPRESSION_ID && \
+ (val) != TOAST_INVALID_COMPRESSION_ID),
+ TOAST_COMPRESSION_LIST
+#undef X
+};
+
+#define TOAST_CMPID_EXTENDED(alg) \
+ (toast_cmpid_extended[alg])
+
+/*
+ * Prototype for a per-datum metadata-size callback:
+ * given a pointer to the extended header, return
+ * how many metadata bytes follow it.
+ */
+typedef uint32 (*ToastMetaSizeFn) (const varatt_cmp_extended *hdr);
+
+/* Callback table—indexed by ToastCompressionId */
+static const ToastMetaSizeFn toast_meta_size_fns[] = {
+#define X(name,val,fn) [TOAST_##name##_COMPRESSION_ID] = fn,
+ TOAST_COMPRESSION_LIST
+#undef X
+};
+
+/* Calculates algorithm metadata size */
+static inline uint32
+toast_cmpid_meta_size(const varatt_cmp_extended *hdr)
+{
+ Assert(hdr != NULL);
+ return toast_meta_size_fns[hdr->cmp_alg] (hdr);
+}
+
+typedef struct CompressionInfo
+{
+ char cmethod;
+ varatt_cmp_extended *cmp_ext; /* non-NULL only if uses extended
+ * compression methods */
+} CompressionInfo;
+
/*
* Built-in compression methods. pg_attribute will store these in the
* attcompression column. In attcompression, InvalidCompressionMethod
@@ -69,5 +128,7 @@ extern struct varlena *lz4_decompress_datum_slice(const struct varlena *value,
extern ToastCompressionId toast_get_compression_id(struct varlena *attr);
extern char CompressionNameToMethod(const char *compression);
extern const char *GetCompressionMethodName(char method);
+extern CompressionInfo setup_compression_info(char cmethod, Form_pg_attribute att);
+extern void free_compression_info(CompressionInfo *info);
#endif /* TOAST_COMPRESSION_H */
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 06ae8583c1e..f4a4829ad17 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -31,21 +31,33 @@ typedef struct toast_compress_header
* Utilities for manipulation of header information for compressed
* toast entries.
*/
-#define TOAST_COMPRESS_EXTSIZE(ptr) \
- (((toast_compress_header *) (ptr))->tcinfo & VARLENA_EXTSIZE_MASK)
-#define TOAST_COMPRESS_METHOD(ptr) \
- (((toast_compress_header *) (ptr))->tcinfo >> VARLENA_EXTSIZE_BITS)
-
-#define TOAST_COMPRESS_SET_SIZE_AND_COMPRESS_METHOD(ptr, len, cm_method) \
- do { \
- Assert((len) > 0 && (len) <= VARLENA_EXTSIZE_MASK); \
- Assert((cm_method) == TOAST_PGLZ_COMPRESSION_ID || \
- (cm_method) == TOAST_LZ4_COMPRESSION_ID); \
- ((toast_compress_header *) (ptr))->tcinfo = \
- (len) | ((uint32) (cm_method) << VARLENA_EXTSIZE_BITS); \
+#define TOAST_COMPRESS_SET_SIZE_AND_COMPRESS_METHOD_INFO(ptr, len, cm_method, cmp_ext) \
+ do { \
+ Assert((len) > 0 && (len) <= VARLENA_EXTSIZE_MASK); \
+ Assert((cm_method) == TOAST_PGLZ_COMPRESSION_ID || \
+ (cm_method) == TOAST_LZ4_COMPRESSION_ID); \
+ if (!TOAST_CMPID_EXTENDED((cm_method))) \
+ { \
+ ((toast_compress_header *)(ptr))->tcinfo = \
+ ((uint32)(len)) | ((uint32)(cm_method) << VARLENA_EXTSIZE_BITS); \
+ } \
+ else \
+ { \
+ /* extended path: mark EXT flag in tcinfo */ \
+ ((toast_compress_header *)(ptr))->tcinfo = \
+ ((uint32)(len)) | \
+ ((uint32)(VARATT_4BCE_MASK) << VARLENA_EXTSIZE_BITS); \
+ Assert((cmp_ext) != NULL); \
+ /* copy header + algorithm-specific metadata */ \
+ memcpy( \
+ VARATT_4BCE_HDR_PTR(ptr), \
+ (const void *)(cmp_ext), sizeof(varatt_cmp_extended) + \
+ toast_cmpid_meta_size((const varatt_cmp_extended *)(cmp_ext)) \
+ ); \
+ } \
} while (0)
-extern Datum toast_compress_datum(Datum value, char cmethod);
+extern Datum toast_compress_datum(Datum value, CompressionInfo cmp);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/varatt.h b/src/include/varatt.h
index 2e8564d4998..91460f313c5 100644
--- a/src/include/varatt.h
+++ b/src/include/varatt.h
@@ -328,7 +328,8 @@ typedef struct
#define VARDATA_COMPRESSED_GET_EXTSIZE(PTR) \
(((varattrib_4b *) (PTR))->va_compressed.va_tcinfo & VARLENA_EXTSIZE_MASK)
#define VARDATA_COMPRESSED_GET_COMPRESS_METHOD(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo >> VARLENA_EXTSIZE_BITS)
+ ( (VARATT_IS_4BCE(PTR)) ? (VARATT_4BCE_CMP_METHOD(PTR)) \
+ : (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo >> VARLENA_EXTSIZE_BITS))
/* Same for external Datums; but note argument is a struct varatt_external */
#define VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) \
@@ -340,8 +341,17 @@ typedef struct
do { \
Assert((cm) == TOAST_PGLZ_COMPRESSION_ID || \
(cm) == TOAST_LZ4_COMPRESSION_ID); \
- ((toast_pointer).va_extinfo = \
- (len) | ((uint32) (cm) << VARLENA_EXTSIZE_BITS)); \
+ if (!TOAST_CMPID_EXTENDED((cm))) \
+ { \
+ /* Store the actual method in va_extinfo */ \
+ ((toast_pointer).va_extinfo = \
+ (len) | ((uint32) (cm) << VARLENA_EXTSIZE_BITS)); \
+ } \
+ else \
+ { \
+ /* Store 11 in the top 2 bits, meaning "extended" method. */ \
+ (toast_pointer).va_extinfo = (uint32)(len) | (VARATT_4BCE_MASK << VARLENA_EXTSIZE_BITS ); \
+ } \
} while (0)
/*
@@ -355,4 +365,63 @@ typedef struct
(VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) < \
(toast_pointer).va_rawsize - VARHDRSZ)
+/*
+ * varatt_cmp_extended: an optional per‐datum header for extended compression method.
+ * Only used when va_tcinfo’s top two bits are “11”.
+ */
+typedef struct varatt_cmp_extended
+{
+ uint8 cmp_alg; /* algorithm id (0–255) */
+ char cmp_meta[FLEXIBLE_ARRAY_MEMBER]; /* algorithm‐specific
+ * metadata */
+} varatt_cmp_extended;
+
+/*
+ * 1) Detect the extended‐compression flag in va_tcinfo
+ * (top 2 bits = 0b11 indicate “cmp_ext” path)
+ */
+#define VARATT_4BCE_MASK 0x3
+
+#define VARATT_IS_4BCE(PTR) \
+ ((((varattrib_4b *)(PTR))->va_compressed.va_tcinfo >> VARLENA_EXTSIZE_BITS) == VARATT_4BCE_MASK)
+
+/*
+ * 2) Pointer to varatt_cmp_extended header (just after the 8-byte varattrib_4b compressed headers)
+ */
+#define VARATT_4BCE_HDR_PTR(PTR) \
+ ((varatt_cmp_extended *) ((char *)(PTR) + VARHDRSZ_COMPRESSED))
+
+/* get the algorithm ID */
+#define VARATT_4BCE_CMP_METHOD(PTR) \
+ ((ToastCompressionId)VARATT_4BCE_HDR_PTR(PTR)->cmp_alg)
+
+/* set the algorithm ID */
+#define VARATT_4BCE_SET_HDR(EXT_PTR, alg) ((varatt_cmp_extended*)(EXT_PTR))->cmp_alg = (uint8)(alg);
+
+/*
+ * 3) Helpers to find metadata vs payload:
+ * – cmp_meta[] pointer
+ * – compressed‐bytes pointer
+ * – compressed-bytes size
+ * – total header size
+ */
+
+/* pointer to compression algorithm's metadata */
+#define VARATT_4BCE_META_PTR(PTR) \
+ ((void *)VARATT_4BCE_HDR_PTR(PTR)->cmp_meta)
+
+/* pointer to compressed bytes (after metadata) */
+#define VARATT_4BCE_PAYLOAD_PTR(PTR) \
+ ((void *) ((char *)VARATT_4BCE_META_PTR(PTR) + toast_cmpid_meta_size(VARATT_4BCE_HDR_PTR(PTR))))
+
+/* number of compressed‐payload bytes */
+#define VARATT_4BCE_PAYLOAD_SIZE(PTR) \
+ ( VARSIZE_4B(PTR) - VARHDRSZ_COMPRESSED - sizeof(varatt_cmp_extended) \
+ - toast_cmpid_meta_size(VARATT_4BCE_HDR_PTR(PTR)))
+
+/* total header+meta size before payload */
+#define VARATT_4BCE_HDRSZ(EXT_PTR) \
+ ( VARHDRSZ_COMPRESSED + sizeof(varatt_cmp_extended) \
+ + toast_cmpid_meta_size(EXT_PTR))
+
#endif
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index e5879e00dff..ea28675e0c9 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -482,6 +482,7 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressFileHandle
+CompressionInfo
CompressionLocation
CompressorState
ComputeXidHorizonsResult
@@ -4153,6 +4154,7 @@ uuid_t
va_list
vacuumingOptions
validate_string_relopt
+varatt_cmp_extended
varatt_expanded
varattrib_1b
varattrib_1b_e
base-commit: a675149e87706d01e4007150a0124b89bdef08be
--
2.47.1
