Hi Michael,
I’m sending a small test-only patch that increases test coverage for
toast_internals.c from 88.5% -> 95.8%.
--
Nikhil Veldanda
v1-0001-Add-tests-coverage-for-TOAST-value-reuse-and-OID-.patch
Description: Binary data
Hi michael,
I have a question regarding TOAST pointer handling.
As I understand, in the current design, each attribute in a HeapTuple
can have its own TOAST pointer, and TOAST pointers are possible only
for top-level attributes.
Would it make sense to maintain an array for ttc_toast_pointer_size
Hi all,
On Sun, Aug 3, 2025 at 8:16 PM Michael Paquier wrote:
> if (VARATT_IS_SHORT(dval))
> {
> data_p = VARDATA_SHORT(dval);
> data_todo = VARSIZE_SHORT(dval) - VARHDRSZ_SHORT;
> toast_pointer.va_rawsize = data_todo + VARHDRSZ;/* as if not
> short */
>
Hi,
> v26-0014-Design-to-extend-the-varattrib_4b-and-toast-poin.patch:
> Design proposal covering varattrib_4b, TOAST pointer layouts, and
> related macro updates.
> v26-0015-Implement-Zstd-compression-no-dictionary-support.patch: Plain
> ZSTD (non dict) support and few basic tests.
Sending v27 p
Hi Michael,
I'm reviewing your patches(toast_64bit_v2 branch) and have prepared
two incremental patches that add ZSTD compression support. While doing
this I made a small refactoring in heaptoast.c (around
ToastTupleContext) to make adding additional TOAST pointer formats
easier.
Added two new on
Hi Michael,
On Tue, Jul 15, 2025 at 9:44 PM Michael Paquier wrote:
>
> I have no idea yet about the fate of the other TOAST patches I have
> proposed for this commit fest, but let's move on with this part of the
> refactoring by splitting the TOAST regression tests for LZ4 and pglz,
> with the ne
On Wed, May 7, 2025 at 5:38 PM Michael Paquier wrote:
> Yes, I was wondering if this is not the most natural approach in terms
> of structure once if we plug an extra byte into the varlena header if
> all the bits of va_extinfo for the compression information are used.
> Having all the bits may no
Hi Michael, Thanks for the feedback.
On Wed, May 7, 2025 at 12:49 AM Michael Paquier wrote:
>
> I have been reading 0001 and I'm finding that the integration does not
> seem to fit much with the existing varatt_external, making the whole
> result slightly confusing. A simple thing: the last bit
Hi Robert,
On Mon, May 5, 2025 at 8:07 AM Robert Haas wrote:
> I don't understand why we need this. I don't see why we need any sort
> of generalized concept of metadata at all here. The zstd-dict
> compression method needs to store a four-byte OID, so let it do that.
> But we don't need to brand
-0001-varattrib_4b-design-proposal-to-make-it-extended.patch:
varattrib_4b extensibility – adds varatt_cmp_extended, metadata size
dispatch and useful macros; behaviour unchanged.
v20-0002-zstd-nodict-compression.patch: Plain ZSTD (non dict) support.
--
Nikhil Veldanda
From dc6172c69453546340
on this design. Thanks!
On Mon, Apr 28, 2025 at 7:50 AM Robert Haas wrote:
>
> On Fri, Apr 25, 2025 at 11:15 AM Nikhil Kumar Veldanda
> wrote:
> > a. 24 bits for length → per-datum compression algorithm metadata is
> > capped at 16 MB, which is far more than any realistic c
Hi Michael,
Thanks for the suggestions. I agree that we should first solve the
“last–free-bit” problem in varattrib_4b compression bits before
layering on any features. Below is the approach I’ve prototyped to
keep the header compact yet fully extensible, followed by a sketch of
the plain-ZSTD(no
Hi Michael,
Thanks for the feedback and the suggested patch sequence. I completely
agree—we must minimize storage overhead when dictionaries aren’t used,
while ensuring varattrib_4b remains extensible enough to handle future
compression metadata beyond dictionary ID (for other algorithms). I’ll
ex
te:
>
> On Tue, Apr 15, 2025 at 2:13 PM Nikhil Kumar Veldanda
> wrote:
> > Addressing Compressed Datum Leaks problem (via CTAS, INSERT INTO ... SELECT
> > ...)
> >
> > As compressed datums can be copied to other unrelated tables via CTAS,
> > INSERT I
Hi,
I reviewed the discussions, and while most agreements focused on
changes to the toast pointer, the design I propose requires no
modifications to it. I’ve carefully considered the design choices made
previously, and I recognize Zstd’s clear advantages in compression
efficiency and performance o
Hi Tom,
On Thu, Mar 6, 2025 at 11:33 AM Tom Lane wrote:
>
> Robert Haas writes:
> > On Thu, Mar 6, 2025 at 12:43 AM Nikhil Kumar Veldanda
> > wrote:
> >> Notably, this is the first compression algorithm for Postgres that can
> >> make use of a di
Hi
On Thu, Mar 6, 2025 at 5:35 AM Aleksander Alekseev
wrote:
>
> Hi Nikhil,
>
> Many thanks for working on this. I proposed a similar patch some time
> ago [1] but the overall feedback was somewhat mixed so I choose to
> focus on something else. Thanks for peeking this up.
>
> > test=# select bui
Hi Robert,
> I think that solving the problems around using a dictionary is going
> to be really hard. Can we see some evidence that the results will be
> worth it?
With the latest patch I've shared,
Using a Kaggle dataset of Nintendo-related tweets[1], we leveraged
PostgreSQL's acquire_sample_r
Hi,
> Overall idea is great.
>
> I just want to mention LZ4 also have API to use dictionary. Its dictionary
> will be as simple as "virtually prepended" text (in contrast to complex
> ZStd dictionary format).
>
> I mean, it would be great if "dictionary" will be common property for
> different alg
Hi Yura,
> So, to support "super-fast" mode you have to accept negative compression
> levels. I didn't check, probably you're already support them?
>
The key point I want to emphasize is that both zstd compression levels
and dictionary size should be configurable based on user preferences
at attr
20 matches
Mail list logo
