Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

I wrote: > ... BTW, a quick grep for other calls of PageIndexTupleDelete suggests > that SPGiST could make very good use of PageIndexTupleOverwrite, and > there may be use for it in GIN too. I've pushed this patch with some editorializing and some followup work. The above remains as possible

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

... BTW, a quick grep for other calls of PageIndexTupleDelete suggests that SPGiST could make very good use of PageIndexTupleOverwrite, and there may be use for it in GIN too. I see one place in btree where there's a PageIndexTupleDelete and then an insert, but it's unlikely to be worth touching

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

Here's a draft patch that replaces BRIN's use of PageIndexDeleteNoCompact and an immediately following PageAddItemExtended with PageIndexTupleOverwrite. It turns out that there are two use-cases in BRIN for PageIndexDeleteNoCompact, and the other one can't be replaced because while there is a

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

Andrey Borodin writes: > That storage assertion fired during usual update table set x=random() without > conditions. Also Make check fails without it (for brin usage, gist is ok with > it). I'm confused by that assertion, because the patch-as-submitted didn't touch BRIN.

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

That storage assertion fired during usual update table set x=random() without conditions. Also Make check fails without it (for brin usage, gist is ok with it). Cannot type quotation properly, I'm on a phone for a long time. Regards, Andrey Borodin. -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

Tom Lane wrote: > Alvaro Herrera writes: > > Tom Lane wrote: > >> That comment seems utterly wrong to me, because both PageIndexTupleDelete > >> and PageIndexMultiDelete certainly contain assertions that every item on > >> the page has storage. Are you expecting that

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

Alvaro Herrera writes: > Tom Lane wrote: >> That comment seems utterly wrong to me, because both PageIndexTupleDelete >> and PageIndexMultiDelete certainly contain assertions that every item on >> the page has storage. Are you expecting that any BRIN index items >>

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

Tom Lane wrote: > Hey Alvaro, can you comment on this bit in the proposed patch? > > + for (i = FirstOffsetNumber; i <= itemcount; i++) > + { > + ItemId ii = PageGetItemId(phdr, i); > + > + /* Normally here was following

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

Hey Alvaro, can you comment on this bit in the proposed patch? + for (i = FirstOffsetNumber; i <= itemcount; i++) + { + ItemId ii = PageGetItemId(phdr, i); + + /* Normally here was following assertion +

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

Tom Lane wrote: > Andrew Borodin writes: > > Thank you for your corrections. > > Here is the patch with suggestions taken into account, except 6th. > > I'll pick this up, unless some other committer is already on it. > > I think that we should buy back the addition of

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

Andrew Borodin writes: > Thank you for your corrections. > Here is the patch with suggestions taken into account, except 6th. I'll pick this up, unless some other committer is already on it. I think that we should buy back the addition of PageIndexTupleOverwrite to

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

Thank you for your corrections. Here is the patch with suggestions taken into account, except 6th. >6) I'd rather use alignednewsize here. > +ItemIdSetNormal(tupid, offset + size_diff, newsize); This behavior is accroding to ubiquitous PageAddItem. Best regards, Andrey Borodin, Octonica &

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

09.08.2016 19:45, Andrew Borodin: Here is new version of the patch, now it includes recommendations from Anastasia Lubennikova. I've investigated anamalous index size decrease. Most probable version appeared to be true. Cube extension, as some others, use Guttman's polynomial time split

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

Here is new version of the patch, now it includes recommendations from Anastasia Lubennikova. I've investigated anamalous index size decrease. Most probable version appeared to be true. Cube extension, as some others, use Guttman's polynomial time split algorithm. It is known to generate

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

06.08.2016 19:51, Andrew Borodin: Anastasia , thank you for your attentive code examine. 2016-08-05 21:19 GMT+05:00 Anastasia Lubennikova : First of all, shouldn't we use MAXALIGN(oldsize) instead of oldsize? Although, I'm quite sure that it was already aligned

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

Anastasia , thank you for your attentive code examine. 2016-08-05 21:19 GMT+05:00 Anastasia Lubennikova : > First of all, shouldn't we use MAXALIGN(oldsize) instead of oldsize? > Although, I'm quite sure that it was already aligned somewhere before. > I doubt that

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

30.07.2016 14:00, Andrew Borodin: Here is BRIN-compatible version of patch. Now function PageIndexTupleOverwrite consumes tuple size as a parameter, this behavior is coherent with PageAddItem. Also, i had to remove asserstion that item has a storage in the loop that adjusts line pointers. It

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

Here is BRIN-compatible version of patch. Now function PageIndexTupleOverwrite consumes tuple size as a parameter, this behavior is coherent with PageAddItem. Also, i had to remove asserstion that item has a storage in the loop that adjusts line pointers. It would be great if someone could check

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

>Can you please patch BRIN to use this new function too? On my machine replacement of both BRIN update cases (see https://github.com/x4m/pggistopt/commit/a6d301ff79104b977619339d53aebf748045418a ) showed no performance changes on folowing update query (6 seconds of updates avg): create table

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

>Can you please patch BRIN to use this new function too? AFAIK Sergey Mirvoda was going to implement and test it. It is easy to do this optimization for brin_samepage_update (see patch draft in attachment, make check passes), but WAL of regular BRIN update is not clear for me. Best regards,

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

Andrew Borodin wrote: > >If a feature changes the shape of WAL records, XLOG_PAGE_MAGIC is bumped to > >prevent any problems. > I've attached patch with a bump, but, obviously, it'll be irrelevant > after any other commit changing WAL shape. > > Here is a patch with updated GiST README. > I

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

Michael Paquier writes: > On Sun, Jul 24, 2016 at 7:18 PM, Andrew Borodin wrote: >> I've attached patch with a bump, but, obviously, it'll be irrelevant >> after any other commit changing WAL shape. > Usually the committer in charge of reviewing

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

On Sun, Jul 24, 2016 at 7:18 PM, Andrew Borodin wrote: >>If a feature changes the shape of WAL records, XLOG_PAGE_MAGIC is bumped to >>prevent any problems. > > I've attached patch with a bump, but, obviously, it'll be irrelevant > after any other commit changing WAL shape.

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

>If a feature changes the shape of WAL records, XLOG_PAGE_MAGIC is bumped to >prevent any problems. I've attached patch with a bump, but, obviously, it'll be irrelevant after any other commit changing WAL shape. Here is a patch with updated GiST README. I haven't found apropriate place to

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

On Fri, Jul 8, 2016 at 2:00 PM, Andrew Borodin wrote: (Please top-post on threads, this is annoying) > One more thing came to my mind: > WAL records done by the GiST before patch will corrupt GiST after > patch if replayed. > Is it a problem? May be it's prevented on some

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

One more thing came to my mind: WAL records done by the GiST before patch will corrupt GiST after patch if replayed. Is it a problem? May be it's prevented on some higher level? Best regards, Andrey Borodin, Octonica & Ural Federal University. 2016-07-06 22:11 GMT+05:00 Andrew Borodin

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

Here is a new patch. Updates: 1. Uses MAXALIGN to allocate space on page 2. Uses ItemIdSetNormal in case when ItemId was not normal for some reasons before call 3. Improved comments and var names Best regards, Andrey Borodin, Octonica & Ural Federal University. 2016-07-05 17:54 GMT+05:00 Andrew

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

Here is a patch with updated WAL behavior. I'm not certain about MAXALIGN for size of appended tuple. Currently if anyone calls PageIndexTupleOverwrite() with unalligned tuple it will ereport(ERROR). I suspect that it should allign size instead. Also I suspect that PageIndexTupleOverwrite()

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

Thank you, Alvaro. Yes, now I see. I'll update gistRedoPageUpdateRecord() to work accordingly with patched gistplacetopage(). Also, i've noticed that PageAddItem uses MAXALIGN() macro to calculate tuple size. So, PageIndexTupleOverwrite should behave the same. About PageIndexDeleteNoCompact() in

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

Andrew Borodin wrote: > Thank you, Amit. > > Currently, if WAL reconstruct page in another order it won't be a problem. We require that replay of WAL leads to identical bytes than the original. Among other things, this enables use of a tool that verifies that WAL is working correctly simply by

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

Thank you, Amit. Currently, if WAL reconstruct page in another order it won't be a problem. How can I check that it works? Would it be sufficient if I kill 9 backend and postmaster after commit and it will start normally executing select queries after restart? I'll make a patch with missing

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

On Mon, Jul 4, 2016 at 4:52 PM, Andrew Borodin wrote: > Here is a patch with corrections from Alexander Korotkov. > My tests, check and check-world show no problems against Ubuntu LTS 14 x64 VM. > > - PageIndexTupleDelete(page, oldoffnum); - gistfillbuffer(page, itup, ntup,