Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2024-03-23 Thread Alexander Korotkov
On Tue, Mar 19, 2024 at 5:20 PM Alexander Korotkov wrote: > On Tue, Nov 28, 2023 at 11:00 AM Pavel Borisov wrote: > > > You're designing new APIs, days before the feature freeze. > > On Wed, 5 Apr 2023 at 06:54, Michael Paquier wrote: > > > > > > On Tue, Apr 04, 2023 at 01:25:46AM +0300,

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2024-03-19 Thread Alexander Korotkov
Hi, Pavel! On Tue, Nov 28, 2023 at 11:00 AM Pavel Borisov wrote: > > You're designing new APIs, days before the feature freeze. > On Wed, 5 Apr 2023 at 06:54, Michael Paquier wrote: > > > > On Tue, Apr 04, 2023 at 01:25:46AM +0300, Alexander Korotkov wrote: > > > Pavel, thank you for you

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-11-28 Thread Pavel Borisov
Hi, hackers! > You're designing new APIs, days before the feature freeze. On Wed, 5 Apr 2023 at 06:54, Michael Paquier wrote: > > On Tue, Apr 04, 2023 at 01:25:46AM +0300, Alexander Korotkov wrote: > > Pavel, thank you for you review, revisions and rebase. > > We'll reconsider this once v17 is

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-04-04 Thread Michael Paquier
On Tue, Apr 04, 2023 at 01:25:46AM +0300, Alexander Korotkov wrote: > Pavel, thank you for you review, revisions and rebase. > We'll reconsider this once v17 is branched. The patch was still in the current CF, so I have moved it to the next one based on the latest updates. -- Michael

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-04-03 Thread Alexander Korotkov
On Mon, Apr 3, 2023 at 5:12 PM Pavel Borisov wrote: > Upon Alexander reverting patches v15 from master, I've rebased what > was correction patches v4 in a message above on a fresh master > (together with patches v15). The resulting patch v16 is attached. Pavel, thank you for you review,

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-04-03 Thread Pavel Borisov
Upon Alexander reverting patches v15 from master, I've rebased what was correction patches v4 in a message above on a fresh master (together with patches v15). The resulting patch v16 is attached. Pavel. v16-0002-Add-EvalPlanQual-delete-returning-isolation-test.patch Description: Binary data

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-04-03 Thread Pavel Borisov
Hi, Alexander! On 2023-04-02 03:37:19 +0300, Alexander Korotkov wrote: > On Sat, Apr 1, 2023 at 8:21 AM Andres Freund wrote: > > Given that the in-tree state has been broken for a week, I think it probably > > is time to revert the commits that already went in. > > The revised patch is attached.

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-04-03 Thread Alexander Korotkov
On Sun, Apr 2, 2023 at 3:47 AM Andres Freund wrote: > On 2023-04-02 03:37:19 +0300, Alexander Korotkov wrote: > > On Sat, Apr 1, 2023 at 8:21 AM Andres Freund wrote: > > > Given that the in-tree state has been broken for a week, I think it > > > probably > > > is time to revert the commits that

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-04-01 Thread Andres Freund
Hi, On 2023-04-02 03:37:19 +0300, Alexander Korotkov wrote: > On Sat, Apr 1, 2023 at 8:21 AM Andres Freund wrote: > > Given that the in-tree state has been broken for a week, I think it probably > > is time to revert the commits that already went in. > > The revised patch is attached. The most

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-04-01 Thread Alexander Korotkov
.Hi! On Sat, Apr 1, 2023 at 8:21 AM Andres Freund wrote: > On 2023-03-31 16:57:41 +0300, Alexander Korotkov wrote: > > On Wed, Mar 29, 2023 at 8:34 PM Alexander Korotkov > > wrote: > > > On Mon, Mar 27, 2023 at 1:49 PM Alexander Korotkov > > > wrote: > > > > On Fri, Mar 24, 2023 at 3:39 AM

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-04-01 Thread Pavel Borisov
Hi, Andres! On Sat, 1 Apr 2023, 09:21 Andres Freund, wrote: > Given that the in-tree state has been broken for a week, I think it > probably > is time to revert the commits that already went in. > It seems that although the patch addressing the issues is not a quick fix, there is a big

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-03-31 Thread Andres Freund
Hi, On 2023-03-31 16:57:41 +0300, Alexander Korotkov wrote: > On Wed, Mar 29, 2023 at 8:34 PM Alexander Korotkov > wrote: > > On Mon, Mar 27, 2023 at 1:49 PM Alexander Korotkov > > wrote: > > > On Fri, Mar 24, 2023 at 3:39 AM Andres Freund wrote: > > > > On 2023-03-23 23:24:19 +0300,

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-03-31 Thread Alexander Korotkov
On Wed, Mar 29, 2023 at 8:34 PM Alexander Korotkov wrote: > On Mon, Mar 27, 2023 at 1:49 PM Alexander Korotkov > wrote: > > On Fri, Mar 24, 2023 at 3:39 AM Andres Freund wrote: > > > On 2023-03-23 23:24:19 +0300, Alexander Korotkov wrote: > > > > On Thu, Mar 23, 2023 at 8:06 PM Andres Freund

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-03-29 Thread Alexander Korotkov
Andres, On Mon, Mar 27, 2023 at 1:49 PM Alexander Korotkov wrote: > On Fri, Mar 24, 2023 at 3:39 AM Andres Freund wrote: > > On 2023-03-23 23:24:19 +0300, Alexander Korotkov wrote: > > > On Thu, Mar 23, 2023 at 8:06 PM Andres Freund wrote: > > > > I seriously doubt that solving this at the

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-03-27 Thread Alexander Korotkov
Hi! On Fri, Mar 24, 2023 at 3:39 AM Andres Freund wrote: > On 2023-03-23 23:24:19 +0300, Alexander Korotkov wrote: > > On Thu, Mar 23, 2023 at 8:06 PM Andres Freund wrote: > > > I seriously doubt that solving this at the tuple locking level is the > > > right > > > thing. If we want to avoid

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-03-23 Thread Andres Freund
Hi, An off-list conversation veered on-topic again. Reposting for posterity: On 2023-03-23 23:24:19 +0300, Alexander Korotkov wrote: > On Thu, Mar 23, 2023 at 8:06 PM Andres Freund wrote: > > I seriously doubt that solving this at the tuple locking level is the right > > thing. If we want to

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-03-23 Thread Andres Freund
Hi, On 2023-03-23 18:08:36 +0300, Alexander Korotkov wrote: > On Thu, Mar 23, 2023 at 3:30 AM Andres Freund wrote: > > On 2023-03-21 01:25:11 +0300, Alexander Korotkov wrote: > > > I'm going to push patchset v15 if no objections. > > > > Just saw that this went in - didn't catch up with the

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-03-23 Thread Alexander Korotkov
Hi! On Thu, Mar 23, 2023 at 3:30 AM Andres Freund wrote: > On 2023-03-21 01:25:11 +0300, Alexander Korotkov wrote: > > I'm going to push patchset v15 if no objections. > > Just saw that this went in - didn't catch up with the thread before, > unfortunately. At the very least I'd like to see some

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-03-22 Thread Andres Freund
Hi, On 2023-03-21 01:25:11 +0300, Alexander Korotkov wrote: > I'm going to push patchset v15 if no objections. Just saw that this went in - didn't catch up with the thread before, unfortunately. At the very least I'd like to see some more work on cleaning up the lazy tuple slot stuff. It's

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-03-12 Thread Alexander Korotkov
On Fri, Mar 10, 2023 at 8:17 PM Chris Travers wrote: > "Right, the improvement this patch gives to the heap is not the full > motivation. Another motivation is the improvement it gives to TableAM API. > Our current API implies that the effort on locating the tuple by tid is > small. This is

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-03-10 Thread Chris Travers
"Right, the improvement this patch gives to the heap is not the full motivation. Another motivation is the improvement it gives to TableAM API. Our current API implies that the effort on locating the tuple by tid is small. This is more or less true for the heap, where we just need to pin and

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-03-10 Thread Alexander Korotkov
On Wed, Mar 8, 2023 at 4:22 AM Andres Freund wrote: > On 2023-03-07 04:45:32 +0300, Alexander Korotkov wrote: > > The second patch now implements a concept of LazyTupleTableSlot, a slot > > which gets allocated only when needed. Also, there is more minor > > refactoring and more comments. > >

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-03-07 Thread Andres Freund
Hi, On 2023-03-07 04:45:32 +0300, Alexander Korotkov wrote: > The second patch now implements a concept of LazyTupleTableSlot, a slot > which gets allocated only when needed. Also, there is more minor > refactoring and more comments. This patch already is pretty big for what it actually

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-03-07 Thread Alexander Korotkov
On Tue, Mar 7, 2023 at 7:26 PM Pavel Borisov wrote: > On Tue, 7 Mar 2023, 10:10 Alexander Korotkov, wrote: >> I don't know what exactly Pavel meant, but average overall numbers for >> low concurrency are. >> master: 420401 (stddev of average 233) >> patchset v11: 420111 (stddev of average 199)

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-03-07 Thread Pavel Borisov
Hi, Andres and Alexander! On Tue, 7 Mar 2023, 10:10 Alexander Korotkov, wrote: > On Tue, Mar 7, 2023 at 4:50 AM Andres Freund wrote: > > On 2023-03-02 14:28:56 +0400, Pavel Borisov wrote: > > > 2. Heap updates with low tuple concurrency: > > > Prepare with pkeys (pgbench -d postgres -i -I

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-03-06 Thread Alexander Korotkov
On Tue, Mar 7, 2023 at 4:50 AM Andres Freund wrote: > On 2023-03-02 14:28:56 +0400, Pavel Borisov wrote: > > 2. Heap updates with low tuple concurrency: > > Prepare with pkeys (pgbench -d postgres -i -I dtGvp -s 300 > > --unlogged-tables) > > Update 3*10^7 rows, 50 conns (pgbench postgres -f > >

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-03-06 Thread Andres Freund
Hi, On 2023-03-02 14:28:56 +0400, Pavel Borisov wrote: > 2. Heap updates with low tuple concurrency: > Prepare with pkeys (pgbench -d postgres -i -I dtGvp -s 300 --unlogged-tables) > Update 3*10^7 rows, 50 conns (pgbench postgres -f > ./update-only-account.sql -s 300 -P10 -M prepared -T 600 -j 5

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-03-06 Thread Alexander Korotkov
On Thu, Mar 2, 2023 at 11:16 PM Alexander Korotkov wrote: > On Thu, Mar 2, 2023 at 9:17 PM Pavel Borisov wrote: > > On Thu, 2 Mar 2023 at 18:53, Alexander Korotkov > > wrote: > > > On Thu, Mar 2, 2023 at 1:29 PM Pavel Borisov > > > wrote: > > > > > Let's see the performance results for the

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-03-02 Thread Alexander Korotkov
On Thu, Mar 2, 2023 at 9:17 PM Pavel Borisov wrote: > On Thu, 2 Mar 2023 at 18:53, Alexander Korotkov wrote: > > On Thu, Mar 2, 2023 at 1:29 PM Pavel Borisov wrote: > > > > Let's see the performance results for the patchset. I'll properly > > > > revise the comments if results will be good. >

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-03-02 Thread Alexander Korotkov
Hi, Pavel! On Thu, Mar 2, 2023 at 1:29 PM Pavel Borisov wrote: > > Let's see the performance results for the patchset. I'll properly > > revise the comments if results will be good. > > > > Pavel, could you please re-run your tests over revised patchset? > > Since last time I've improved the

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-03-01 Thread Gregory Stark
It looks like this patch received some feedback from Andres and hasn't had any further work posted. I'm going to move it to "Waiting on Author". It doesn't sound like this is likely to get committed this release cycle unless responding to Andres's points simpler than I expect.

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-03-01 Thread Alexander Korotkov
Hi, Andres. Thank you for your review. Sorry for the late reply. I took some time for me to figure out how to revise the patch. The revised patchset is attached. I decided to split the patch into two: 1) Avoid re-fetching the "original" row version during update and delete. 2) Save the

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-02-28 Thread Alexander Korotkov
On Wed, Mar 1, 2023 at 12:03 AM Gregory Stark wrote: > It looks like this patch received some feedback from Andres and hasn't > had any further work posted. I'm going to move it to "Waiting on > Author". I'll post the updated version in the next couple of days. > It doesn't sound like this is

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-01-09 Thread Andres Freund
Hi, I'm a bit worried that this is optimizing the rare case while hurting the common case. See e.g. my point below about creating additional slots in the happy path. It's also not clear that change is right directionally. If we want to avoid re-fetching the "original" row version, why don't we

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-01-09 Thread Andres Freund
Hi, On 2023-01-09 13:46:50 +0300, Alexander Korotkov wrote: > I'm going to push v9. Could you hold off for a bit? I'd like to look at this, I'm not sure I like the direction. Greetings, Andres Freund

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-01-09 Thread Alexander Korotkov
Hi, Pavel! On Mon, Jan 9, 2023 at 1:41 PM Pavel Borisov wrote: > On Mon, 9 Jan 2023 at 13:29, Alexander Korotkov wrote: > > On Mon, Jan 9, 2023 at 1:10 PM Alexander Korotkov > > wrote: > > > On Mon, Jan 9, 2023 at 12:56 PM Aleksander Alekseev > > > wrote: > > > > > I'm going to push this if

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-01-09 Thread Aleksander Alekseev
Alexander, Pavel, > Sorry for being so detailed in small details. In my opinion the patch > now is ready to be committed. Agree. Personally I liked the version with (lockUpdated, lockedSlot) pair a bit more since it is a bit more readable, however the version without lockUpdated is less error

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-01-09 Thread Pavel Borisov
Hi, Alexander! On Mon, 9 Jan 2023 at 13:29, Alexander Korotkov wrote: > > On Mon, Jan 9, 2023 at 1:10 PM Alexander Korotkov > wrote: > > On Mon, Jan 9, 2023 at 12:56 PM Aleksander Alekseev > > wrote: > > > > I'm going to push this if no objections. > > > > > > I took a fresh look at the patch

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-01-09 Thread Alexander Korotkov
On Mon, Jan 9, 2023 at 1:10 PM Alexander Korotkov wrote: > On Mon, Jan 9, 2023 at 12:56 PM Aleksander Alekseev > wrote: > > > I'm going to push this if no objections. > > > > I took a fresh look at the patch and it LGTM. I only did a few > > cosmetic changes, PFA v7. > > > > Changes since v6

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-01-09 Thread Alexander Korotkov
Hi Aleksander, On Mon, Jan 9, 2023 at 12:56 PM Aleksander Alekseev wrote: > > I'm going to push this if no objections. > > I took a fresh look at the patch and it LGTM. I only did a few > cosmetic changes, PFA v7. > > Changes since v6 are: Thank you for looking into this. It appears that I've

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-01-09 Thread Aleksander Alekseev
Hi Alexander, > I'm going to push this if no objections. I took a fresh look at the patch and it LGTM. I only did a few cosmetic changes, PFA v7. Changes since v6 are: ``` @@ -318,12 +318,12 @@ heapam_tuple_delete(Relation relation, ItemPointer tid, CommandId cid, result =

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-01-08 Thread Alexander Korotkov
Hi, Mason! Thank you very much for your review. On Sun, Jan 8, 2023 at 9:33 PM Mason Sharp wrote: > On Fri, Jan 6, 2023 at 4:46 AM Pavel Borisov wrote: >> Besides, the new version has only some minor changes in the comments >> and the commit message. > It looks good, and the greater the

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-01-08 Thread Mason Sharp
On Fri, Jan 6, 2023 at 4:46 AM Pavel Borisov wrote: > Hi, Alexander! > > On Thu, 5 Jan 2023 at 15:11, Alexander Korotkov > wrote: > > > > On Wed, Jan 4, 2023 at 5:05 PM Alexander Korotkov > wrote: > > > On Wed, Jan 4, 2023 at 3:43 PM Pavel Borisov > wrote: > > > > One more update of a

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-01-06 Thread Pavel Borisov
Hi, Alexander! On Thu, 5 Jan 2023 at 15:11, Alexander Korotkov wrote: > > On Wed, Jan 4, 2023 at 5:05 PM Alexander Korotkov > wrote: > > On Wed, Jan 4, 2023 at 3:43 PM Pavel Borisov wrote: > > > One more update of a patchset to avoid compiler warnings. > > > > Thank you for your help. I'm

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-01-05 Thread Alexander Korotkov
On Wed, Jan 4, 2023 at 5:05 PM Alexander Korotkov wrote: > On Wed, Jan 4, 2023 at 3:43 PM Pavel Borisov wrote: > > One more update of a patchset to avoid compiler warnings. > > Thank you for your help. I'm going to provide the revised version of > patch with comments and commit message in the

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-01-04 Thread Alexander Korotkov
Hi, Pavel! On Wed, Jan 4, 2023 at 3:43 PM Pavel Borisov wrote: > On Wed, 4 Jan 2023 at 12:52, Pavel Borisov wrote: > > On Wed, 4 Jan 2023 at 12:41, vignesh C wrote: > > > > > > On Fri, 1 Jul 2022 at 16:49, Alexander Korotkov > > > wrote: > > > > > > > > Hackers, > > > > > > > > When working

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-01-04 Thread Pavel Borisov
On Wed, 4 Jan 2023 at 12:52, Pavel Borisov wrote: > > Hi, Vignesh! > > On Wed, 4 Jan 2023 at 12:41, vignesh C wrote: > > > > On Fri, 1 Jul 2022 at 16:49, Alexander Korotkov > > wrote: > > > > > > Hackers, > > > > > > When working in the read committed transaction isolation mode > > >

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-01-04 Thread Pavel Borisov
Hi, Vignesh! On Wed, 4 Jan 2023 at 12:41, vignesh C wrote: > > On Fri, 1 Jul 2022 at 16:49, Alexander Korotkov wrote: > > > > Hackers, > > > > When working in the read committed transaction isolation mode > > (default), we have the following sequence of actions when > > tuple_update() or

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-01-04 Thread vignesh C
On Fri, 1 Jul 2022 at 16:49, Alexander Korotkov wrote: > > Hackers, > > When working in the read committed transaction isolation mode > (default), we have the following sequence of actions when > tuple_update() or tuple_delete() find concurrently updated tuple. > > 1.

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2022-07-29 Thread Alexander Korotkov
Hi, Pavel! On Fri, Jul 29, 2022 at 11:12 AM Pavel Borisov wrote: > I ran the following benchmark on master branch (15) vs patch (15-lock): > > On the 36-vcore AWS server, I've run an UPDATE-only pgbench script with 50 > connections on pgbench_tellers with 100 rows. The idea was to introduce as

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2022-07-12 Thread Alexander Korotkov
Hi Aleksander! Thank you for your efforts reviewing this patch. On Thu, Jul 7, 2022 at 12:43 PM Aleksander Alekseev wrote: > > I'm going to need more time to meditate on the proposed changes and to > > figure out the performance impact. > > OK, turned out this patch is slightly more

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2022-07-07 Thread Aleksander Alekseev
Hi Alexander, > I'm going to need more time to meditate on the proposed changes and to figure > out the performance impact. OK, turned out this patch is slightly more complicated than I initially thought, but I think I managed to get some vague understanding of what's going on. I tried to

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2022-07-05 Thread Aleksander Alekseev
Hi again, > +if (!updated) > +{ > +/* Should not encounter speculative tuple on recheck */ > +Assert(!HeapTupleHeaderIsSpeculative(tuple->t_data)); > - ReleaseBuffer(buffer); > +ReleaseBuffer(buffer); > +} > +else > +

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2022-07-05 Thread Aleksander Alekseev
Hi Alexander, > Thoughts and feedback are welcome. I took some preliminary look at the patch. I'm going to need more time to meditate on the proposed changes and to figure out the performance impact. So far I just wanted to let you know that the patch applied OK for me and passed all the tests.

POC: Lock updated tuples in tuple_update() and tuple_delete()

2022-07-01 Thread Alexander Korotkov
Hackers, When working in the read committed transaction isolation mode (default), we have the following sequence of actions when tuple_update() or tuple_delete() find concurrently updated tuple. 1. tuple_update()/tuple_delete() returns TM_Updated 2. tuple_lock() 3. Re-evaluate plan qual (recheck