On Sun, Nov 17, 2019 at 9:18 PM Alexander Korotkov
wrote:
> So, I've put this explanation into README patch. I just change
> designation to better match with Lehman & Yao paper and did some minor
> enchantments.
>
> I'm going to push this patchset if no objections.
So, pushed with minor changes
On Mon, Nov 11, 2019 at 2:42 AM Alexander Korotkov
wrote:
> I'm sorry for late reply. I was busy with various things. Also
> digging into these details took some time. Please find my explanation
> below.
>
> On Wed, Oct 30, 2019 at 2:34 AM Peter Geoghegan wrote:
> > In general, it seems very
Hi!
I'm sorry for late reply. I was busy with various things. Also
digging into these details took some time. Please find my explanation
below.
On Wed, Oct 30, 2019 at 2:34 AM Peter Geoghegan wrote:
> In general, it seems very important to be clear about exactly how the
> key space works.
On Sun, Oct 27, 2019 at 12:04 PM Alexander Korotkov
wrote:
> (0001-Fix-deadlock-between-ginDeletePage-and-ginStepRigh-3.patch)
Some thoughts on the first patch:
* How do comparisons of items in each type of B-Tree work?
I would like to see a statement about invariants similar to nbtree's
On Mon, Oct 28, 2019 at 2:00 PM Peter Geoghegan wrote:
> On Sun, Oct 27, 2019 at 7:04 PM Alexander Korotkov
> wrote:
> > It tool me a lot of efforts to revise a concurrency section in README.
> > I can't judge the result, but I probably did my best. I'd like to
> > commit this patchset
Hi Alexander,
On Sun, Oct 27, 2019 at 7:04 PM Alexander Korotkov
wrote:
> It tool me a lot of efforts to revise a concurrency section in README.
> I can't judge the result, but I probably did my best. I'd like to
> commit this patchset including both bug fixes and README update. But
> I'd like
Hi, Peter!
On Fri, Oct 4, 2019 at 12:05 AM Alexander Korotkov
wrote:
> I'm planning to continue work on README, comments and commit messages.
It tool me a lot of efforts to revise a concurrency section in README.
I can't judge the result, but I probably did my best. I'd like to
commit this
On Tue, Oct 1, 2019 at 5:55 AM Alexander Korotkov
wrote:
> On Mon, Sep 30, 2019 at 10:54 PM Peter Geoghegan wrote:
> >
> > On Sun, Sep 29, 2019 at 8:12 AM Alexander Korotkov
> > wrote:
> > > I just managed to reproduce this using two sessions on master branch.
> > >
> > > session 1
> > >
On Mon, Sep 30, 2019 at 10:54 PM Peter Geoghegan wrote:
>
> On Sun, Sep 29, 2019 at 8:12 AM Alexander Korotkov
> wrote:
> > I just managed to reproduce this using two sessions on master branch.
> >
> > session 1
> > session 2
>
> Was the involvement of the pending list stuff in Chen's
On Sun, Sep 29, 2019 at 8:12 AM Alexander Korotkov
wrote:
> I just managed to reproduce this using two sessions on master branch.
>
> session 1
> session 2
Was the involvement of the pending list stuff in Chen's example just a
coincidence? Can you recreate the problem while eliminating that
On Mon, Sep 30, 2019 at 11:00 AM Alexander Korotkov
wrote:
> Thank you for pointing. I remember this thread, but don't remember
> details. I'll reread it.
I think that doing this now would be a good idea:
"""
Defensively checking the page type (pending, posting, etc) within
On Sun, Sep 29, 2019 at 10:38 PM Andrey Borodin wrote:
> As far as I understand deleted page is stamped with
> GinPageSetDeleteXid(page, ReadNewTransactionId());
> It will not be recycled until that Xid is far behind.
That only gets used within posting tree pages, though.
ginInsertCleanup() is
On Sun, Sep 29, 2019 at 10:53 PM Peter Geoghegan wrote:
> On Sun, Sep 29, 2019 at 7:38 AM Alexander Korotkov
> wrote:
> > Starting from root seems OK for me, because vacuum blocks all
> > concurrent inserts before doing this. But this needs to be properly
> > documented in readme.
>
> I never
On Mon, Sep 30, 2019 at 8:38 AM Andrey Borodin wrote:
>
> > 29 сент. 2019 г., в 21:27, Alexander Korotkov
> > написал(а):
> >
> > Patch with fix is attached. Idea is simple: ginScanToDelete() now
> > keeps exclusive lock on left page eliminating the need to relock it.
> > So, we preserve
> 29 сент. 2019 г., в 21:27, Alexander Korotkov
> написал(а):
>
> Patch with fix is attached. Idea is simple: ginScanToDelete() now
> keeps exclusive lock on left page eliminating the need to relock it.
> So, we preserve left-to-right locking order and can't deadlock with
> ginStepRight().
On Sun, Sep 29, 2019 at 7:38 AM Alexander Korotkov
wrote:
> Starting from root seems OK for me, because vacuum blocks all
> concurrent inserts before doing this. But this needs to be properly
> documented in readme.
I never got an adequate answer to this closely related question almost
two
On Sun, Sep 29, 2019 at 6:12 PM Alexander Korotkov
wrote:
> On Sun, Sep 29, 2019 at 5:38 PM Alexander Korotkov
> wrote:
> > On Sun, Sep 29, 2019 at 11:17 AM chenhj wrote:
> > > Does the locking order of autovacuum process(root->right->left) correct?
> > > While insert process lock gin buffer
On Sun, Sep 29, 2019 at 5:38 PM Alexander Korotkov
wrote:
> On Sun, Sep 29, 2019 at 11:17 AM chenhj wrote:
> > Does the locking order of autovacuum process(root->right->left) correct?
> > While insert process lock gin buffer by order of bottom->top and
> > left->right.
> >
> > 1.
Hi!
Thank you for reporting.
On Sun, Sep 29, 2019 at 11:17 AM chenhj wrote:
> Does the locking order of autovacuum process(root->right->left) correct?
> While insert process lock gin buffer by order of bottom->top and left->right.
>
> 1. vacuum(root->right->left):
Starting from root seems OK
Hi,all
In our PostgreSQL 10.7(rhel 6.3) database, autovacuum process and many insert
processes blocked in gin index's LWLock:buffer_content for long time.
In other words, the following gin index lwlock deadlock phenomenon has occurred
again. Since the following bug in 10.7 has been fixed.
On Sat, Mar 23, 2019 at 11:43 AM Alexander Korotkov
wrote:
> On Fri, Mar 22, 2019 at 11:05 AM Alexander Korotkov
> wrote:
> > On Fri, Mar 22, 2019 at 12:06 AM Alvaro Herrera
> > wrote:
> > > On 2019-Mar-21, Alexander Korotkov wrote:
> > >
> > > > However, I think this still can be backpatched
On Fri, Mar 22, 2019 at 11:05 AM Alexander Korotkov
wrote:
> On Fri, Mar 22, 2019 at 12:06 AM Alvaro Herrera
> wrote:
> > On 2019-Mar-21, Alexander Korotkov wrote:
> >
> > > However, I think this still can be backpatched correctly. We can
> > > determine whether xlog record data contains
On Fri, Mar 22, 2019 at 12:06 AM Alvaro Herrera
wrote:
> On 2019-Mar-21, Alexander Korotkov wrote:
>
> > However, I think this still can be backpatched correctly. We can
> > determine whether xlog record data contains deleteXid by its size.
> > See the attached patch. I haven't test this yet.
On 2019-Mar-21, Alexander Korotkov wrote:
> However, I think this still can be backpatched correctly. We can
> determine whether xlog record data contains deleteXid by its size.
> See the attached patch. I haven't test this yet. I'm going to test
> it. If OK, then push.
Wow, this is so
On Thu, 21 Mar 2019 at 15:18, Alexander Korotkov
wrote:
> On Thu, Mar 21, 2019 at 9:26 PM Simon Riggs wrote:
>
>
> > It's been pointed out to me that 52ac6cd2d0cd70e01291e0ac4ee6d068b69bc478
> > introduced a WAL incompatibility that has not been flagged.
> >
> > In ginRedoDeletePage() we use
On Thu, Mar 21, 2019 at 9:26 PM Simon Riggs wrote:
> On Thu, 13 Dec 2018 at 14:48, Alexander Korotkov wrote:
>> On Thu, Dec 13, 2018 at 10:46 PM Andres Freund wrote:
>> > On 2018-12-13 22:40:59 +0300, Alexander Korotkov wrote:
>> > > It doesn't mater, because we release all locks on every
On Thu, 13 Dec 2018 at 14:48, Alexander Korotkov
wrote:
> On Thu, Dec 13, 2018 at 10:46 PM Andres Freund wrote:
> > On 2018-12-13 22:40:59 +0300, Alexander Korotkov wrote:
> > > It doesn't mater, because we release all locks on every buffer at one
> > > time. The unlock order can have effect
On Thu, Dec 13, 2018 at 10:40:59PM +0300, Alexander Korotkov wrote:
> On Thu, Dec 13, 2018 at 8:06 PM Andrey Borodin wrote:
> > That's the same variable, one place is definition while other is potential
> > misuse.
> > Seems like these 2 lines [0]
> >
> > + if (BufferIsValid(lbuffer))
> >
On Thu, Dec 13, 2018 at 10:46 PM Andres Freund wrote:
> On 2018-12-13 22:40:59 +0300, Alexander Korotkov wrote:
> > It doesn't mater, because we release all locks on every buffer at one
> > time. The unlock order can have effect on what waiter will acquire
> > the lock next after
Hi,
On 2018-12-13 22:40:59 +0300, Alexander Korotkov wrote:
> It doesn't mater, because we release all locks on every buffer at one
> time. The unlock order can have effect on what waiter will acquire
> the lock next after ginRedoDeletePage(). However, I don't see why one
> unlock order is
On Thu, Dec 13, 2018 at 8:06 PM Andrey Borodin wrote:
> > if (BufferIsValid(lbuffer))
> > UnlockReleaseBuffer(lbuffer);
> > if (BufferIsValid(pbuffer))
> > UnlockReleaseBuffer(pbuffer);
> > if (BufferIsValid(dbuffer))
> > UnlockReleaseBuffer(dbuffer);
> > ==>
> > if (BufferIsValid(pbuffer))
> >
Hi!
> 13 дек. 2018 г., в 17:03, Alexander Korotkov
> написал(а):
>
> Thank you. I've revised your patch and pushed it. As long as two
> other patches in this thread.
That's great! Thanks!
> 13 дек. 2018 г., в 20:12, chjis...@163.com написал(а):
>
>
> hi
> I Have a question. Why the
Bruce Momjian writes:
> I am seeing this warning in the 9.4 branch:
> ginxlog.c:756:5: warning: ‘lbuffer’ may be used uninitialized
> in this function [-Wmaybe-uninitialized]
I'm also getting that, just in 9.4, but at a different line number:
ginxlog.c: In function
On Thu, Dec 13, 2018 at 03:03:54PM +0300, Alexander Korotkov wrote:
> On Wed, Dec 12, 2018 at 4:08 PM Andrey Borodin wrote:
> > > 12 дек. 2018 г., в 3:26, Alexander Korotkov
> > > написал(а):
> > >
> > > BTW, I still can't see you're setting deleteXid field of
> > > ginxlogDeletePage struct
On Wed, Dec 12, 2018 at 4:08 PM Andrey Borodin wrote:
> > 12 дек. 2018 г., в 3:26, Alexander Korotkov
> > написал(а):
> >
> > BTW, I still can't see you're setting deleteXid field of
> > ginxlogDeletePage struct anywhere.
> Oops. Fixed.
> >
> > Also, I note that protection against re-usage of
Hi!
> 12 дек. 2018 г., в 3:26, Alexander Korotkov написал(а):
>
> BTW, I still can't see you're setting deleteXid field of
> ginxlogDeletePage struct anywhere.
Oops. Fixed.
>
> Also, I note that protection against re-usage of recently deleted
> pages is implemented differently than it is in
On Tue, Dec 11, 2018 at 1:50 AM Alexander Korotkov wrote:
> On Sun, Dec 9, 2018 at 10:25 PM Alexander Korotkov
> wrote:
> > On Sat, Dec 8, 2018 at 12:48 PM Andrey Borodin wrote:
> > > > 8 дек. 2018 г., в 6:54, Alexander Korotkov
> > > > написал(а):
> > > >
> > > > Yep, please find attached
On Tue, Dec 11, 2018 at 10:14 AM Andrey Borodin wrote:
> > 11 дек. 2018 г., в 3:43, Alexander Korotkov
> > написал(а):
> >
> > Attached patch appears to be incomplete. GinPageSetDeleteXid() is
> > called only in ginRedoDeletePage(), so only in recovery, while it
> > should be set during normal
Hi!
> 11 дек. 2018 г., в 3:43, Alexander Korotkov написал(а):
>
> Attached patch appears to be incomplete. GinPageSetDeleteXid() is
> called only in ginRedoDeletePage(), so only in recovery, while it
> should be set during normal work too. deleteXid field of
> ginxlogDeletePage is never set.
On Sun, Dec 9, 2018 at 10:25 PM Alexander Korotkov wrote:
> On Sat, Dec 8, 2018 at 12:48 PM Andrey Borodin wrote:
> > > 8 дек. 2018 г., в 6:54, Alexander Korotkov
> > > написал(а):
> > >
> > > Yep, please find attached draft patch.
> >
> > Patch seems good to me, I'll check it in more detail.
On Mon, Dec 10, 2018 at 4:58 PM Andrey Borodin wrote:
> > 10 дек. 2018 г., в 18:56, Andrey Borodin написал(а):
> >
> > PFA this part. In thread about GiST vacuum Heikki was not very happy with
> > reusing PageHeader->pd_prune_xid for this. But we somewhat concluded that
> > this might be OK.
>
> 10 дек. 2018 г., в 18:56, Andrey Borodin написал(а):
>
> PFA this part. In thread about GiST vacuum Heikki was not very happy with
> reusing PageHeader->pd_prune_xid for this. But we somewhat concluded that
> this might be OK.
> Anyway, there's whole page, we can store those bits anywhere.
Hi!
> 10 дек. 2018 г., в 0:25, Alexander Korotkov написал(а):
>>
>> There's a patch above in this thread
>> 0001-Use-correct-locking-protocol-during-GIN-posting-tree.patch where I
>> propose stamping every deleted page with GinPageSetDeleteXid(page,
>> ReadNewTransactionId()); and avoid
On Sat, Dec 8, 2018 at 12:48 PM Andrey Borodin wrote:
> > 8 дек. 2018 г., в 6:54, Alexander Korotkov
> > написал(а):
> >
> > Yep, please find attached draft patch.
>
> Patch seems good to me, I'll check it in more detail.
> The patch gets posting item at FirstOffsetNumber instead of
>
Hi!
Thanks, Alexander!
> 8 дек. 2018 г., в 6:54, Alexander Korotkov написал(а):
>
> Yep, please find attached draft patch.
Patch seems good to me, I'll check it in more detail.
The patch gets posting item at FirstOffsetNumber instead of
btree->getLeftMostChild(). This seem OK, since
On Fri, Dec 7, 2018 at 12:50 AM Peter Geoghegan wrote:
> On Thu, Dec 6, 2018 at 12:51 PM Alexander Korotkov
> wrote:
> > So, algorithm introduced by 218f51584d5 is broken. It tries to
> > guarantee that there are no inserters in the subtree by placing
> > cleanup lock to subtree root, assuming
On Fri, Dec 7, 2018 at 12:14 PM Andrey Borodin wrote:
> > 7 дек. 2018 г., в 2:50, Peter Geoghegan написал(а):
> > On Thu, Dec 6, 2018 at 12:51 PM Alexander Korotkov
> > wrote:
> >> However, I'd like to note that 218f51584d5 introduces two changes:
> >> 1) Cleanup locking only if there pages to
Hi!
> 7 дек. 2018 г., в 2:50, Peter Geoghegan написал(а):
>
> On Thu, Dec 6, 2018 at 12:51 PM Alexander Korotkov
> wrote:
>>
>> However, I'd like to note that 218f51584d5 introduces two changes:
>> 1) Cleanup locking only if there pages to delete
>> 2) Cleanup locking only subtree root
>>
On Thu, Dec 6, 2018 at 12:51 PM Alexander Korotkov wrote:
> So, algorithm introduced by 218f51584d5 is broken. It tries to
> guarantee that there are no inserters in the subtree by placing
> cleanup lock to subtree root, assuming that inserter holds pins on the
> path from root to leaf. But due
On Thu, Dec 6, 2018 at 10:56 PM Alexander Korotkov wrote:
> On Tue, Dec 4, 2018 at 8:01 PM Andres Freund wrote:
> > On 2018-11-10 17:42:16 -0800, Peter Geoghegan wrote:
> > > On Wed, Nov 7, 2018 at 5:46 PM Peter Geoghegan wrote:
> > > > Teodor: Do you think that the issue is fixable? It looks
On Tue, Dec 4, 2018 at 8:01 PM Andres Freund wrote:
> On 2018-11-10 17:42:16 -0800, Peter Geoghegan wrote:
> > On Wed, Nov 7, 2018 at 5:46 PM Peter Geoghegan wrote:
> > > Teodor: Do you think that the issue is fixable? It looks like there
> > > are serious issues with the design of 218f51584d5
Hi,
On 2018-11-10 17:42:16 -0800, Peter Geoghegan wrote:
> On Wed, Nov 7, 2018 at 5:46 PM Peter Geoghegan wrote:
> > Teodor: Do you think that the issue is fixable? It looks like there
> > are serious issues with the design of 218f51584d5 to me. I don't think
> > the general "there can't be any
On Fri, Nov 23, 2018 at 2:18 AM Andrey Borodin wrote:
> I found no practical way to fix concept of "subtree-lock". Pre-locking all
> left siblings for cleanup would suffice, but does not look very practical.
> GIN Posting tree has no all useful B-tree inventory like left links and high
> keys
Hi!
> 15 нояб. 2018 г., в 17:30, Andrey Borodin написал(а):
>
> I think I can compose patch for consideration.
I found no practical way to fix concept of "subtree-lock". Pre-locking all left
siblings for cleanup would suffice, but does not look very practical.
GIN Posting tree has no all
Hi!
Mail.App somehow borken my reponse with , which
is far beyond of my understanding of what is plain text, that's why I'll quote
all my previous message here. Hope this mail client is OK.
14.11.2018, 23:13, "Andrey Borodin" :
> Hi everyone!
>
> I didn't noticed this thread for too long
Hi everyone!I didn't noticed this thread for too long somehow, sorry.8 нояб. 2018 г., в 6:46, Peter Geoghegan написал(а):I don't thinkthe general "there can't be any inserters at this subtree" thing worksgiven that we have to couple buffer locks when moving right for otherreasons. We call
There's a mistake in the last mail.About how to reproduct the issue,the following SQL should be:set enable_seqscan = false;select count(*) from tb1 where id =1;4. execute select SQL set enable_seqscan = false; select count(*) from tb1 where id =2;Regard,Chen Huajun
Hi
Before we only discussed the connection hang on the primary, the connection
hang on the standby database should be another problem.
When the recovery process replays the gin's delete WAL record, the order of
get lwlock is not the same as the select process,
resulting in a deadlock. Accord
On Wed, Nov 7, 2018 at 5:46 PM Peter Geoghegan wrote:
> Teodor: Do you think that the issue is fixable? It looks like there
> are serious issues with the design of 218f51584d5 to me. I don't think
> the general "there can't be any inserters at this subtree" thing works
> given that we have to
On Wed, Nov 7, 2018 at 5:46 PM Peter Geoghegan wrote:
> I think that you have to be doing a multi-level delete for a deadlock
> to take place, which isn't particularly likely to coincide with a
> concurrent insertion in general. That may explain why it's taken a
> year to get a high-quality bug
On Mon, Oct 29, 2018 at 12:04 PM chenhj wrote:
> ## stack of autovacuum:Acquire lock 0x2aaab670dfe4 and hold 0x2aaab4009564
> --
> (gdb) bt
> #0 0x7fe11552379b in do_futex_wait.constprop.1 () from
> /lib64/libpthread.so.0
> #1
On Mon, Oct 29, 2018 at 12:04 PM chenhj wrote:
> #4 0x007095ac in LWLockAcquire (lock=0x2aaab4009564,
> mode=LW_EXCLUSIVE) at lwlock.c:1233
> #5 0x00490a32 in ginStepRight (buffer=6713826, index= out>, lockmode=lockmode@entry=2) at ginbtree.c:174
> #6
On Tue, Nov 6, 2018 at 10:05 AM chenhj wrote:
> I analyzed the btree block where lwlock deadlock occurred, as follows:
Thank you for doing this important work.
You're using Postgres 10.2. While that version isn't current with all
GIN bug fixes, it does have this important one:
"Ensure that
Hi,all
In our PostgreSQL 10.2 database, two sessions of insert and autovacuum of gin
index blocked in LWLock:buffer_content.
This blocked checkpoint and dead tuple recycle,and we had to restart the
datebase.
According to the information collected from gcore, a deadlock occurred when
acquiring
64 matches
Mail list logo
