On Fri, May 27, 2022 at 08:55:02AM -0700, Andres Freund wrote:
> On 2022-05-27 15:44:39 +0530, Amit Kapila wrote:
>> Won't in theory the similar cache in transam.c is also prone to
>> similar behavior?
TransactionIdDidCommit() and TransactionIdDidAbort() are used in much
more code paths for
On Fri, May 27, 2022 at 11:59 AM Andres Freund wrote:
> On 2022-05-27 11:48:45 -0700, Peter Geoghegan wrote:
> > I find it hard to believe that there wasn't even a cursory effort at
> > performance validation before this was committed, but that's what it
> > looks like.
>
> Yea. Imo this pretty
On 2022-05-27 11:48:45 -0700, Peter Geoghegan wrote:
> On Fri, May 27, 2022 at 8:55 AM Andres Freund wrote:
> > > Anyway, how about if we clear this cache for subtrans whenever
> > > TransactionXmin is advanced and cachedFetchSubXid precedes it? The
> > > comments atop
On Fri, May 27, 2022 at 8:55 AM Andres Freund wrote:
> > Anyway, how about if we clear this cache for subtrans whenever
> > TransactionXmin is advanced and cachedFetchSubXid precedes it? The
> > comments atop SubTransGetTopmostTransaction seem to state that we
> > don't care about the exact
Hi,
On 2022-05-27 15:44:39 +0530, Amit Kapila wrote:
> On Thu, May 26, 2022 at 12:53 PM Michael Paquier wrote:
> >
> > On Tue, May 24, 2022 at 04:52:50PM -0700, Andres Freund wrote:
> >
> > > 2) xid wraparound. There's nothing forcing
> > > SubTransGetTopmostTransaction() to
> > >be called
On Thu, May 26, 2022 at 12:53 PM Michael Paquier wrote:
>
> On Tue, May 24, 2022 at 04:52:50PM -0700, Andres Freund wrote:
>
> > 2) xid wraparound. There's nothing forcing SubTransGetTopmostTransaction()
> > to
> >be called regularly, so even if a backend isn't idle, the cache could
> >
On Tue, May 24, 2022 at 04:52:50PM -0700, Andres Freund wrote:
> As is, this strikes me as dangerous. At the very least this ought to be
> structured so it can have assertions verifying that the cache contents are
> correct.
Well, under USE_ASSERT_CHECKING we could force a recalculation of the
Hi,
On 2022-04-07 14:36:35 +0900, Michael Paquier wrote:
> On Mon, Mar 07, 2022 at 10:17:41PM +0800, Julien Rouhaud wrote:
> > On Mon, Mar 07, 2022 at 01:27:40PM +, Simon Riggs wrote:
> >> The patch was posted because TransactionLogFetch() has a cache, yet
> >> SubTransGetTopmostTransaction()
On Thu, 7 Apr 2022 at 00:36, Michael Paquier wrote:
>
> On Mon, Mar 07, 2022 at 10:17:41PM +0800, Julien Rouhaud wrote:
> > On Mon, Mar 07, 2022 at 01:27:40PM +, Simon Riggs wrote:
> >> The patch was posted because TransactionLogFetch() has a cache, yet
> >> SubTransGetTopmostTransaction()
On Mon, Mar 07, 2022 at 10:17:41PM +0800, Julien Rouhaud wrote:
> On Mon, Mar 07, 2022 at 01:27:40PM +, Simon Riggs wrote:
>> The patch was posted because TransactionLogFetch() has a cache, yet
>> SubTransGetTopmostTransaction() does not, yet the argument should be
>> identical in both cases.
On Mon, Mar 07, 2022 at 01:27:40PM +, Simon Riggs wrote:
> > +/*
> > + * Single-item cache for results of SubTransGetTopmostTransaction. It's
> > worth having
> > + * such a cache because we frequently find ourselves repeatedly checking
> > the
> > + * same XID, for example when scanning a
On Mon, 7 Mar 2022 at 09:49, Julien Rouhaud wrote:
>
> Hi,
>
> On Mon, Jan 17, 2022 at 01:44:02PM +, Simon Riggs wrote:
> >
> > Re-attached, so that the CFapp isn't confused between the multiple
> > patches on this thread.
>
> Thanks a lot for working on this!
>
> The patch is simple and
Hi,
On Mon, Jan 17, 2022 at 01:44:02PM +, Simon Riggs wrote:
>
> Re-attached, so that the CFapp isn't confused between the multiple
> patches on this thread.
Thanks a lot for working on this!
The patch is simple and overall looks good to me. A few comments though:
+/*
+ * Single-item
> 17 янв. 2022 г., в 18:44, Simon Riggs
> написал(а):
>
> Re-attached, so that the CFapp isn't confused between the multiple
> patches on this thread.
FWIW I've looked into the patch and it looks good to me. Comments describing
when the cache is useful seem valid.
Thanks!
Best regards,
On Tue, 30 Nov 2021 at 12:19, Simon Riggs wrote:
>
> On Mon, 30 Aug 2021 at 11:25, Andrey Borodin wrote:
> >
> > Hi Pengcheng!
> >
> > You are solving important problem, thank you!
> >
> > > 30 авг. 2021 г., в 13:43, Pengchengliu
> > > написал(а):
> > >
> > > To resolve this performance
Hi,
On Wed, Dec 08, 2021 at 04:39:11PM +, Simon Riggs wrote:
>
> This patch shows where I'm going, with changes in GetSnapshotData()
> and XidInMVCCSnapshot() and XactLockTableWait().
> Passes make check, but needs much more, so this is review-only at this
> stage to give a flavour of what
On Fri, 3 Dec 2021 at 06:27, Dilip Kumar wrote:
>
> On Tue, Nov 30, 2021 at 5:49 PM Simon Riggs
> wrote:
>
> > transam.c uses a single item cache to prevent thrashing from repeated
> > lookups, which reduces problems with shared access to SLRUs.
> > multitrans.c also has similar.
> >
> > I
On Wed, 1 Dec 2021 at 06:41, Andrey Borodin wrote:
> > On review, I think it is also possible that we update subtrans ONLY if
> > someone uses >PGPROC_MAX_CACHED_SUBXIDS.
> > This would make subtrans much smaller and avoid one-entry-per-page
> > which is a major source of cacheing.
> > This
On Fri, Dec 3, 2021 at 5:00 PM Simon Riggs wrote:
>
> On Fri, 3 Dec 2021 at 01:27, Dilip Kumar wrote:
>
> > > On review, I think it is also possible that we update subtrans ONLY if
> > > someone uses >PGPROC_MAX_CACHED_SUBXIDS.
> > > This would make subtrans much smaller and avoid
On Fri, 3 Dec 2021 at 01:27, Dilip Kumar wrote:
> > On review, I think it is also possible that we update subtrans ONLY if
> > someone uses >PGPROC_MAX_CACHED_SUBXIDS.
> > This would make subtrans much smaller and avoid one-entry-per-page
> > which is a major source of cacheing.
> > This would
On Tue, Nov 30, 2021 at 5:49 PM Simon Riggs
wrote:
> transam.c uses a single item cache to prevent thrashing from repeated
> lookups, which reduces problems with shared access to SLRUs.
> multitrans.c also has similar.
>
> I notice that subtrans. doesn't have this, but could easily do so.
>
> 30 нояб. 2021 г., в 17:19, Simon Riggs
> написал(а):
>
> On Mon, 30 Aug 2021 at 11:25, Andrey Borodin wrote:
>>
>> Hi Pengcheng!
>>
>> You are solving important problem, thank you!
>>
>>> 30 авг. 2021 г., в 13:43, Pengchengliu написал(а):
>>>
>>> To resolve this performance problem,
On Mon, 30 Aug 2021 at 11:25, Andrey Borodin wrote:
>
> Hi Pengcheng!
>
> You are solving important problem, thank you!
>
> > 30 авг. 2021 г., в 13:43, Pengchengliu написал(а):
> >
> > To resolve this performance problem, we think about a solution which cache
> > SubtransSLRU to local cache.
> >
6602.635 ms (00:06.603)
Thanks
Pengcheng
-Original Message-
From: Andrey Borodin
Sent: 2021年9月3日 14:51
To: Pengchengliu
Cc: pgsql-hack...@postgresql.org
Subject: Re: suboverflowed subtransactions concurrency performance optimize
Sorry, for some reason Mail.app converted message to html
Sorry, for some reason Mail.app converted message to html and mailing list
mangled this html into mess. I'm resending previous message as plain text
again. Sorry for the noise.
> 31 авг. 2021 г., в 11:43, Pengchengliu написал(а):
>
> Hi Andrey,
> Thanks a lot for your replay and reference
31 авг. 2021 г., в 11:43, Pengchengliu написал(а):Hi Andrey, Thanks a lot for your replay and reference information. The default NUM_SUBTRANS_BUFFERS is 32. My implementation is local_cache_subtrans_pages can be adjusted dynamically. If we configure local_cache_subtrans_pages as 64, every backend
On Mon, Aug 30, 2021 at 1:43 AM Pengchengliu
wrote:
> Hi hackers,
> I wrote a patch to resolve the subtransactions concurrency performance
> problems when suboverflowed.
>
> When we use more than PGPROC_MAX_CACHED_SUBXIDS(64) subtransactions per
> transaction concurrency, it will lead to
/34/2627/
[1]
https://www.postgresql.org/message-id/flat/20180814213500.GA74618%4060f81dc409fc.ant.amazon.com
Thanks
Pengcheng
-Original Message-
From: Andrey Borodin
Sent: 2021年8月30日 18:25
To: Pengchengliu
Cc: pgsql-hack...@postgresql.org
Subject: Re: suboverflowed subtransacti
Hi Pengcheng!
You are solving important problem, thank you!
> 30 авг. 2021 г., в 13:43, Pengchengliu написал(а):
>
> To resolve this performance problem, we think about a solution which cache
> SubtransSLRU to local cache.
> First we can query parent transaction id from SubtransSLRU, and copy
29 matches
Mail list logo
