On Tue, 20 Dec 2022 at 09:23, Dilip Kumar wrote:
>
> On Tue, Dec 20, 2022 at 2:32 AM Robert Haas wrote:
> >
> > On Mon, Dec 19, 2022 at 3:48 PM Ted Yu wrote:
> > > It seems the comment for `backend_subxact_overflowed` missed a word.
> > >
> > > Please see the patch.
> >
> > Committed this fix, t
On Tue, Dec 20, 2022 at 2:32 AM Robert Haas wrote:
>
> On Mon, Dec 19, 2022 at 3:48 PM Ted Yu wrote:
> > It seems the comment for `backend_subxact_overflowed` missed a word.
> >
> > Please see the patch.
>
> Committed this fix, thanks.
Thanks, Robert!
--
Regards,
Dilip Kumar
EnterpriseDB: http
On Mon, Dec 19, 2022 at 3:48 PM Ted Yu wrote:
> It seems the comment for `backend_subxact_overflowed` missed a word.
>
> Please see the patch.
Committed this fix, thanks.
--
Robert Haas
EDB: http://www.enterprisedb.com
On Mon, Dec 19, 2022 at 11:57 AM Robert Haas wrote:
> On Tue, Dec 13, 2022 at 2:29 AM Julien Rouhaud wrote:
> > > > Makes sense.
> > >
> > > +1
> >
> > +1
>
> Committed with a bit more word-smithing on the documentation.
>
> --
> Robert Haas
> EDB: http://www.enterprisedb.com
>
> Hi,
It seems th
On Tue, Dec 13, 2022 at 2:29 AM Julien Rouhaud wrote:
> > > Makes sense.
> >
> > +1
>
> +1
Committed with a bit more word-smithing on the documentation.
--
Robert Haas
EDB: http://www.enterprisedb.com
On Tue, Dec 13, 2022 at 5:09 AM Dilip Kumar wrote:
>
> On Mon, Dec 12, 2022 at 11:21 PM Robert Haas wrote:
> >
> > On Mon, Dec 12, 2022 at 12:42 PM Justin Pryzby wrote:
> > > diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
> > > index 4efa1d5fca0..ac15e2ce789 100644
> >
On Mon, Dec 12, 2022 at 11:21 PM Robert Haas wrote:
>
> On Mon, Dec 12, 2022 at 12:42 PM Justin Pryzby wrote:
> > diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
> > index 4efa1d5fca0..ac15e2ce789 100644
> > --- a/doc/src/sgml/monitoring.sgml
> > +++ b/doc/src/sgml/monito
On Mon, Dec 12, 2022 at 12:42 PM Justin Pryzby wrote:
> diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
> index 4efa1d5fca0..ac15e2ce789 100644
> --- a/doc/src/sgml/monitoring.sgml
> +++ b/doc/src/sgml/monitoring.sgml
> @@ -5680,12 +5680,12 @@ FROM pg_stat_get_backend_idse
On Mon, Dec 12, 2022 at 09:33:51AM -0800, Nathan Bossart wrote:
> On Mon, Dec 12, 2022 at 11:15:43AM -0500, Robert Haas wrote:
> > Any strenuous objections?
>
> Nope. In fact, +1. Until more work is done to alleviate the performance
> issues, this information will likely prove useful.
The docs
On Mon, Dec 12, 2022 at 11:15:43AM -0500, Robert Haas wrote:
> Any strenuous objections?
Nope. In fact, +1. Until more work is done to alleviate the performance
issues, this information will likely prove useful.
--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com
On Wed, Nov 30, 2022 at 11:01 AM Robert Haas wrote:
> That's not responsive to the need that I have. I need users to be able
> to figure out which backend(s) are overflowing their snapshots -- and
> perhaps how badly and how often --- not which backends are incurring
> an expense as a result. Ther
On Wed, Nov 23, 2022 at 3:56 PM Andres Freund wrote:
> Indeed. This is why I was thinking that just alerting for overflowed xact
> isn't particularly helpful. You really want to see how much they overflow and
> how often.
I think if we just expose the is-overflowed feld and the count, people
can
On Thu, Nov 24, 2022 at 2:26 AM Andres Freund wrote:
>
> Indeed. This is why I was thinking that just alerting for overflowed xact
> isn't particularly helpful. You really want to see how much they overflow and
> how often.
I think the way of monitoring the subtransaction count and overflow
statu
Hi,
On 2022-11-23 15:25:39 -0500, Robert Haas wrote:
> One thing that I'd really like to see better documented is exactly
> what it is that causes a problem. But first we'd have to understand it
> ourselves. It's not as simple as "if you have more than 64 subxacts in
> any top-level xact, kiss per
On Wed, Nov 23, 2022 at 2:01 PM Bruce Momjian wrote:
> I originally thought having this value in pg_stat_activity was overkill,
> but seeing the other internal/warning columns in that view, I think it
> makes sense. Oddly, is our 64 snapshot performance limit even
> documented anywhere? I know i
On Mon, Nov 14, 2022 at 10:09:57AM -0500, Robert Haas wrote:
> I think I fundamentally disagree with the idea that we should refuse
> to expose instrumentation data because some day the internals might
> change. If we accepted that argument categorically, we wouldn't have
> things like backend_xmin
On Tue, Nov 15, 2022 at 2:29 PM Andres Freund wrote:
> Hence the suggestion to show the pid of the session with the most subxacts. We
> probably also should add a bunch of accessor functions for people that want
> more detail... But just seeing in one place what's problematic would be the
> big ge
On Tue, Nov 15, 2022 at 7:34 PM Robert Haas wrote:
>
> On Mon, Nov 14, 2022 at 4:17 PM Andres Freund wrote:
> > Perhaps this would better be tackled by a new "visibility" view. It could
> > show
> > - number of sessions with a snapshot
> > - max age of backend xmin
> > - pid with max backend xmi
Hi,
On 2022-11-15 09:04:25 -0500, Robert Haas wrote:
> On Mon, Nov 14, 2022 at 4:17 PM Andres Freund wrote:
> > Perhaps this would better be tackled by a new "visibility" view. It could
> > show
> > - number of sessions with a snapshot
> > - max age of backend xmin
> > - pid with max backend xmi
On Mon, Nov 14, 2022 at 4:17 PM Andres Freund wrote:
> Perhaps this would better be tackled by a new "visibility" view. It could show
> - number of sessions with a snapshot
> - max age of backend xmin
> - pid with max backend xmin
> - number of sessions that suboverflowed
> - pid of the session wi
On Mon, Nov 14, 2022 at 10:18 PM David G. Johnston
wrote:
>> Do you have a view on this point?
>>
>
> NULL when overflowed seems like the opposite of the desired effect, calling
> attention to the exceptional status. Make it a text column and write
> "overflow" or "###" as appropriate. Anyone
On Tue, Nov 15, 2022 at 2:47 AM Andres Freund wrote:
>
> First, it's not good to have a cliff that you can't see coming - presumbly
> you'd want to warn *before* you regularly reach PGPROC_MAX_CACHED_SUBXIDS
> subxids, rather when the shit has hit the fan already.
I agree with the point that it i
Hi,
On 2022-11-14 13:43:41 -0500, Robert Haas wrote:
> On Mon, Nov 14, 2022 at 12:47 PM Andres Freund wrote:
> > I'd go the other way. It's pretty unimportant whether it overflowed, it's
> > important how many subtxns there are. The cases where overflowing causes
> > real
> > problems are when t
Robert Haas writes:
> On Mon, Nov 14, 2022 at 12:47 PM Andres Freund wrote:
>> I'd go the other way. It's pretty unimportant whether it overflowed, it's
>> important how many subtxns there are. The cases where overflowing causes real
>> problems are when there's many thousand subtxns - which one
On Mon, Nov 14, 2022 at 2:17 PM David G. Johnston
wrote:
> Assuming getting an actual count value to print is fairly cheap, or even a
> sunk cost if you are going to report overflow, I don't see why we wouldn't
> want to provide the more detailed data.
>
> My concern, through ignorance, with rep
On Mon, Nov 14, 2022 at 11:43 AM Robert Haas wrote:
> On Mon, Nov 14, 2022 at 12:47 PM Andres Freund wrote:
> > I'd go the other way. It's pretty unimportant whether it overflowed, it's
> > important how many subtxns there are. The cases where overflowing causes
> real
> > problems are when ther
On Mon, Nov 14, 2022 at 12:47 PM Andres Freund wrote:
> I'd go the other way. It's pretty unimportant whether it overflowed, it's
> important how many subtxns there are. The cases where overflowing causes real
> problems are when there's many thousand subtxns - which one can't judge just
> from su
Hi,
On 2022-11-14 12:29:58 -0500, Tom Lane wrote:
> I'd vote for just overflowed true/false. Why do people need to know
> the exact number of subtransactions? (If there is a use-case, that
> would definitely be material for an auxiliary function instead of a
> view column.)
I'd go the other way
"David G. Johnston" writes:
> On Mon, Nov 14, 2022 at 9:41 AM Robert Haas wrote:
>> The overhead of fetching the information is not large, but Justin is
>> concerned about the effect on the display width. I feel that's kind of
>> a lost cause because it's so wide already anyway, but I don't see a
On Mon, Nov 14, 2022 at 9:41 AM Robert Haas wrote:
> On Mon, Nov 14, 2022 at 11:35 AM Amit Singh
> wrote:
> > Making the information available in pg_stat_activity makes it a lot
> easier to identify the pid which has caused the subtran overflow. Debugging
> through the app code can be an endless
On Mon, Nov 14, 2022 at 11:35 AM Amit Singh wrote:
> Making the information available in pg_stat_activity makes it a lot easier to
> identify the pid which has caused the subtran overflow. Debugging through the
> app code can be an endless exercise and logging every statement in postgresql
> lo
Making the information available in pg_stat_activity makes it a lot easier
to identify the pid which has caused the subtran overflow. Debugging
through the app code can be an endless exercise and logging every statement
in postgresql logs is not practical either. If the overhead of fetching the
inf
On Mon, Nov 14, 2022 at 11:18 AM David G. Johnston
wrote:
>> I guess that's OK. I don't particularly favor that approach here but I
>> can live with it. I agree that too-wide views are annoying, but as far
>> as pg_stat_activity goes, that ship has pretty much sailed already,
>> and the same is tr
On Mon, Nov 14, 2022 at 9:04 AM Robert Haas wrote:
> On Mon, Nov 14, 2022 at 10:57 AM Justin Pryzby
> wrote:
> > > First, we're just talking about an extra couple of columns in
> > > pg_stat_activity here, which does not seem like a heavy price to pay.
> >
> > The most recent patch adds a separa
On Mon, Nov 14, 2022 at 10:57 AM Justin Pryzby wrote:
> > First, we're just talking about an extra couple of columns in
> > pg_stat_activity here, which does not seem like a heavy price to pay.
>
> The most recent patch adds a separate function rather than adding more
> columns to pg_stat_activity
On Mon, Nov 14, 2022 at 10:09:57AM -0500, Robert Haas wrote:
> On Mon, Mar 21, 2022 at 7:45 PM Andres Freund wrote:
> > > It feels to me like far too much effort is being invested in fundamentally
> > > the wrong direction here. If the subxact overflow business is causing
> > > real-world perform
On Mon, Nov 14, 2022 at 10:41 AM Tom Lane wrote:
> Maybe the original patch took an hour to write, but it's sure been
> bikeshedded to death :-(. I was complaining about the total amount
> of attention spent more than the patch itself.
Unfortunately, that problem is not unique to this patch, and
Robert Haas writes:
> In short, I think this is a good idea, and if somebody thinks that we
> should solve the underlying problem instead, I'd like to hear what
> people think a realistic solution might be. Because to me, it looks
> like we're refusing to commit a patch that probably took an hour
On Mon, Mar 21, 2022 at 7:45 PM Andres Freund wrote:
> > It feels to me like far too much effort is being invested in fundamentally
> > the wrong direction here. If the subxact overflow business is causing
> > real-world performance problems, let's find a way to fix that, not put
> > effort into
On Tue, Mar 22, 2022 at 5:15 AM Andres Freund wrote:
> There seems to be some agreement on this (I certainly do agree). Thus it seems
> we should mark the CF entry as rejected?
>
> It's been failing on cfbot for weeks...
> https://cirrus-ci.com/task/5289336424890368?logs=docs_build#L347
I have
On 2022-01-14 11:25:45 -0500, Tom Lane wrote:
> Julien Rouhaud writes:
> > Like many I previously had to investigate a slowdown due to sub-transaction
> > overflow, and even with the information available in a monitoring view (I
> > had
> > to rely on a quick hackish extension as I couldn't patch
On Fri, Jan 14, 2022 at 9:55 PM Tom Lane wrote:
> Julien Rouhaud writes:
> > Like many I previously had to investigate a slowdown due to
> sub-transaction
> > overflow, and even with the information available in a monitoring view
> (I had
> > to rely on a quick hackish extension as I couldn't pa
On Sat, Jan 15, 2022 at 12:13:39AM -0500, Tom Lane wrote:
>
> The discussion just upthread was envisioning not only that we'd add
> infrastructure for this, but then that other projects would build
> more infrastructure on top of that. That's an awful lot of work
> that will become useless --- in
Julien Rouhaud writes:
> On Fri, Jan 14, 2022 at 07:42:29PM +, Bossart, Nathan wrote:
>> On 1/14/22, 8:26 AM, "Tom Lane" wrote:
>>> It feels to me like far too much effort is being invested in fundamentally
>>> the wrong direction here.
> Agreed, it would be better but if that leads to signi
On Fri, Jan 14, 2022 at 07:42:29PM +, Bossart, Nathan wrote:
> On 1/14/22, 8:26 AM, "Tom Lane" wrote:
> >
> > It feels to me like far too much effort is being invested in fundamentally
> > the wrong direction here. If the subxact overflow business is causing
> > real-world performance problem
On 1/14/22, 8:26 AM, "Tom Lane" wrote:
> Julien Rouhaud writes:
>> Like many I previously had to investigate a slowdown due to sub-transaction
>> overflow, and even with the information available in a monitoring view (I had
>> to rely on a quick hackish extension as I couldn't patch postgres) it'
Julien Rouhaud writes:
> Like many I previously had to investigate a slowdown due to sub-transaction
> overflow, and even with the information available in a monitoring view (I had
> to rely on a quick hackish extension as I couldn't patch postgres) it's not
> terribly fun to do this way. On top
On Fri, Jan 14, 2022 at 1:17 PM Julien Rouhaud wrote:
> On Thu, Jan 13, 2022 at 10:27:31PM +, Bossart, Nathan wrote:
> > Thanks for the new patch!
> >
> > +
> > +Returns a record of information about the backend's
> subtransactions.
> > +The fields returned are subxact_
On Thu, Jan 13, 2022 at 10:27:31PM +, Bossart, Nathan wrote:
> Thanks for the new patch!
>
> +
> +Returns a record of information about the backend's subtransactions.
> +The fields returned are subxact_count
> identifies
> +number of active subtransaction and su
Thanks for the new patch!
+
+Returns a record of information about the backend's subtransactions.
+The fields returned are subxact_count identifies
+number of active subtransaction and subxact_overflow
+ shows whether the backend's subtransaction cache is
+
On Tue, Dec 14, 2021 at 6:23 PM Ashutosh Sharma
wrote:
In the latest patch I have fixed comments given here except a few.
I have looked into the v2 patch and here are my comments:
+ PG_RETURN_INT32(local_beentry->subxact_overflowed);
+}
Should this be PG_RETURN_BOOL instead of PG_RETURN_INT3
On Fri, Dec 17, 2021 at 9:32 AM Justin Pryzby wrote:
> On Fri, Dec 17, 2021 at 09:00:04AM +0530, Dilip Kumar wrote:
> > On Tue, Dec 14, 2021 at 3:57 AM Bossart, Nathan
> wrote:
> > >
> > > On 12/13/21, 6:30 AM, "Dilip Kumar" wrote:
> > > > On Tue, Dec 7, 2021 at 11:11 AM Justin Pryzby
> wrote:
On Fri, Dec 17, 2021 at 09:00:04AM +0530, Dilip Kumar wrote:
> On Tue, Dec 14, 2021 at 3:57 AM Bossart, Nathan wrote:
> >
> > On 12/13/21, 6:30 AM, "Dilip Kumar" wrote:
> > > On Tue, Dec 7, 2021 at 11:11 AM Justin Pryzby
> > > wrote:
> > >> Since I think this field is usually not interesting to
On Tue, Dec 14, 2021 at 6:23 PM Ashutosh Sharma wrote:
>
> Hi,
>
> I have looked into the v2 patch and here are my comments:
>
> + PG_RETURN_INT32(local_beentry->subxact_overflowed);
> +}
>
> Should this be PG_RETURN_BOOL instead of PG_RETURN_INT32??
>
> --
>
> +{ oid => '6107', descr => 'statis
On Tue, Dec 14, 2021 at 3:57 AM Bossart, Nathan wrote:
>
> On 12/13/21, 6:30 AM, "Dilip Kumar" wrote:
> > On Tue, Dec 7, 2021 at 11:11 AM Justin Pryzby wrote:
> >> Since I think this field is usually not interesting to most users of
> >> pg_stat_activity, maybe this should instead be implemented
Hi,
I have looked into the v2 patch and here are my comments:
+ PG_RETURN_INT32(local_beentry->subxact_overflowed);
+}
Should this be PG_RETURN_BOOL instead of PG_RETURN_INT32??
--
+{ oid => '6107', descr => 'statistics: cached subtransaction count of
backend',
+ proname => 'pg_stat_get_bac
On 12/13/21, 6:30 AM, "Dilip Kumar" wrote:
> On Tue, Dec 7, 2021 at 11:11 AM Justin Pryzby wrote:
>> Since I think this field is usually not interesting to most users of
>> pg_stat_activity, maybe this should instead be implemented as a function like
>> pg_backend_get_subxact_status(pid).
>>
>> P
On Tue, Dec 7, 2021 at 11:11 AM Justin Pryzby wrote:
>
> You added this to pg_stat_activity, which already has a lot of fields.
> We talked a few months ago about not adding more fields that weren't commonly
> used.
> https://www.postgresql.org/message-id/flat/20210426191811.sp3o77doinphyjhu%40al
I also want to +1 this this effort. Exposing subtransaction usage is very
useful.
It would also be extremely beneficial to add both subtransaction usage and
overflow counters to pg_stat_database.
Monitoring tools that capture deltas on pg_stat_database will be able to
generate historical anal
On 12/6/21, 8:19 PM, "Dilip Kumar" wrote:
> If the subtransaction cache is overflowed in some of the transactions
> then it will affect all the concurrent queries as they need to access
> the SLRU for checking the visibility of each tuple. But currently
> there is no way to identify whether in an
On Tue, Dec 7, 2021 at 10:29 AM Nikolay Samokhvalov
wrote:
>
> On Mon, Dec 6, 2021 at 8:16 PM Dilip Kumar wrote:
>>
>> If the subtransaction cache is overflowed in some of the transactions
>> then it will affect all the concurrent queries as they need to access
>> the SLRU for checking the visibi
On Tue, Dec 7, 2021 at 11:11 AM Justin Pryzby wrote:
Thanks for the review I will work on these comments.
> > +
> > + subxact_count xid
> > +
> > +
> > + The current backend's active subtransactions count.
>
> subtransaction (no s)
>
> > + Set to true if curre
> +
> + subxact_count xid
> +
> +
> + The current backend's active subtransactions count.
subtransaction (no s)
> + Set to true if current backend's subtransaction cache is overflowed.
Say "has overflowed"
> + if (local_beentry->subxact_count > 0)
On Mon, Dec 6, 2021 at 8:16 PM Dilip Kumar wrote:
> If the subtransaction cache is overflowed in some of the transactions
> then it will affect all the concurrent queries as they need to access
> the SLRU for checking the visibility of each tuple. But currently
> there is no way to identify whet
On Mon, Dec 6, 2021 at 8:17 PM Dilip Kumar wrote:
> If the subtransaction cache is overflowed in some of the transactions
> then it will affect all the concurrent queries as they need to access
> the SLRU for checking the visibility of each tuple. But currently
> there is no way to identify whet
65 matches
Mail list logo
