Re: RFC: Logging plan of the running query

2024-03-25 Thread Andres Freund
Hi, On 2024-03-13 15:33:02 -0400, Robert Haas wrote: > But also ... having to wrap the entire plan tree like this seems > pretty awful. I don't really like the idea of a large-scan plan > modification like this in the middle of the query. It's not great. But I also don't really see an

Re: RFC: Logging plan of the running query

2024-03-17 Thread torikoshia
On 2024-03-14 04:33, Robert Haas wrote: Thanks for your review! On Wed, Mar 13, 2024 at 1:28 AM torikoshia wrote: - I saw no way to find the next node to be executed from the planstate tree, so the patch wraps all the ExecProcNode of the planstate tree at CHECK_FOR_INTERRUPTS(). I don't

Re: RFC: Logging plan of the running query

2024-03-13 Thread jian he
On Wed, Mar 13, 2024 at 1:28 PM torikoshia wrote: > > On Fri, Feb 16, 2024 at 11:42 PM torikoshia > wrote: > > I'm not so sure about the implementation now, i.e. finding the next > > node > > to be executed from the planstate tree, but I'm going to try this > > approach. > > Attached a patch

Re: RFC: Logging plan of the running query

2024-03-13 Thread Robert Haas
On Wed, Mar 13, 2024 at 1:28 AM torikoshia wrote: > - I saw no way to find the next node to be executed from the planstate > tree, so the patch wraps all the ExecProcNode of the planstate tree at > CHECK_FOR_INTERRUPTS(). I don't think it does this correctly, because some node types have

Re: RFC: Logging plan of the running query

2024-03-12 Thread torikoshia
On Fri, Feb 16, 2024 at 11:42 PM torikoshia wrote: I'm not so sure about the implementation now, i.e. finding the next node to be executed from the planstate tree, but I'm going to try this approach. Attached a patch which takes this approach. - I saw no way to find the next node to be

Re: RFC: Logging plan of the running query

2024-03-04 Thread Robert Haas
On Sat, Mar 2, 2024 at 10:46 AM James Coleman wrote: > If I can rephrase this idea: it's basically "delay this interrupt > until inline to the next ExecProcNode execution". Yes, but it's not just that. It also means that the code which would handle the interrupt doesn't need to be called at

Re: RFC: Logging plan of the running query

2024-03-02 Thread James Coleman
On Wed, Feb 28, 2024 at 1:18 AM Robert Haas wrote: > > On Mon, Feb 26, 2024 at 5:31 PM torikoshia wrote: > > It would be nice if there was a place accessed once every few seconds or > > so.. > > I think this comment earlier from Andres deserves close attention: > > # If we went with something

Re: RFC: Logging plan of the running query

2024-02-27 Thread Robert Haas
On Mon, Feb 26, 2024 at 5:31 PM torikoshia wrote: > It would be nice if there was a place accessed once every few seconds or > so.. I think this comment earlier from Andres deserves close attention: # If we went with something like tht approach, I think we'd have to do something # like

Re: RFC: Logging plan of the running query

2024-02-26 Thread Julien Rouhaud
On Mon, Feb 26, 2024 at 01:56:44PM +0530, Robert Haas wrote: > On Sun, Feb 25, 2024 at 5:00 PM Julien Rouhaud wrote: > > Yeah, trying to find a generalized solution seems like worth investing some > > time to avoid having a bunch of CHECK_FOR_XXX() calls scattered in the code > > a > > few years

Re: RFC: Logging plan of the running query

2024-02-26 Thread torikoshia
On 2024-02-24 00:23, Robert Haas wrote: On Fri, Feb 23, 2024 at 7:50 PM Julien Rouhaud wrote: On Fri, Feb 23, 2024 at 10:22:32AM +0530, Robert Haas wrote: > On Thu, Feb 22, 2024 at 6:25 AM James Coleman wrote: > > This is potentially a bit of a wild idea, but I wonder if having some > > kind

Re: RFC: Logging plan of the running query

2024-02-26 Thread Robert Haas
On Sun, Feb 25, 2024 at 5:00 PM Julien Rouhaud wrote: > Yeah, trying to find a generalized solution seems like worth investing some > time to avoid having a bunch of CHECK_FOR_XXX() calls scattered in the code a > few years down the road. I just don't really see how to do it. I suspect that

Re: RFC: Logging plan of the running query

2024-02-26 Thread Ashutosh Bapat
On Mon, Feb 26, 2024 at 1:25 PM Julien Rouhaud wrote: > > > > > > > > #define CHECK_FOR_INTERRUPTS_X(x, f, CFI_IMPL, ...) CFI_IMPL > > > > > > #define CHECK_FOR_INTERRUPTS(...) \ > > > CHECK_FOR_INTERRUPTS_X(, ##__VA_ARGS__, \ > > >

Re: RFC: Logging plan of the running query

2024-02-25 Thread Julien Rouhaud
On Mon, Feb 26, 2024 at 12:19:42PM +0530, Ashutosh Bapat wrote: > On Sun, Feb 25, 2024 at 5:00 PM Julien Rouhaud wrote: > > > > > > On Fri, Feb 23, 2024 at 7:50 PM Julien Rouhaud > > > > wrote: > > > > > > > > > > But it doesn't have to be all or nothing right? I mean each call > > > > >

Re: RFC: Logging plan of the running query

2024-02-25 Thread Ashutosh Bapat
On Sun, Feb 25, 2024 at 5:00 PM Julien Rouhaud wrote: > > On Sat, Feb 24, 2024 at 08:56:41AM -0500, James Coleman wrote: > > On Fri, Feb 23, 2024 at 10:23 AM Robert Haas wrote: > > > > > > On Fri, Feb 23, 2024 at 7:50 PM Julien Rouhaud wrote: > > > > > > > > But it doesn't have to be all or

Re: RFC: Logging plan of the running query

2024-02-25 Thread Julien Rouhaud
On Sat, Feb 24, 2024 at 08:56:41AM -0500, James Coleman wrote: > On Fri, Feb 23, 2024 at 10:23 AM Robert Haas wrote: > > > > On Fri, Feb 23, 2024 at 7:50 PM Julien Rouhaud wrote: > > > > > > But it doesn't have to be all or nothing right? I mean each call could > > > say > > > what the

Re: RFC: Logging plan of the running query

2024-02-24 Thread James Coleman
On Fri, Feb 23, 2024 at 10:23 AM Robert Haas wrote: > > On Fri, Feb 23, 2024 at 7:50 PM Julien Rouhaud wrote: > > On Fri, Feb 23, 2024 at 10:22:32AM +0530, Robert Haas wrote: > > > On Thu, Feb 22, 2024 at 6:25 AM James Coleman wrote: > > > > This is potentially a bit of a wild idea, but I

Re: RFC: Logging plan of the running query

2024-02-23 Thread Robert Haas
On Fri, Feb 23, 2024 at 7:50 PM Julien Rouhaud wrote: > On Fri, Feb 23, 2024 at 10:22:32AM +0530, Robert Haas wrote: > > On Thu, Feb 22, 2024 at 6:25 AM James Coleman wrote: > > > This is potentially a bit of a wild idea, but I wonder if having some > > > kind of argument to

Re: RFC: Logging plan of the running query

2024-02-23 Thread Julien Rouhaud
Hi, On Fri, Feb 23, 2024 at 10:22:32AM +0530, Robert Haas wrote: > On Thu, Feb 22, 2024 at 6:25 AM James Coleman wrote: > > This is potentially a bit of a wild idea, but I wonder if having some > > kind of argument to CHECK_FOR_INTERRUPTS() signifying we're in > > "normal" as opposed to

Re: RFC: Logging plan of the running query

2024-02-22 Thread Robert Haas
On Thu, Feb 22, 2024 at 6:25 AM James Coleman wrote: > This is potentially a bit of a wild idea, but I wonder if having some > kind of argument to CHECK_FOR_INTERRUPTS() signifying we're in > "normal" as opposed to "critical" (using that word differently than > the existing critical sections)

Re: RFC: Logging plan of the running query

2024-02-21 Thread Ashutosh Bapat
On Thu, Feb 22, 2024 at 6:25 AM James Coleman wrote: > > > > ...I think the current approach is just plain dead, because of this > > issue. We can't take an approach that creates an unbounded number of > > unclear reentrancy issues and then insert hacks one by one to cure > > them (or hack around

Re: RFC: Logging plan of the running query

2024-02-21 Thread James Coleman
On Mon, Feb 19, 2024 at 11:53 PM Robert Haas wrote: > > On Fri, Feb 16, 2024 at 12:29 AM Andres Freund wrote: > > If we went with something like tht approach, I think we'd have to do > > something > > like redirecting node->ExecProcNode to a wrapper, presumably from within a > > CFI. That

Re: RFC: Logging plan of the running query

2024-02-19 Thread Robert Haas
On Fri, Feb 16, 2024 at 12:29 AM Andres Freund wrote: > If we went with something like tht approach, I think we'd have to do something > like redirecting node->ExecProcNode to a wrapper, presumably from within a > CFI. That wrapper could then implement the explain support, without slowing > down

Re: RFC: Logging plan of the running query

2024-02-16 Thread torikoshia
On Thu, Feb 15, 2024 at 6:12 PM Robert Haas wrote: Hi, I've just been catching up on this thread. + if (MyProc->heldLocks) + { + ereport(LOG_SERVER_ONLY, + errmsg("ignored request for logging query plan due to lock conflicts"), + errdetail("You can try again in a moment.")); + return; + }

Re: RFC: Logging plan of the running query

2024-02-15 Thread Andres Freund
Hi, On 2024-02-15 14:42:11 +0530, Robert Haas wrote: > I think the issue is very general. We have lots of subsystems that > both (a) use global variables and (b) contain CHECK_FOR_INTERRUPTS(). > If we process an interrupt while that code is in the middle of > manipulating its global variables

Re: RFC: Logging plan of the running query

2024-02-15 Thread Robert Haas
Hi, I've just been catching up on this thread. + if (MyProc->heldLocks) + { + ereport(LOG_SERVER_ONLY, + errmsg("ignored request for logging query plan due to lock conflicts"), + errdetail("You can try again in a moment.")); + return; + } I don't like this for several reasons. First, I think

Re: RFC: Logging plan of the running query

2024-02-13 Thread torikoshia
On 2024-02-13 11:30, torikoshia wrote: I'll update the patch including other points such as removing ensuring no page lock code. Updated the patch. Using injection point support we should be able to add tests for testing pg_log_query_plan behaviour when there are page locks held or when

Re: RFC: Logging plan of the running query

2024-02-12 Thread torikoshia
On 2024-02-12 09:00, jian he wrote: Thanks for you comments. On Mon, Jan 29, 2024 at 9:02 PM torikoshia wrote: Hi, Updated the patch to fix typos and move ProcessLogQueryPlanInterruptActive from errfinish() to AbortTransaction. + + + + pg_log_query_plan +

Re: RFC: Logging plan of the running query

2024-02-12 Thread torikoshia
On 2024-02-07 19:14, torikoshia wrote: On 2024-02-07 13:58, Ashutosh Bapat wrote: The prologue refers to a very populated lock hash table. I think that will happen if thousands of tables are queried in a single query OR a query runs on a partitioned table with thousands of partitions. May be

Re: RFC: Logging plan of the running query

2024-02-11 Thread jian he
On Mon, Feb 12, 2024 at 12:42 PM Ashutosh Bapat wrote: > > On Mon, Feb 12, 2024 at 5:31 AM jian he wrote: > > > > On Wed, Feb 7, 2024 at 12:58 PM Ashutosh Bapat > > wrote: > > > > > > > > > > > > */ > > > > > How bad this performance could be. Let's assume that a query is taking > > > > > time

Re: RFC: Logging plan of the running query

2024-02-11 Thread Ashutosh Bapat
On Mon, Feb 12, 2024 at 5:31 AM jian he wrote: > > On Wed, Feb 7, 2024 at 12:58 PM Ashutosh Bapat > wrote: > > > > > > > > > */ > > > > How bad this performance could be. Let's assume that a query is taking > > > > time and pg_log_query_plan() is invoked to examine the plan of this > > > >

Re: RFC: Logging plan of the running query

2024-02-11 Thread Ashutosh Bapat
On Mon, Feb 12, 2024 at 5:31 AM jian he wrote: > > I found a bug: > src8=# select *, pg_sleep(10) from tenk1 for update; > 2024-02-11 15:54:17.944 CST [48602] LOG: query plan running on > backend with PID 48602 is: > Query Text: select *, pg_sleep(10) from tenk1 for update; >

Re: RFC: Logging plan of the running query

2024-02-11 Thread jian he
On Wed, Feb 7, 2024 at 12:58 PM Ashutosh Bapat wrote: > > > > > > */ > > > How bad this performance could be. Let's assume that a query is taking > > > time and pg_log_query_plan() is invoked to examine the plan of this > > > query. Is it possible that the looping over all the locks itself takes

Re: RFC: Logging plan of the running query

2024-02-11 Thread jian he
On Mon, Jan 29, 2024 at 9:02 PM torikoshia wrote: > > Hi, > > Updated the patch to fix typos and move > ProcessLogQueryPlanInterruptActive from errfinish() to AbortTransaction. > + + + + pg_log_query_plan + +pg_log_query_plan ( pid integer ) +

Re: RFC: Logging plan of the running query

2024-02-07 Thread torikoshia
On 2024-02-07 13:58, Ashutosh Bapat wrote: On Wed, Feb 7, 2024 at 9:38 AM torikoshia wrote: Hi Ashutosh, On 2024-02-06 19:51, Ashutosh Bapat wrote: > Thanks for the summary. It is helpful. I think patch is also getting > better. > > I have a few questions and suggestions Thanks for your

Re: RFC: Logging plan of the running query

2024-02-06 Thread Ashutosh Bapat
On Wed, Feb 7, 2024 at 9:38 AM torikoshia wrote: > > Hi Ashutosh, > > On 2024-02-06 19:51, Ashutosh Bapat wrote: > > > Thanks for the summary. It is helpful. I think patch is also getting > > better. > > > > I have a few questions and suggestions > > Thanks for your comments. > > > 1. Prologue of

Re: RFC: Logging plan of the running query

2024-02-06 Thread torikoshia
Hi Ashutosh, On 2024-02-06 19:51, Ashutosh Bapat wrote: Thanks for the summary. It is helpful. I think patch is also getting better. I have a few questions and suggestions Thanks for your comments. 1. Prologue of GetLockMethodLocalHash() mentions * NOTE: When there are many entries in

Re: RFC: Logging plan of the running query

2024-02-06 Thread Ashutosh Bapat
Hi Atsushi, On Mon, Jan 29, 2024 at 6:32 PM torikoshia wrote: > > Hi, > > Updated the patch to fix typos and move > ProcessLogQueryPlanInterruptActive from errfinish() to AbortTransaction. > > > BTW since the thread is getting long, I list the some points of the > discussion so far: > > #

Re: RFC: Logging plan of the running query

2024-01-29 Thread torikoshia
Hi, Updated the patch to fix typos and move ProcessLogQueryPlanInterruptActive from errfinish() to AbortTransaction. BTW since the thread is getting long, I list the some points of the discussion so far: # Safety concern ## Catalog access inside CFI - it seems safe if the CFI call is

Re: RFC: Logging plan of the running query

2023-12-10 Thread torikoshia
On 2023-12-07 08:33, Rafael Thofehrn Castro wrote: Hello hackers, Last Saturday I submitted a patch to the pgsql-hackers list with the title "Proposal: In-flight explain logging" with a patch proposing a feature very similar to the one being worked on in this thread. I should have done a better

Re: RFC: Logging plan of the running query

2023-12-06 Thread Rafael Thofehrn Castro
Hello hackers, Last Saturday I submitted a patch to the pgsql-hackers list with the title "Proposal: In-flight explain logging" with a patch proposing a feature very similar to the one being worked on in this thread. I should have done a better search in the commitfest before implementing

Re: RFC: Logging plan of the running query

2023-11-09 Thread Ashutosh Bapat
On Thu, Nov 9, 2023 at 4:56 PM torikoshia wrote: > > On 2023-11-09 16:11, Ashutosh Bapat wrote: > > On Thu, Nov 9, 2023 at 12:03 PM torikoshia > > wrote: > >> >> > >> >> 1. When a backend is running nested queries, we will see the plan of > >> >> the innermost query. That query may not be the

Re: RFC: Logging plan of the running query

2023-11-09 Thread torikoshia
On 2023-11-09 16:11, Ashutosh Bapat wrote: On Thu, Nov 9, 2023 at 12:03 PM torikoshia wrote: >> >> 1. When a backend is running nested queries, we will see the plan of >> the innermost query. That query may not be the actual culprit if the >> user query is running slowly. E.g a query being run

Re: RFC: Logging plan of the running query

2023-11-08 Thread Ashutosh Bapat
On Thu, Nov 9, 2023 at 12:03 PM torikoshia wrote: > >> > >> 1. When a backend is running nested queries, we will see the plan of > >> the innermost query. That query may not be the actual culprit if the > >> user query is running slowly. E.g a query being run as part of inner > >> side nested

Re: RFC: Logging plan of the running query

2023-11-08 Thread torikoshia
On 2023-11-06 15:32, Ashutosh Bapat wrote: Thanks for the suggestion and example. On Fri, Nov 3, 2023 at 7:31 PM Ashutosh Bapat wrote: I have following questions related to the functionality. (Please point me to the relevant discussion if this has been already discussed.) 1. When a backend

Re: RFC: Logging plan of the running query

2023-11-05 Thread Ashutosh Bapat
On Fri, Nov 3, 2023 at 7:31 PM Ashutosh Bapat wrote: > > I have following questions related to the functionality. (Please point > me to the relevant discussion if this has been already discussed.) > > 1. When a backend is running nested queries, we will see the plan of > the innermost query. That

Re: RFC: Logging plan of the running query

2023-11-03 Thread Ashutosh Bapat
I have following questions related to the functionality. (Please point me to the relevant discussion if this has been already discussed.) 1. When a backend is running nested queries, we will see the plan of the innermost query. That query may not be the actual culprit if the user query is running

Re: RFC: Logging plan of the running query

2023-10-27 Thread Étienne BERSAC
> Hi, > > If we use client log message, that message is shown on the target > process whose pid is specified by the parameter of > pg_log_query_plan(): > >    (pid:1000)=# select pg_sleep(60); >    (pid:1001)=# select pg_log_query_plan(1000); >    (pid:1000)=# LOG:  query plan running on

Re: RFC: Logging plan of the running query

2023-10-27 Thread torikoshia
On 2023-10-27 16:06, Étienne BERSAC wrote: Hi Torikoshia, If so, we once tried to implement such function for getting memory contexts. However, this attempt didn't succeed because it could lead dead lock situation[1]. Thanks for the pointer. Why not use client log message to allow client to

Re: RFC: Logging plan of the running query

2023-10-27 Thread Étienne BERSAC
Hi Torikoshia, > If so, we once tried to implement such function for getting memory > contexts. > However, this attempt didn't succeed because it could lead dead lock > situation[1]. Thanks for the pointer. Why not use client log message to allow client to get plan without locking backend

Re: RFC: Logging plan of the running query

2023-10-26 Thread torikoshia
On 2023-10-25 12:40, Ashutosh Bapat wrote: On Wed, Oct 18, 2023 at 10:04 PM James Coleman wrote: While that decision as regards auto_explain has long since been made (and there would be work to undo it), I don't believe that we should repeat that choice here. I'm -10 on moving this into

Re: RFC: Logging plan of the running query

2023-10-24 Thread Ashutosh Bapat
On Wed, Oct 18, 2023 at 10:04 PM James Coleman wrote: > > While that decision as regards auto_explain has long since been made > (and there would be work to undo it), I don't believe that we should > repeat that choice here. I'm -10 on moving this into auto_explain. > Right. > However perhaps

Re: RFC: Logging plan of the running query

2023-10-24 Thread torikoshia
On 2023-10-24 16:12, Étienne BERSAC wrote: Hi, Yeah, and when we have a situation where we want to run pg_log_query_plan(), we can run it in any environment as long as it is bundled with the core. Is it possible to get the plan of a backend programmatically without access to the logs?

Re: RFC: Logging plan of the running query

2023-10-24 Thread Étienne BERSAC
> Hi, > > Yeah, and when we have a situation where we want to run > pg_log_query_plan(), we can run it in any environment as long as it > is bundled with the core. Is it possible to get the plan of a backend programmatically without access to the logs? Something like pg_get_query_plan(pid,

Re: RFC: Logging plan of the running query

2023-10-18 Thread James Coleman
On Wed, Oct 18, 2023 at 2:09 AM torikoshia wrote: > > On 2023-10-16 18:46, Ashutosh Bapat wrote: > > On Thu, Oct 12, 2023 at 6:51 PM torikoshia > > wrote: > >> > >> On 2023-10-11 16:22, Ashutosh Bapat wrote: > >> > > >> > Considering the similarity with auto_explain I wondered whether this > >>

Re: RFC: Logging plan of the running query

2023-10-18 Thread Ashutosh Bapat
On Wed, Oct 18, 2023 at 11:39 AM torikoshia wrote: > > > > I am more interested in avoiding the duplication of code, esp. the > > first comment in my reply > > If there are no objections, I will try porting it to auto_explain and > see its feasibility. If you want it in core, you will need to

Re: RFC: Logging plan of the running query

2023-10-18 Thread torikoshia
On 2023-10-16 18:46, Ashutosh Bapat wrote: On Thu, Oct 12, 2023 at 6:51 PM torikoshia wrote: On 2023-10-11 16:22, Ashutosh Bapat wrote: > > Considering the similarity with auto_explain I wondered whether this > function should be part of auto_explain contrib module itself? If we > do that

Re: RFC: Logging plan of the running query

2023-10-16 Thread Ashutosh Bapat
On Thu, Oct 12, 2023 at 6:51 PM torikoshia wrote: > > On 2023-10-11 16:22, Ashutosh Bapat wrote: > > > > Considering the similarity with auto_explain I wondered whether this > > function should be part of auto_explain contrib module itself? If we > > do that users will need to load auto_explain

Re: RFC: Logging plan of the running query

2023-10-12 Thread torikoshia
On 2023-10-11 16:22, Ashutosh Bapat wrote: Like many others I think this feature is useful to debug a long running query. Sorry for jumping late into this. I have a few of high level comments Thanks for your comments! There is a lot of similarity between what this feature does and what

Re: RFC: Logging plan of the running query

2023-10-11 Thread Ashutosh Bapat
Hi, On Tue, Oct 10, 2023 at 7:00 PM torikoshia wrote: > Thanks for your comments! > > I tested v30 patch with > v28-0002-Testing-attempt-logging-plan-on-ever-CFI-call.patch which makes > CFI() call ProcessLogQueryPlanInterrupt() internally, and confirmed that > very few logging queries failed

Re: RFC: Logging plan of the running query

2023-10-10 Thread torikoshia
On 2023-10-04 03:00, James Coleman wrote: and I think what we need to do is explicitly disallow running this code any time we are inside of lock acquisition code. Updated patch to check if any locks have already been acquired by examining MyProc->heldLocks. I'm not sure this change can

Re: RFC: Logging plan of the running query

2023-10-06 Thread Andres Freund
Hi, On 2023-10-06 21:58:46 +0900, torikoshia wrote: > Yeah, I think it's a straightforward workaround. > And I'm also concerned that the condition of being in the process > of acquiring some kind of lock is too strict and will make it almost > impossible to output a running plan. How so? We

Re: RFC: Logging plan of the running query

2023-10-06 Thread James Coleman
On Fri, Oct 6, 2023 at 8:58 AM torikoshia wrote: > > On 2023-10-04 03:00, James Coleman wrote: > > On Thu, Sep 7, 2023 at 2:09 AM torikoshia > > wrote: > >> > >> On 2023-09-06 11:17, James Coleman wrote: > >> > >> >> > I've never been able to reproduce it (haven't tested the new version, > >> >>

Re: RFC: Logging plan of the running query

2023-10-06 Thread torikoshia
On 2023-10-04 03:00, James Coleman wrote: On Thu, Sep 7, 2023 at 2:09 AM torikoshia wrote: On 2023-09-06 11:17, James Coleman wrote: >> > I've never been able to reproduce it (haven't tested the new version, >> > but v28 at least) on my M1 Mac; where I've reproduced it is on Debian >> >

Re: RFC: Logging plan of the running query

2023-10-03 Thread James Coleman
On Thu, Sep 7, 2023 at 2:09 AM torikoshia wrote: > > On 2023-09-06 11:17, James Coleman wrote: > > >> > I've never been able to reproduce it (haven't tested the new version, > >> > but v28 at least) on my M1 Mac; where I've reproduced it is on Debian > >> > (first buster and now bullseye). > >> >

Re: RFC: Logging plan of the running query

2023-09-27 Thread Andrey Lepikhov
On 28/9/2023 09:04, torikoshia wrote: On 2023-09-25 18:49, Andrey Lepikhov wrote: On 25/9/2023 14:21, torikoshia wrote: On 2023-09-20 14:39, Lepikhov Andrei wrote: Hmm, as a test, I made sure to call ProcessLogQueryPlanInterrupt() on all CFI using

Re: RFC: Logging plan of the running query

2023-09-27 Thread torikoshia
On 2023-09-25 18:49, Andrey Lepikhov wrote: On 25/9/2023 14:21, torikoshia wrote: On 2023-09-20 14:39, Lepikhov Andrei wrote: Hmm, as a test, I made sure to call ProcessLogQueryPlanInterrupt() on all CFI using v28-0002-Testing-attempt-logging-plan-on-ever-CFI-call.patch, and then ran the

Re: RFC: Logging plan of the running query

2023-09-25 Thread Andrey Lepikhov
On 25/9/2023 14:21, torikoshia wrote: On 2023-09-20 14:39, Lepikhov Andrei wrote: Hmm, as a test, I made sure to call ProcessLogQueryPlanInterrupt() on all CFI using v28-0002-Testing-attempt-logging-plan-on-ever-CFI-call.patch, and then ran the following query but did not cause any problems.

Re: RFC: Logging plan of the running query

2023-09-25 Thread torikoshia
On 2023-09-20 14:39, Lepikhov Andrei wrote: Thanks for your reply. On Tue, Sep 19, 2023, at 8:39 PM, torikoshia wrote: On 2023-09-15 15:21, Lepikhov Andrei wrote: On Thu, Sep 7, 2023, at 1:09 PM, torikoshia wrote: I have explored this patch and, by and large, agree with the code. But some

Re: RFC: Logging plan of the running query

2023-09-19 Thread Lepikhov Andrei
On Tue, Sep 19, 2023, at 8:39 PM, torikoshia wrote: > On 2023-09-15 15:21, Lepikhov Andrei wrote: >> On Thu, Sep 7, 2023, at 1:09 PM, torikoshia wrote: >> I have explored this patch and, by and large, agree with the code. But >> some questions I want to discuss: >> 1. Maybe add a hook to give a

Re: RFC: Logging plan of the running query

2023-09-19 Thread torikoshia
On 2023-09-15 15:21, Lepikhov Andrei wrote: On Thu, Sep 7, 2023, at 1:09 PM, torikoshia wrote: On 2023-09-06 11:17, James Coleman wrote: It seems that we can know what queries were running from the stack traces you shared. As described above, I suspect a lock which was acquired prior to

Re: RFC: Logging plan of the running query

2023-09-15 Thread Lepikhov Andrei
On Thu, Sep 7, 2023, at 1:09 PM, torikoshia wrote: > On 2023-09-06 11:17, James Coleman wrote: > It seems that we can know what queries were running from the stack > traces you shared. > As described above, I suspect a lock which was acquired prior to > ProcessLogQueryPlanInterrupt() caused the

Re: RFC: Logging plan of the running query

2023-09-07 Thread torikoshia
On 2023-09-06 11:17, James Coleman wrote: > I've never been able to reproduce it (haven't tested the new version, > but v28 at least) on my M1 Mac; where I've reproduced it is on Debian > (first buster and now bullseye). > > I'm attaching several stacktraces in the hope that they provide some >

Re: RFC: Logging plan of the running query

2023-09-05 Thread James Coleman
On Tue, Sep 5, 2023 at 9:59 AM torikoshia wrote: > > On 2023-08-28 22:47, James Coleman wrote: > > On Mon, Aug 28, 2023 at 3:01 AM torikoshia > > wrote: > >> > >> On 2023-08-26 21:03, James Coleman wrote: > >> > On Fri, Aug 25, 2023 at 7:43 AM James Coleman wrote: > >> >> > >> >> On Thu, Aug

Re: RFC: Logging plan of the running query

2023-09-05 Thread torikoshia
On 2023-08-28 22:47, James Coleman wrote: On Mon, Aug 28, 2023 at 3:01 AM torikoshia wrote: On 2023-08-26 21:03, James Coleman wrote: > On Fri, Aug 25, 2023 at 7:43 AM James Coleman wrote: >> >> On Thu, Aug 17, 2023 at 10:02 AM torikoshia >> wrote: >> > >> > On 2023-06-16 01:34, James

Re: RFC: Logging plan of the running query

2023-08-28 Thread James Coleman
On Mon, Aug 28, 2023 at 3:01 AM torikoshia wrote: > > On 2023-08-26 21:03, James Coleman wrote: > > On Fri, Aug 25, 2023 at 7:43 AM James Coleman wrote: > >> > >> On Thu, Aug 17, 2023 at 10:02 AM torikoshia > >> wrote: > >> > > >> > On 2023-06-16 01:34, James Coleman wrote: > >> > > Attached is

Re: RFC: Logging plan of the running query

2023-08-28 Thread torikoshia
On 2023-08-26 21:03, James Coleman wrote: On Fri, Aug 25, 2023 at 7:43 AM James Coleman wrote: On Thu, Aug 17, 2023 at 10:02 AM torikoshia wrote: > > On 2023-06-16 01:34, James Coleman wrote: > > Attached is v28 > > which sets ProcessLogQueryPlanInterruptActive to false in errfinish > >

Re: RFC: Logging plan of the running query

2023-08-26 Thread James Coleman
On Fri, Aug 25, 2023 at 7:43 AM James Coleman wrote: > > On Thu, Aug 17, 2023 at 10:02 AM torikoshia > wrote: > > > > On 2023-06-16 01:34, James Coleman wrote: > > > Attached is v28 > > > which sets ProcessLogQueryPlanInterruptActive to false in errfinish > > > when necessary. Once built with

Re: RFC: Logging plan of the running query

2023-08-25 Thread James Coleman
On Thu, Aug 17, 2023 at 10:02 AM torikoshia wrote: > > On 2023-06-16 01:34, James Coleman wrote: > > Attached is v28 > > which sets ProcessLogQueryPlanInterruptActive to false in errfinish > > when necessary. Once built with those two patches I'm simply running > > `make check`. > > With v28-0001

Re: RFC: Logging plan of the running query

2023-08-17 Thread torikoshia
On 2023-06-16 01:34, James Coleman wrote: Attached is v28 which sets ProcessLogQueryPlanInterruptActive to false in errfinish when necessary. Once built with those two patches I'm simply running `make check`. With v28-0001 and v28-0002 patch, I confirmed backend processes consume huge amount

Re: RFC: Logging plan of the running query

2023-06-27 Thread torikoshia
On 2023-06-16 01:34, James Coleman wrote: On Thu, Jun 15, 2023 at 9:00 AM torikoshia wrote: On 2023-06-15 01:48, James Coleman wrote: > On Tue, Jun 13, 2023 at 11:53 AM James Coleman > wrote: >> >> ... >> I'm going to re-run tests with my patch version + resetting the flag >> on SIGINT (and

Re: RFC: Logging plan of the running query

2023-06-15 Thread James Coleman
On Thu, Jun 15, 2023 at 9:00 AM torikoshia wrote: > > On 2023-06-15 01:48, James Coleman wrote: > > On Tue, Jun 13, 2023 at 11:53 AM James Coleman > > wrote: > >> > >> ... > >> I'm going to re-run tests with my patch version + resetting the flag > >> on SIGINT (and any other error condition) to

Re: RFC: Logging plan of the running query

2023-06-15 Thread torikoshia
On 2023-06-15 01:48, James Coleman wrote: On Tue, Jun 13, 2023 at 11:53 AM James Coleman wrote: ... I'm going to re-run tests with my patch version + resetting the flag on SIGINT (and any other error condition) to be certain that the issue you uncovered (where backends get stuck after a

Re: RFC: Logging plan of the running query

2023-06-14 Thread James Coleman
On Tue, Jun 13, 2023 at 11:53 AM James Coleman wrote: > > ... > I'm going to re-run tests with my patch version + resetting the flag > on SIGINT (and any other error condition) to be certain that the issue > you uncovered (where backends get stuck after a SIGINT not responding > to the requested

Re: RFC: Logging plan of the running query

2023-06-13 Thread James Coleman
On Tue, Jun 13, 2023 at 11:22 AM torikoshia wrote: > > On 2023-06-13 00:52, James Coleman wrote: > >> > >> > I've attached v27. The important change here in 0001 is that it > >> > guarantees the interrupt handler is re-entrant, since that was a bug > >> > exposed by my testing. I've also included

Re: RFC: Logging plan of the running query

2023-06-13 Thread torikoshia
On 2023-06-13 00:52, James Coleman wrote: > I've attached v27. The important change here in 0001 is that it > guarantees the interrupt handler is re-entrant, since that was a bug > exposed by my testing. I've also included 0002 which is only meant for > testing (it attempts to log in the plan

Re: RFC: Logging plan of the running query

2023-06-12 Thread James Coleman
On Sun, Jun 11, 2023 at 11:07 PM torikoshia wrote: > > On 2023-06-06 03:26, James Coleman wrote: > > On Mon, Jun 5, 2023 at 4:30 AM torikoshia > > wrote: > >> > >> On 2023-06-03 02:51, James Coleman wrote: > >> > Hello, > >> > > >> > Thanks for working on this patch! > > > > Sure thing! I'm

Re: RFC: Logging plan of the running query

2023-06-11 Thread torikoshia
On 2023-06-06 03:26, James Coleman wrote: On Mon, Jun 5, 2023 at 4:30 AM torikoshia wrote: On 2023-06-03 02:51, James Coleman wrote: > Hello, > > Thanks for working on this patch! Sure thing! I'm *very interested* in seeing this available, and I think it paves the way for some additional

Re: RFC: Logging plan of the running query

2023-06-05 Thread James Coleman
On Mon, Jun 5, 2023 at 4:30 AM torikoshia wrote: > > On 2023-06-03 02:51, James Coleman wrote: > > Hello, > > > > Thanks for working on this patch! Sure thing! I'm *very interested* in seeing this available, and I think it paves the way for some additional features later on... > > On Thu, Dec

Re: RFC: Logging plan of the running query

2023-06-05 Thread torikoshia
On 2023-06-03 02:51, James Coleman wrote: Hello, Thanks for working on this patch! On Thu, Dec 8, 2022 at 12:10 AM torikoshia wrote: BTW, since this patch depends on ProcessInterrupts() and EXPLAIN codes which is used in auto_explain, I'm feeling that the following discussion also

Re: RFC: Logging plan of the running query

2023-06-02 Thread James Coleman
Hello, Thanks for working on this patch! On Thu, Dec 8, 2022 at 12:10 AM torikoshia wrote: > > BTW, since this patch depends on ProcessInterrupts() and EXPLAIN codes > which is used in auto_explain, I'm feeling that the following discussion > also applies to this patch. > > > -- > >

Re: RFC: Logging plan of the running query

2022-12-07 Thread torikoshia
On 2022-12-07 03:41, Andres Freund wrote: Hi, This patch does not currently build, due to a conflicting oid: I suggest you choose a random oid out of the "development purposes" range: Thanks for your advice! Attached updated patch. BTW, since this patch depends on ProcessInterrupts() and

Re: RFC: Logging plan of the running query

2022-12-06 Thread Andres Freund
Hi, This patch does not currently build, due to a conflicting oid: https://cirrus-ci.com/task/4638460594618368?logs=build#L108 [17:26:44.602] /usr/bin/perl ../src/include/catalog/../../backend/catalog/genbki.pl --include-path=../src/include --set-version=16 --output=src/include/catalog

Re: RFC: Logging plan of the running query

2022-09-23 Thread Alena Rybakina
Sorry, I wrote confusingly at that time. No, I suggested adding comment about the explanation of HandleLogQueryPlanInterrupt() only in the explain.h and not removing from the explain.c. I seemed to be necessary separating declaration function for 'explaining feature' of executed query from

Re: RFC: Logging plan of the running query

2022-09-22 Thread torikoshia
On 2022-09-21 17:30, Alena Rybakina wrote: Thanks for your reply! I also noticed it. However I also discovered that above function declarations to be aplied for explain command and used to be printed details of the executed query. We have a similar task to print the plan of an interrupted

Re: RFC: Logging plan of the running query

2022-09-21 Thread Alena Rybakina
 Ok, I get it. Since GetLockMethodLocalHash() is only used for assertions, this is only defined when USE_ASSERT_CHECKING is enabled. This patch uses GetLockMethodLocalHash() not only for the assertion purpose, so I removed "ifdef USE_ASSERT_CHECKING" for this function. I belive it does not

Re: RFC: Logging plan of the running query

2022-09-20 Thread torikoshia
On 2022-09-19 17:47, Алена Рыбакина wrote: Thanks for your review and comments! Hi, I'm sorry,if this message is duplicated previous this one, but I'm not sure that the previous message is sent correctly. I sent it from email address a.rybak...@postgrespro.ru and I couldn't send this one email

Re: RFC: Logging plan of the running query

2022-09-19 Thread a.rybakina
Hi, I'm sorry,if this message is duplicated previous this one, but the previous message is sent incorrectly. I sent it from email address lena.riback...@yandex.ru. I liked this idea and after reviewing code I noticed some moments and I'd rather ask you some questions. Firstly, I suggest

Re: RFC: Logging plan of the running query

2022-09-19 Thread Алена Рыбакина
Hi,I'm sorry,if this message is duplicated previous this one, but I'm not sure that the previous message is sent correctly. I sent it from email address a.rybak...@postgrespro.ru and I couldn't send this one email from those address.I like idea to create patch for logging query plan. After

RFC: Logging plan of the running query

2022-09-16 Thread a.rybakina
Hi, I liked this idea and after reviewing code I noticed some moments and I'd rather ask you some questions. Firstly, I suggest some editing in the comment of commit. I think, it is turned out the more laconic and the same clear. I wrote it below since I can't think of any other way to add

Re: RFC: Logging plan of the running query

2022-09-08 Thread torikoshia
On 2022-05-16 17:02, torikoshia wrote: On 2022-03-09 19:04, torikoshia wrote: On 2022-02-08 01:13, Fujii Masao wrote: AbortSubTransaction() should reset ActiveQueryDesc to save_ActiveQueryDesc that ExecutorRun() set, instead of NULL? Otherwise ActiveQueryDesc of top-level statement will be

  1   2   >