Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-12-24 3:23 GMT+01:00 Michael Paquier : > On Thu, Dec 17, 2015 at 6:45 PM, Shulgin, Oleksandr > wrote: > > On Wed, Dec 16, 2015 at 8:39 PM, Tomas Vondra < > tomas.von...@2ndquadrant.com> > > wrote: > >> On 12/01/2015 10:34 AM, Shulgin,

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Thu, Dec 17, 2015 at 6:45 PM, Shulgin, Oleksandr wrote: > On Wed, Dec 16, 2015 at 8:39 PM, Tomas Vondra > wrote: >> On 12/01/2015 10:34 AM, Shulgin, Oleksandr wrote: >>> >>> >>> I have the plans to make something from this on top of

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Thu, Dec 24, 2015 at 1:57 PM, Pavel Stehule wrote: > > > 2015-12-24 3:23 GMT+01:00 Michael Paquier : >> >> On Thu, Dec 17, 2015 at 6:45 PM, Shulgin, Oleksandr >> wrote: >> > On Wed, Dec 16, 2015 at 8:39 PM,

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Wed, Dec 16, 2015 at 8:39 PM, Tomas Vondra wrote: > Hi, > > On 12/01/2015 10:34 AM, Shulgin, Oleksandr wrote: > >> >> I have the plans to make something from this on top of >> pg_stat_statements and auto_explain, as I've mentioned last time. The >> next

Re: [HACKERS] On-demand running query plans using auto_explain and signals

Hi, On 12/01/2015 10:34 AM, Shulgin, Oleksandr wrote: I have the plans to make something from this on top of pg_stat_statements and auto_explain, as I've mentioned last time. The next iteration will be based on the two latest patches above, so it still makes sense to review them. As for

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Tue, Dec 1, 2015 at 12:04 AM, Simon Riggs wrote: > On 30 November 2015 at 22:27, Julien Rouhaud > wrote: > > >> I registered as reviewer on this, but after reading the whole thread for >> the second time, it's still not clear to me if the

Re: [HACKERS] On-demand running query plans using auto_explain and signals

Hello, On 15/10/2015 16:04, Pavel Stehule wrote: > > > 2015-10-15 15:42 GMT+02:00 Tom Lane >: > > "Shulgin, Oleksandr" > writes: > > I was thinking about this and

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On 30 November 2015 at 22:27, Julien Rouhaud wrote: > I registered as reviewer on this, but after reading the whole thread for > the second time, it's still not clear to me if the last two submitted > patches (0001-Add-auto_explain.publish_plans.patch and >

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Tue, Sep 29, 2015 at 7:52 PM, Shulgin, Oleksandr < oleksandr.shul...@zalando.de> wrote: > > This is not a change of the direction, but rather of the approach. > Hitting a process with a signal and hoping it will produce a meaningful > response in all circumstances without disrupting its

Re: [HACKERS] On-demand running query plans using auto_explain and signals

"Shulgin, Oleksandr" writes: > I was thinking about this and what seems to be the biggest problem is when > to actually turn the feature on. It seems unlikely that someone will want > to enable it unconditionally. Enabling per-backend also doesn't seem to be > a

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-10-15 15:42 GMT+02:00 Tom Lane : > "Shulgin, Oleksandr" writes: > > I was thinking about this and what seems to be the biggest problem is > when > > to actually turn the feature on. It seems unlikely that someone will > want > > to enable

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Tue, Sep 29, 2015 at 1:52 PM, Shulgin, Oleksandr wrote: > This is not a change of the direction, but rather of the approach. Hitting > a process with a signal and hoping it will produce a meaningful response in > all circumstances without disrupting its current

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Tue, Sep 29, 2015 at 12:28 AM, Jim Nasby wrote: > On 9/18/15 5:05 AM, Shulgin, Oleksandr wrote: > >> >> So this has to be the responsibility of the reply sending backend in the >> end: to create and release the DSM *at some point*. >> > > What's wrong with just

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Tue, Sep 29, 2015 at 12:57 PM, Andres Freund wrote: > On 2015-09-25 19:13:13 +0200, Shulgin, Oleksandr wrote: > > the auto_explain contrib module. I now propose the most simple thing > > possible in my opinion: a new GUC option for auto_explain. It will make > > every

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On 2015-09-25 19:13:13 +0200, Shulgin, Oleksandr wrote: > the auto_explain contrib module. I now propose the most simple thing > possible in my opinion: a new GUC option for auto_explain. It will make > every backend, in which the module is loaded via *_preload_libraries > mechanism or using

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On 25 September 2015 at 12:13, Shulgin, Oleksandr < oleksandr.shul...@zalando.de> wrote: > I now believe that use of ProcessInterrupts() in the recently proposed > design as well as manipulation of proc latch due to use of shared memory > queue are major blockers. > > In order to simplify things

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On 29 September 2015 at 20:52, Shulgin, Oleksandr < oleksandr.shul...@zalando.de> wrote: > On Tue, Sep 29, 2015 at 8:34 PM, Simon Riggs > wrote: > >> On 29 September 2015 at 12:52, Shulgin, Oleksandr < >> oleksandr.shul...@zalando.de> wrote: >> >> >>> Hitting a process

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Tue, Sep 29, 2015 at 7:01 PM, Simon Riggs wrote: > On 25 September 2015 at 12:13, Shulgin, Oleksandr < > oleksandr.shul...@zalando.de> wrote: > > >> I now believe that use of ProcessInterrupts() in the recently proposed >> design as well as manipulation of proc latch

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On 29 September 2015 at 12:52, Shulgin, Oleksandr < oleksandr.shul...@zalando.de> wrote: > Hitting a process with a signal and hoping it will produce a meaningful > response in all circumstances without disrupting its current task was way > too naive. > Hmm, I would have to disagree, sorry. For

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Tue, Sep 29, 2015 at 8:34 PM, Simon Riggs wrote: > On 29 September 2015 at 12:52, Shulgin, Oleksandr < > oleksandr.shul...@zalando.de> wrote: > > >> Hitting a process with a signal and hoping it will produce a meaningful >> response in all circumstances without

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Mon, Sep 28, 2015 at 7:09 PM, Pavel Stehule wrote: > > 2015-09-28 12:37 GMT+02:00 Shulgin, Oleksandr < > oleksandr.shul...@zalando.de>: > >> >>> I didn't propose too different solution. There is only one difference - >>> sharing DSM for smaller data. It is similar to

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-28 12:37 GMT+02:00 Shulgin, Oleksandr : > On Mon, Sep 28, 2015 at 12:05 PM, Pavel Stehule > wrote: > >> >> 2015-09-28 12:01 GMT+02:00 Shulgin, Oleksandr < >> oleksandr.shul...@zalando.de>: >> >>> On Sun, Sep 27, 2015 at 8:05 AM,

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-28 19:13 GMT+02:00 Shulgin, Oleksandr : > On Mon, Sep 28, 2015 at 7:09 PM, Pavel Stehule > wrote: > >> >> 2015-09-28 12:37 GMT+02:00 Shulgin, Oleksandr < >> oleksandr.shul...@zalando.de>: >> >>> I didn't propose too different

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Fri, Sep 25, 2015 at 7:13 PM, Shulgin, Oleksandr < oleksandr.shul...@zalando.de> wrote: > > Some implementation details: > > For every backend that might be running (MaxBackends) we reserve a > dsm_handle slot in the addins shared memory. When the new option is turned > on, the ExecutorRun

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Mon, Sep 28, 2015 at 12:05 PM, Pavel Stehule wrote: > > 2015-09-28 12:01 GMT+02:00 Shulgin, Oleksandr < > oleksandr.shul...@zalando.de>: > >> On Sun, Sep 27, 2015 at 8:05 AM, Pavel Stehule >> wrote: >> >> the preparing of content before

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Sun, Sep 27, 2015 at 8:05 AM, Pavel Stehule wrote: the preparing of content before execution is interesting idea, that can be > used more. The almost queries and plans are not too big, so when the size > of content is not too big - less than 1MB, then can be used one

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-28 12:01 GMT+02:00 Shulgin, Oleksandr : > On Sun, Sep 27, 2015 at 8:05 AM, Pavel Stehule > wrote: > > the preparing of content before execution is interesting idea, that can be >> used more. The almost queries and plans are not too

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On 9/18/15 5:05 AM, Shulgin, Oleksandr wrote: It should not be true - the data sender create DSM and fills it. Then set caller slot and send signal to caller. Caller can free DSM any time, because data sender send newer touch it. But the requester can timeout on waiting for reply

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-25 19:13 GMT+02:00 Shulgin, Oleksandr : > > > Some problems: > > There is a potential problem with the limited total number of DSM > segments: it is reserved in a way to only allow 2 per backend (on average) > and 64 additional per server, so if you run with

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Fri, Sep 18, 2015 at 7:04 PM, Robert Haas wrote: > > Frankly, I think you guys are making this out to be way more > complicated than it really is. Basically, I think the process being > queried should publish a DSM via a slot it owns. The recipient is > responsible

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-18 10:59 GMT+02:00 Shulgin, Oleksandr : > On Thu, Sep 17, 2015 at 10:13 PM, Robert Haas > wrote: > >> On Thu, Sep 17, 2015 at 11:16 AM, Pavel Stehule >> wrote: >> >> >> Second, using a shm_mq manipulates the

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Thu, Sep 17, 2015 at 10:13 PM, Robert Haas wrote: > On Thu, Sep 17, 2015 at 11:16 AM, Pavel Stehule > wrote: > > >> Second, using a shm_mq manipulates the state of the process latch. I > >> don't think you can make the assumption that it's

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Fri, Sep 18, 2015 at 11:25 AM, Pavel Stehule wrote: > 2015-09-18 10:59 GMT+02:00 Shulgin, Oleksandr < > oleksandr.shul...@zalando.de>: > >> >> If we take the per-backend slot approach the locking seems unnecessary >> and there are principally two options: >> >> 1) The

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Fri, Sep 18, 2015 at 12:59 PM, Pavel Stehule wrote: > 2015-09-18 12:05 GMT+02:00 Shulgin, Oleksandr < > oleksandr.shul...@zalando.de>: > >> On Fri, Sep 18, 2015 at 11:25 AM, Pavel Stehule >> wrote: >> >>> >>> It should not be true - the data

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-18 12:05 GMT+02:00 Shulgin, Oleksandr : > On Fri, Sep 18, 2015 at 11:25 AM, Pavel Stehule > wrote: > >> 2015-09-18 10:59 GMT+02:00 Shulgin, Oleksandr < >> oleksandr.shul...@zalando.de>: >> >>> >>> If we take the per-backend slot

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-18 13:22 GMT+02:00 Shulgin, Oleksandr : > On Fri, Sep 18, 2015 at 12:59 PM, Pavel Stehule > wrote: > >> 2015-09-18 12:05 GMT+02:00 Shulgin, Oleksandr < >> oleksandr.shul...@zalando.de>: >> >>> On Fri, Sep 18, 2015 at 11:25 AM, Pavel

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Thu, Sep 17, 2015 at 12:47 PM, Shulgin, Oleksandr wrote: > But if we make the sender backend create the DSM with the response payload, > it suddenly becomes really unclear at which point and who should make the > final detach of that DSM. We're getting back to

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Fri, Sep 18, 2015 at 6:59 AM, Pavel Stehule wrote: > I am afraid so it has not simple and nice solution - when data sender will > wait for to moment when data are received, then we have same complexity like > we use shm_mq. > > Isn't better to introduce new background

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Wed, Sep 16, 2015 at 8:07 PM, Pavel Stehule wrote: > > 2015-09-16 16:31 GMT+02:00 Shulgin, Oleksandr < > oleksandr.shul...@zalando.de>: > >> >> I've added the timeout parameter to the pg_cmdstatus call, and more >> importantly to the sending side of the queue, so that

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-17 11:55 GMT+02:00 Shulgin, Oleksandr : > On Wed, Sep 16, 2015 at 8:07 PM, Pavel Stehule > wrote: > >> >> 2015-09-16 16:31 GMT+02:00 Shulgin, Oleksandr < >> oleksandr.shul...@zalando.de>: >> >>> >>> I've added the timeout parameter

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Tue, Sep 15, 2015 at 5:00 AM, Shulgin, Oleksandr wrote: > Please see attached for implementation of this approach. The most > surprising thing is that it actually works :) It's cool to see these interesting ideas for using some of the code I've been working on

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Wed, Sep 16, 2015 at 10:31 AM, Shulgin, Oleksandr wrote: > I've also decided we really ought to suppress any possible ERROR level > messages generated during the course of processing the status request in > order not to prevent the originally running transaction

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Thu, Sep 17, 2015 at 2:14 PM, Robert Haas wrote: > On Wed, Sep 16, 2015 at 10:31 AM, Shulgin, Oleksandr > wrote: > > I've also decided we really ought to suppress any possible ERROR level > > messages generated during the course of

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-17 16:46 GMT+02:00 Robert Haas : > On Thu, Sep 17, 2015 at 10:28 AM, Pavel Stehule > wrote: > > Please, can you explain what is wrong on using of shm_mq? It works pretty > > well now. > > > > There can be a contra argument, why don't use

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-17 16:06 GMT+02:00 Shulgin, Oleksandr : > On Thu, Sep 17, 2015 at 12:06 PM, Pavel Stehule > wrote: > >> >>> That won't work really well with something like I use to do when testing >>> this patch, namely: >>> >>> postgres=# select

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-17 14:06 GMT+02:00 Robert Haas : > On Tue, Sep 15, 2015 at 5:00 AM, Shulgin, Oleksandr > wrote: > > Please see attached for implementation of this approach. The most > > surprising thing is that it actually works :) > > It's cool to

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Thu, Sep 17, 2015 at 12:06 PM, Pavel Stehule wrote: > >> That won't work really well with something like I use to do when testing >> this patch, namely: >> >> postgres=# select pid, array(select pg_cmdstatus(pid, 1, 10)) from >> pg_stat_activity where

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Thu, Sep 17, 2015 at 5:16 PM, Pavel Stehule wrote: > 2015-09-17 16:46 GMT+02:00 Robert Haas : > >> >> Second, using a shm_mq manipulates the state of the process latch. I >> don't think you can make the assumption that it's safe to reset the >>

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Thu, Sep 17, 2015 at 11:16 AM, Pavel Stehule wrote: > Is there some risk if we take too much large DSM segment? And what is max > size of DSM segment? When we use shm_mq, we don't need to solve limits. I can't really think you are going to have a problem. How much

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-17 16:37 GMT+02:00 Pavel Stehule : > > > 2015-09-17 16:06 GMT+02:00 Shulgin, Oleksandr < > oleksandr.shul...@zalando.de>: > >> On Thu, Sep 17, 2015 at 12:06 PM, Pavel Stehule >> wrote: >> >>> That won't work really well with

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-17 22:13 GMT+02:00 Robert Haas : > On Thu, Sep 17, 2015 at 11:16 AM, Pavel Stehule > wrote: > > Is there some risk if we take too much large DSM segment? And what is max > > size of DSM segment? When we use shm_mq, we don't need to solve

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Tue, Sep 15, 2015 at 11:00 AM, Shulgin, Oleksandr < oleksandr.shul...@zalando.de> wrote: > On Mon, Sep 14, 2015 at 7:27 PM, Pavel Stehule > wrote: > >> >> 2015-09-14 18:46 GMT+02:00 Shulgin, Oleksandr < >> oleksandr.shul...@zalando.de>: >> >>> >>> ... This way the

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-16 16:31 GMT+02:00 Shulgin, Oleksandr : > On Tue, Sep 15, 2015 at 11:00 AM, Shulgin, Oleksandr < > oleksandr.shul...@zalando.de> wrote: > >> On Mon, Sep 14, 2015 at 7:27 PM, Pavel Stehule >> wrote: >> >>> >>> 2015-09-14 18:46

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Mon, Sep 14, 2015 at 7:27 PM, Pavel Stehule wrote: > > 2015-09-14 18:46 GMT+02:00 Shulgin, Oleksandr < > oleksandr.shul...@zalando.de>: > >> >> I have a radical proposal to remove the need for locking: make the >> CmdStatusSlot struct consist of a mere dsm_handle and

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-14 14:11 GMT+02:00 Tomas Vondra : > > > On 09/14/2015 01:15 PM, Shulgin, Oleksandr wrote: > >> On Mon, Sep 14, 2015 at 11:53 AM, Tomas Vondra >> > >> wrote: >> >> >> On 09/14/2015 10:23

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Mon, Sep 14, 2015 at 11:53 AM, Tomas Vondra wrote: > > On 09/14/2015 10:23 AM, Shulgin, Oleksandr wrote: > >> On Sat, Sep 12, 2015 at 11:50 AM, Tomas Vondra >> > >> wrote: >> > ... > >> -

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Mon, Sep 14, 2015 at 2:11 PM, Tomas Vondra wrote: > >> Now the backend that has been signaled on the second call to >> pg_cmdstatus (it can be either some other backend, or the backend B >> again) will not find an unprocessed slot, thus it will not try to >>

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On 09/14/2015 01:15 PM, Shulgin, Oleksandr wrote: On Mon, Sep 14, 2015 at 11:53 AM, Tomas Vondra > wrote: On 09/14/2015 10:23 AM, Shulgin, Oleksandr wrote: On Sat, Sep 12, 2015 at 11:50 AM, Tomas Vondra

Re: [HACKERS] On-demand running query plans using auto_explain and signals

> > Well, I didn't attach the updated patch (doing that now). > This time for real. Sorry, it's Monday :-p diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c index 32ac58f..2e3beaf 100644 --- a/src/backend/storage/ipc/ipci.c +++ b/src/backend/storage/ipc/ipci.c @@ -43,6

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-14 18:46 GMT+02:00 Shulgin, Oleksandr : > On Mon, Sep 14, 2015 at 3:09 PM, Shulgin, Oleksandr < > oleksandr.shul...@zalando.de> wrote: > >> On Mon, Sep 14, 2015 at 2:11 PM, Tomas Vondra < >> tomas.von...@2ndquadrant.com> wrote: >> >>> Now the backend

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Mon, Sep 14, 2015 at 3:09 PM, Shulgin, Oleksandr < oleksandr.shul...@zalando.de> wrote: > On Mon, Sep 14, 2015 at 2:11 PM, Tomas Vondra < > tomas.von...@2ndquadrant.com> wrote: > >> >>> Now the backend that has been signaled on the second call to >>> pg_cmdstatus (it can be either some other

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Sat, Sep 12, 2015 at 11:50 AM, Tomas Vondra wrote: > Hi, > > I did a quick initial review of this patch today, so here are my comments > so far: > Hi Tomas, First of all, thanks for the review! - ipcs.c should include utils/cmdstatus.h (the compiler complains

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-14 10:23 GMT+02:00 Shulgin, Oleksandr : > > > > I can think of something like: > > EXPLAIN [ ( option [, ...] ) ] PROCESS ; > > where option is extended with: > > QUERY > PROGRESS > BACKTRACE > > in addition to the usual ANALYZE, VERBOSE, FORMAT, etc.

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On 09/14/2015 10:23 AM, Shulgin, Oleksandr wrote: On Sat, Sep 12, 2015 at 11:50 AM, Tomas Vondra > wrote: ... - Attempts to get plan for simple insert queries like this INSERT INTO x SELECT * FROM x;

Re: [HACKERS] On-demand running query plans using auto_explain and signals

Hi Thank you for precious check. 2015-09-12 11:50 GMT+02:00 Tomas Vondra : > Hi, > > I did a quick initial review of this patch today, so here are my comments > so far: > > - ipcs.c should include utils/cmdstatus.h (the compiler complains > about implicit

Re: [HACKERS] On-demand running query plans using auto_explain and signals

Hi, I did a quick initial review of this patch today, so here are my comments so far: - ipcs.c should include utils/cmdstatus.h (the compiler complains about implicit declaration of two functions) - Attempts to get plan for simple insert queries like this INSERT INTO x SELECT * FROM

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-08 18:53 GMT+02:00 Shulgin, Oleksandr : > On Tue, Sep 8, 2015 at 11:49 AM, Shulgin, Oleksandr < > oleksandr.shul...@zalando.de> wrote: > >> >> >> The real problem could be if the process that was signaled to connect >> to the message queue never handles the

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Wed, Sep 9, 2015 at 8:36 AM, Pavel Stehule wrote: > >> Please find attached v4. >> > > It is better > One important thing to notice, and probably deserves a code comment, that any modification of the slot fields done by the info sender backend must be done before

Re: [HACKERS] On-demand running query plans using auto_explain and signals

Two notices: > > 1. The communication mechanism can be used more wide, than only for this > purpose. We can introduce a SendInfoHook - and it can be used for any > customer probes - memory, cpu, ... > Not sure if for CPU you can get any more insight than an external tool like top(1) will provide.

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Wed, Sep 9, 2015 at 10:22 AM, Shulgin, Oleksandr < oleksandr.shul...@zalando.de> wrote: > On Wed, Sep 9, 2015 at 8:36 AM, Pavel Stehule > wrote: > >> >>> Please find attached v4. >>> >> >> It is better >> > > One important thing to notice, and probably deserves a code

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Tue, Sep 8, 2015 at 6:58 AM, Pavel Stehule wrote: >> >> But you will still lock on the slots list to find an unused one. How is that substantially different from what I'm doing? > > It is not necessary - you can use similar technique to what it does PGPROC. I am

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Tue, Sep 8, 2015 at 11:49 AM, Shulgin, Oleksandr < oleksandr.shul...@zalando.de> wrote: > > >> The real problem could be if the process that was signaled to connect > to the message queue never handles the interrupt, and we keep waiting > forever in shm_mq_receive(). We could add a timeout

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Fri, Sep 4, 2015 at 6:11 AM, Pavel Stehule wrote: > Sorry, but I still don't see how the slots help this issue - could you >> please elaborate? >> > with slot (or some similiar) there is not global locked resource. If I'll > have a time at weekend I'll try to write

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-07 11:55 GMT+02:00 Shulgin, Oleksandr : > On Fri, Sep 4, 2015 at 6:11 AM, Pavel Stehule > wrote: > >> Sorry, but I still don't see how the slots help this issue - could you >>> please elaborate? >>> >> with slot (or some similiar)

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Wed, Sep 2, 2015 at 3:07 PM, Shulgin, Oleksandr < oleksandr.shul...@zalando.de> wrote: > On Wed, Sep 2, 2015 at 3:04 PM, Pavel Stehule > wrote: >> >> >>> Well, maybe I'm missing something, but sh_mq_create() will just >>> overwrite the contents of the struct, so it

Re: [HACKERS] On-demand running query plans using auto_explain and signals

Hi 2015-09-03 18:30 GMT+02:00 Shulgin, Oleksandr : > On Wed, Sep 2, 2015 at 3:07 PM, Shulgin, Oleksandr < > oleksandr.shul...@zalando.de> wrote: > >> On Wed, Sep 2, 2015 at 3:04 PM, Pavel Stehule >> wrote: >>> >>> Well, maybe I'm

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-03 22:06 GMT+02:00 Pavel Stehule : > Hi > > > > > 2015-09-03 18:30 GMT+02:00 Shulgin, Oleksandr < > oleksandr.shul...@zalando.de>: > >> On Wed, Sep 2, 2015 at 3:07 PM, Shulgin, Oleksandr < >> oleksandr.shul...@zalando.de> wrote: >> >>> On Wed, Sep 2, 2015 at 3:04

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Sep 3, 2015 10:14 PM, "Pavel Stehule" wrote: >>> >>> Please find attached a v3. >>> >>> It uses a shared memory queue and also has the ability to capture plans nested deeply in the call stack. Not sure about using the executor hook, since this is not an extension...

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-04 5:50 GMT+02:00 Shulgin, Oleksandr : > On Sep 3, 2015 10:14 PM, "Pavel Stehule" wrote: > >>> > >>> Please find attached a v3. > >>> > >>> It uses a shared memory queue and also has the ability to capture > plans nested deeply in

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Tue, Sep 1, 2015 at 7:02 PM, Pavel Stehule wrote: > >> But do we really need the slots mechanism? Would it not be OK to just >> let the LWLock do the sequencing of concurrent requests? Given that we >> only going to use one message queue per cluster, there's not

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-02 11:01 GMT+02:00 Shulgin, Oleksandr : > On Tue, Sep 1, 2015 at 7:02 PM, Pavel Stehule > wrote: > >> >>> But do we really need the slots mechanism? Would it not be OK to just >>> let the LWLock do the sequencing of concurrent

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Wed, Sep 2, 2015 at 11:16 AM, Pavel Stehule wrote: > > > 2015-09-02 11:01 GMT+02:00 Shulgin, Oleksandr < > oleksandr.shul...@zalando.de>: > >> On Tue, Sep 1, 2015 at 7:02 PM, Pavel Stehule >> wrote: >> >>> But do we really need the slots

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Wed, Sep 2, 2015 at 2:56 PM, Pavel Stehule wrote: > > > 2015-09-02 12:36 GMT+02:00 Shulgin, Oleksandr < > oleksandr.shul...@zalando.de>: > >> On Wed, Sep 2, 2015 at 11:16 AM, Pavel Stehule >> wrote: >> >>> >>> >>> 2015-09-02 11:01 GMT+02:00

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-02 15:00 GMT+02:00 Shulgin, Oleksandr : > On Wed, Sep 2, 2015 at 2:56 PM, Pavel Stehule > wrote: > >> >> >> 2015-09-02 12:36 GMT+02:00 Shulgin, Oleksandr < >> oleksandr.shul...@zalando.de>: >> >>> On Wed, Sep 2, 2015 at 11:16 AM,

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Wed, Sep 2, 2015 at 3:04 PM, Pavel Stehule wrote: > > >> Well, maybe I'm missing something, but sh_mq_create() will just overwrite >> the contents of the struct, so it doesn't care about sender/receiver: only >> sh_mq_set_sender/receiver() do. >> > > if you create

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-02 12:36 GMT+02:00 Shulgin, Oleksandr : > On Wed, Sep 2, 2015 at 11:16 AM, Pavel Stehule > wrote: > >> >> >> 2015-09-02 11:01 GMT+02:00 Shulgin, Oleksandr < >> oleksandr.shul...@zalando.de>: >> >>> On Tue, Sep 1, 2015 at 7:02 PM,

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-01 17:20 GMT+02:00 Shulgin, Oleksandr : > I'm not familiar with the shared memory handling, but could we not >>> allocate just enough shared memory to fit the data we're going to write >>> instead of the fixed 8k? It's not that we cannot know the length of

Re: [HACKERS] On-demand running query plans using auto_explain and signals

> > I'd say we should hide the so-designed pg_cmdstatus() interface behind >> more friendly calls like pg_explain_backend() and pg_backend_progress() to >> give some naming examples, to remove the need for magic numbers in the >> second arg. >> > > I had similar idea - this is good enough for

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-01 15:00 GMT+02:00 Shulgin, Oleksandr : > I'd say we should hide the so-designed pg_cmdstatus() interface behind >>> more friendly calls like pg_explain_backend() and pg_backend_progress() to >>> give some naming examples, to remove the need for magic

Re: [HACKERS] On-demand running query plans using auto_explain and signals

> > I'm not familiar with the shared memory handling, but could we not >> allocate just enough shared memory to fit the data we're going to write >> instead of the fixed 8k? It's not that we cannot know the length of the >> resulting plan text in advance. >> > > the shared memory cannot be reused

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Mon, Aug 31, 2015 at 12:35 PM, Pavel Stehule wrote: > >>> >>> http://www.postgresql.org/message-id/cafj8praxcs9b8abgim-zauvggqdhpzoarz5ysp1_nhv9hp8...@mail.gmail.com >>> >> >> Ah, thanks! Somehow I've missed this mail. You didn't add the patch to >> a commitfest

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-08-31 11:30 GMT+02:00 Shulgin, Oleksandr : > Do you still have the code somewhere around? Did it see production use? >>> I sent it to mailing list year ago >> >> >>

Re: [HACKERS] On-demand running query plans using auto_explain and signals

> > Do you still have the code somewhere around? Did it see production use? >>> >> I sent it to mailing list year ago > > > http://www.postgresql.org/message-id/cafj8praxcs9b8abgim-zauvggqdhpzoarz5ysp1_nhv9hp8...@mail.gmail.com > Ah, thanks! Somehow I've missed this mail. You didn't add the

Re: [HACKERS] On-demand running query plans using auto_explain and signals

Hi 2015-08-31 19:09 GMT+02:00 Shulgin, Oleksandr : > On Mon, Aug 31, 2015 at 12:35 PM, Pavel Stehule > wrote: > >> http://www.postgresql.org/message-id/cafj8praxcs9b8abgim-zauvggqdhpzoarz5ysp1_nhv9hp8...@mail.gmail.com >>>

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Aug 29, 2015 7:31 PM, Pavel Stehule pavel.steh...@gmail.com wrote: 2015-08-29 18:36 GMT+02:00 Andres Freund and...@anarazel.de: On 2015-08-29 18:27:59 +0200, Pavel Stehule wrote: 2015-08-29 18:25 GMT+02:00 Shulgin, Oleksandr oleksandr.shul...@zalando.de Good point. There's still

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-08-30 10:30 GMT+02:00 Shulgin, Oleksandr oleksandr.shul...@zalando.de : On Aug 29, 2015 7:31 PM, Pavel Stehule pavel.steh...@gmail.com wrote: 2015-08-29 18:36 GMT+02:00 Andres Freund and...@anarazel.de: On 2015-08-29 18:27:59 +0200, Pavel Stehule wrote: 2015-08-29 18:25

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-08-30 10:34 GMT+02:00 Pavel Stehule pavel.steh...@gmail.com: 2015-08-30 10:30 GMT+02:00 Shulgin, Oleksandr oleksandr.shul...@zalando.de: On Aug 29, 2015 7:31 PM, Pavel Stehule pavel.steh...@gmail.com wrote: 2015-08-29 18:36 GMT+02:00 Andres Freund and...@anarazel.de: On

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Sat, Aug 29, 2015 at 5:44 PM, Andres Freund and...@anarazel.de wrote: On 2015-08-29 17:33:22 +0200, Shulgin, Oleksandr wrote: Probably using SIGUSR2 would be more appropriate, but I'm not sure if there are other extensions out there that might be already using it for some other reason

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On 2015-08-29 18:27:59 +0200, Pavel Stehule wrote: 2015-08-29 18:25 GMT+02:00 Shulgin, Oleksandr oleksandr.shul...@zalando.de Good point. There's still hope to set a flag and process it later on. Will have to check if it's possible to stay in the scope of a loaded module though. I had a

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-08-29 18:36 GMT+02:00 Andres Freund and...@anarazel.de: On 2015-08-29 18:27:59 +0200, Pavel Stehule wrote: 2015-08-29 18:25 GMT+02:00 Shulgin, Oleksandr oleksandr.shul...@zalando.de Good point. There's still hope to set a flag and process it later on. Will have to check if it's

  1   2   >