Re: [HACKERS] Tracking wait event for latches

2016-10-05 Thread Michael Paquier
On Wed, Oct 5, 2016 at 9:25 PM, Robert Haas wrote: > On Tue, Oct 4, 2016 at 5:07 PM, Michael Paquier > wrote: >> More seriously, the Windows animals have been complaining about >> pg_sleep() crashing the system: >> SELECT pg_sleep(0.1); >> !

Re: [HACKERS] Tracking wait event for latches

2016-10-05 Thread Robert Haas
On Tue, Oct 4, 2016 at 5:07 PM, Michael Paquier wrote: > More seriously, the Windows animals have been complaining about > pg_sleep() crashing the system: > SELECT pg_sleep(0.1); > ! server closed the connection unexpectedly > ! This probably means the server

Re: [HACKERS] Tracking wait event for latches

2016-10-05 Thread Robert Haas
On Tue, Oct 4, 2016 at 3:28 PM, Thomas Munro wrote: > Nitpicking: the includes in bgworker.c weren't sorted properly, and > then this patch added "pgstat.h" in the wrong position. See attached > suggestion. Committed. -- Robert Haas EnterpriseDB:

Re: [HACKERS] Tracking wait event for latches

2016-10-04 Thread Michael Paquier
On Wed, Oct 5, 2016 at 4:28 AM, Thomas Munro wrote: > On Wed, Oct 5, 2016 at 4:59 AM, Robert Haas wrote: >> On Mon, Oct 3, 2016 at 8:43 PM, Michael Paquier >> wrote: >>> The rest looks good to me. Thanks for the

Re: [HACKERS] Tracking wait event for latches

2016-10-04 Thread Thomas Munro
On Wed, Oct 5, 2016 at 4:59 AM, Robert Haas wrote: > On Mon, Oct 3, 2016 at 8:43 PM, Michael Paquier > wrote: >> The rest looks good to me. Thanks for the feedback and the time! > > Thanks for the fixes. I committed this with an additional

Re: [HACKERS] Tracking wait event for latches

2016-10-04 Thread Robert Haas
On Mon, Oct 3, 2016 at 8:43 PM, Michael Paquier wrote: > The rest looks good to me. Thanks for the feedback and the time! Thanks for the fixes. I committed this with an additional compile fix, but the buildfarm turned up a few more problems that my 'make check-world'

Re: [HACKERS] Tracking wait event for latches

2016-10-03 Thread Michael Paquier
On Tue, Oct 4, 2016 at 1:55 AM, Robert Haas wrote: > On Mon, Oct 3, 2016 at 3:30 AM, Michael Paquier > wrote: >> [ new patch ] > > I think this is unnecessarily awkward for callers; the attached > version takes a different approach which I think

Re: [HACKERS] Tracking wait event for latches

2016-10-03 Thread Robert Haas
On Mon, Oct 3, 2016 at 3:30 AM, Michael Paquier wrote: > [ new patch ] I think this is unnecessarily awkward for callers; the attached version takes a different approach which I think will be more convenient. The attached version also (1) moves a lot more of the logic

Re: [HACKERS] Tracking wait event for latches

2016-10-03 Thread Michael Paquier
On Mon, Oct 3, 2016 at 12:35 PM, Thomas Munro wrote: > Hmm. I like the use of pgstat in that name. It helps with the > confusion created by the overloading of the term 'wait event' in the > pg_stat_activity view and the WaitEventSet API, by putting it into a >

Re: [HACKERS] Tracking wait event for latches

2016-10-02 Thread Thomas Munro
On Fri, Sep 30, 2016 at 5:47 PM, Michael Paquier wrote: > On Fri, Sep 30, 2016 at 1:48 AM, Robert Haas wrote: >> The way that we're constructing the wait event ID that ultimately gets >> advertised in pg_stat_activity is a bit silly in this

Re: [HACKERS] Tracking wait event for latches

2016-09-29 Thread Michael Paquier
On Fri, Sep 30, 2016 at 1:48 AM, Robert Haas wrote: > It seems to me that you haven't tested this patch very carefully, > because as far as I can see it breaks wait event reporting for both > heavyweight locks and buffer pins - or in other words two out of the > three

Re: [HACKERS] Tracking wait event for latches

2016-09-29 Thread Robert Haas
On Wed, Sep 28, 2016 at 9:40 PM, Michael Paquier wrote: > On Wed, Sep 28, 2016 at 9:45 PM, Robert Haas wrote: >> On Wed, Sep 28, 2016 at 8:38 AM, Michael Paquier >> wrote: >>> So should I change back the patch to have

Re: [HACKERS] Tracking wait event for latches

2016-09-28 Thread Michael Paquier
On Wed, Sep 28, 2016 at 9:45 PM, Robert Haas wrote: > On Wed, Sep 28, 2016 at 8:38 AM, Michael Paquier > wrote: >> So should I change back the patch to have only one argument for the >> eventId, and guess classId from it? > > Why would you need

Re: [HACKERS] Tracking wait event for latches

2016-09-28 Thread Robert Haas
On Wed, Sep 28, 2016 at 8:38 AM, Michael Paquier wrote: > On Wed, Sep 28, 2016 at 9:35 PM, Robert Haas wrote: >> On Tue, Sep 27, 2016 at 8:39 PM, Thomas Munro >> wrote: >>> Ok, if they really are independent then

Re: [HACKERS] Tracking wait event for latches

2016-09-28 Thread Michael Paquier
On Wed, Sep 28, 2016 at 9:35 PM, Robert Haas wrote: > On Tue, Sep 27, 2016 at 8:39 PM, Thomas Munro > wrote: >> Ok, if they really are independent then shouldn't we take advantage of >> that at call sites where we might be idle but we might

Re: [HACKERS] Tracking wait event for latches

2016-09-28 Thread Robert Haas
On Tue, Sep 27, 2016 at 8:39 PM, Thomas Munro wrote: > Ok, if they really are independent then shouldn't we take advantage of > that at call sites where we might be idle but we might also be waiting > for the network? I certainly didn't intend for them to be

Re: [HACKERS] Tracking wait event for latches

2016-09-28 Thread Michael Paquier
On Wed, Sep 28, 2016 at 3:40 PM, Thomas Munro wrote: > On Wed, Sep 28, 2016 at 6:25 PM, Michael Paquier > wrote: >> wait-event-set-v8.patch > > Ok, I'm just about ready to mark this as 'Ready for Committer'. Thanks. > Just a couple of

Re: [HACKERS] Tracking wait event for latches

2016-09-28 Thread Thomas Munro
On Wed, Sep 28, 2016 at 6:25 PM, Michael Paquier wrote: > wait-event-set-v8.patch Ok, I'm just about ready to mark this as 'Ready for Committer'. Just a couple of things: + pgstat_report_wait_start((uint8) classId, (uint16) eventId); Unnecessary casts. + +

Re: [HACKERS] Tracking wait event for latches

2016-09-27 Thread Michael Paquier
On Wed, Sep 28, 2016 at 9:39 AM, Thomas Munro wrote: > Ok, if they really are independent then shouldn't we take advantage of > that at call sites where we might be idle but we might also be waiting > for the network? For example we could do this: > >

Re: [HACKERS] Tracking wait event for latches

2016-09-27 Thread Thomas Munro
On Mon, Sep 26, 2016 at 7:07 PM, Michael Paquier wrote: > On Mon, Sep 26, 2016 at 1:46 PM, Thomas Munro > wrote: >> If the class really is strictly implied by the WaitEventIdentifier, >> then do we really need to supply it everywhere when

Re: [HACKERS] Tracking wait event for latches

2016-09-26 Thread Michael Paquier
On Mon, Sep 26, 2016 at 1:46 PM, Thomas Munro wrote: > If the class really is strictly implied by the WaitEventIdentifier, > then do we really need to supply it everywhere when calling the > various wait functions? That's going to be quite a few functions: >

Re: [HACKERS] Tracking wait event for latches

2016-09-25 Thread Thomas Munro
On Sat, Sep 24, 2016 at 1:44 AM, Michael Paquier wrote: > After digesting all the comments given, I have produced the patch > attached that uses the following categories: > +const char *const WaitEventNames[] = { > + /* activity */ > + "ArchiverMain", > +

Re: [HACKERS] Tracking wait event for latches

2016-09-24 Thread Michael Paquier
On Thu, Sep 22, 2016 at 10:49 PM, Robert Haas wrote: > Finally, extensions got their own category in this taxonomy, though I > wonder if it would be better to instead have > Activity/ExtensionActivity, Client/ExtensionClient, > Timeout/ExtensionTimeout, and IPC/ExtensionIPC

Re: [HACKERS] Tracking wait event for latches

2016-09-23 Thread Michael Paquier
On Fri, Sep 23, 2016 at 10:32 AM, Robert Haas wrote: > On Thu, Sep 22, 2016 at 7:10 PM, Thomas Munro > wrote: >> I was thinking about suggesting a category "Replication" to cover the >> waits for client IO relating to replication, as opposed

Re: [HACKERS] Tracking wait event for latches

2016-09-22 Thread Amit Kapila
On Fri, Sep 23, 2016 at 7:02 AM, Robert Haas wrote: > On Thu, Sep 22, 2016 at 7:10 PM, Thomas Munro > wrote: > >> I was thinking about suggesting a category "Replication" to cover the >> waits for client IO relating to replication, as opposed

Re: [HACKERS] Tracking wait event for latches

2016-09-22 Thread Amit Kapila
On Thu, Sep 22, 2016 at 7:19 PM, Robert Haas wrote: > On Wed, Sep 21, 2016 at 5:49 PM, Thomas Munro > wrote: > > So, I tried to classify these. Here's what I came up with. > > Activity: ArchiverMain, AutoVacuumMain, BgWriterMain, >

Re: [HACKERS] Tracking wait event for latches

2016-09-22 Thread Robert Haas
On Thu, Sep 22, 2016 at 7:10 PM, Thomas Munro wrote: > Interesting. OK, I agree that it'd be useful to show that we're > waiting because there's nothing happening, or waiting because the user > asked us to sleep, or waiting on IO, or waiting for an IPC response >

Re: [HACKERS] Tracking wait event for latches

2016-09-22 Thread Thomas Munro
On Fri, Sep 23, 2016 at 1:49 AM, Robert Haas wrote: > On Wed, Sep 21, 2016 at 5:49 PM, Thomas Munro > wrote: >>> Moreover, it's pretty confusing that we have this general concept of >>> wait events in pg_stat_activity, and then here the

Re: [HACKERS] Tracking wait event for latches

2016-09-22 Thread Robert Haas
On Thu, Sep 22, 2016 at 1:52 AM, Michael Paquier wrote: > Then comes the interesting bits: I don't think that it is a good idea > to associate only one event for a waiting point in the system views. > Say that at a waiting point WaitLatch is called with >

Re: [HACKERS] Tracking wait event for latches

2016-09-22 Thread Robert Haas
On Wed, Sep 21, 2016 at 5:49 PM, Thomas Munro wrote: >> Moreover, it's pretty confusing that we have this general concept of >> wait events in pg_stat_activity, and then here the specific type of >> wait event we're waiting for is the ... wait event kind. Uh, what?

Re: [HACKERS] Tracking wait event for latches

2016-09-21 Thread Michael Paquier
On Thu, Sep 22, 2016 at 6:49 AM, Thomas Munro wrote: > On Thu, Sep 22, 2016 at 1:23 AM, Robert Haas wrote: >> Moreover, it's pretty confusing that we have this general concept of >> wait events in pg_stat_activity, and then here the specific

Re: [HACKERS] Tracking wait event for latches

2016-09-21 Thread Thomas Munro
On Thu, Sep 22, 2016 at 1:23 AM, Robert Haas wrote: > On Tue, Sep 20, 2016 at 7:13 PM, Thomas Munro > wrote: >> This paragraph seems a bit confused. I suggest something more like >> this: "The server process is waiting for one or more

Re: [HACKERS] Tracking wait event for latches

2016-09-21 Thread Robert Haas
On Wed, Sep 21, 2016 at 10:23 AM, Michael Paquier wrote: > On Wed, Sep 21, 2016 at 11:18 PM, Robert Haas wrote: >> No, that's not what I want to do. I think we should categorize the >> events administratively by their main purpose, rather than

Re: [HACKERS] Tracking wait event for latches

2016-09-21 Thread Robert Haas
On Wed, Sep 21, 2016 at 10:02 AM, Michael Paquier wrote: > On Wed, Sep 21, 2016 at 10:23 PM, Robert Haas wrote: >> I have to admit that I like the individual event names quite a bit, >> and I think the detail will be useful to users. But I

Re: [HACKERS] Tracking wait event for latches

2016-09-21 Thread Michael Paquier
On Wed, Sep 21, 2016 at 11:18 PM, Robert Haas wrote: > No, that's not what I want to do. I think we should categorize the > events administratively by their main purpose, rather than > technologically by what we're waiting for. So we'd just have three class IDs instead of

Re: [HACKERS] Tracking wait event for latches

2016-09-21 Thread Michael Paquier
On Wed, Sep 21, 2016 at 10:23 PM, Robert Haas wrote: > I have to admit that I like the individual event names quite a bit, > and I think the detail will be useful to users. But I wonder if > there's a better way to describe the class of events that we're > talking about

Re: [HACKERS] Tracking wait event for latches

2016-09-21 Thread Robert Haas
On Tue, Sep 20, 2016 at 7:13 PM, Thomas Munro wrote: > This paragraph seems a bit confused. I suggest something more like > this: "The server process is waiting for one or more sockets, a timer > or an interprocess latch. The wait happens in a WaitEventSet, >

Re: [HACKERS] Tracking wait event for latches

2016-09-20 Thread Michael Paquier
On Wed, Sep 21, 2016 at 1:03 PM, Thomas Munro wrote: > Yeah but that's at run time. I meant you could help developers > discover ASAP if they add a new item to one place but not the other > with a compile time assertion: > const char * >

Re: [HACKERS] Tracking wait event for latches

2016-09-20 Thread Thomas Munro
On Wed, Sep 21, 2016 at 3:40 PM, Michael Paquier wrote: > On Wed, Sep 21, 2016 at 8:13 AM, Thomas Munro > wrote: >> It looks like this array wants to be in alphabetical order, but it >> isn't quite. Also, perhaps a compile time assertion

Re: [HACKERS] Tracking wait event for latches

2016-09-20 Thread Michael Paquier
On Wed, Sep 21, 2016 at 8:13 AM, Thomas Munro wrote: > + > + EventSet: The server process is waiting on a socket > + or a timer. This wait happens in a latch, an inter-process facility > + using boolean variables letting a process

Re: [HACKERS] Tracking wait event for latches

2016-09-20 Thread Thomas Munro
+ + EventSet: The server process is waiting on a socket + or a timer. This wait happens in a latch, an inter-process facility + using boolean variables letting a process sleep until it is set. + Latches support several type of operations, like

Re: [HACKERS] Tracking wait event for latches

2016-08-23 Thread Michael Paquier
On Mon, Aug 22, 2016 at 6:46 PM, Alexander Korotkov wrote: > On Mon, Aug 22, 2016 at 12:09 PM, Alexander Korotkov > wrote: >> >> I took a look at your patch. Couple of notes from me. >> >>> const char * >>> GetEventIdentifier(uint16 eventId)

Re: [HACKERS] Tracking wait event for latches

2016-08-22 Thread Michael Paquier
On Tue, Aug 23, 2016 at 12:44 AM, Robert Haas wrote: > On Mon, Aug 22, 2016 at 9:49 AM, Michael Paquier > wrote: >> The reason why I chose this way is that there are a lot of them. It is >> painful to maintain the order of the array elements in

Re: [HACKERS] Tracking wait event for latches

2016-08-22 Thread Robert Haas
On Mon, Aug 22, 2016 at 9:49 AM, Michael Paquier wrote: > The reason why I chose this way is that there are a lot of them. It is > painful to maintain the order of the array elements in perfect mapping > with the list of IDs... You can use stupid macro tricks to help

Re: [HACKERS] Tracking wait event for latches

2016-08-22 Thread Michael Paquier
On Mon, Aug 22, 2016 at 6:09 PM, Alexander Korotkov wrote: > Hi, Michael! > > On Thu, Aug 4, 2016 at 8:26 AM, Michael Paquier > wrote: > I took a look at your patch. Couple of notes from me. Thanks! >> const char * >>

Re: [HACKERS] Tracking wait event for latches

2016-08-22 Thread Alexander Korotkov
On Mon, Aug 22, 2016 at 12:09 PM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > I took a look at your patch. Couple of notes from me. > > const char * >> GetEventIdentifier(uint16 eventId) >> { >> const char *res; >> switch (eventId) >> { >> case EVENT_ARCHIVER_MAIN: >> res =

Re: [HACKERS] Tracking wait event for latches

2016-08-22 Thread Alexander Korotkov
Hi, Michael! On Thu, Aug 4, 2016 at 8:26 AM, Michael Paquier wrote: > On Tue, Aug 2, 2016 at 10:31 PM, Michael Paquier > wrote: > > Attached is an updated patch. > > Updated version for 2 minor issues: > 1) s/stram/stream/ > 2) Docs used

Re: [HACKERS] Tracking wait event for latches

2016-08-03 Thread Michael Paquier
On Tue, Aug 2, 2016 at 10:31 PM, Michael Paquier wrote: > Attached is an updated patch. Updated version for 2 minor issues: 1) s/stram/stream/ 2) Docs used incorrect number -- Michael diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c

Re: [HACKERS] Tracking wait event for latches

2016-08-02 Thread Michael Paquier
On Wed, Jun 8, 2016 at 10:11 AM, Michael Paquier wrote: > Yeah, that's as well my line of thoughts on the matter since the > beginning: keep it simple and done. What is written just after those > words is purely hand-waving and I have no way to prove it, but my >

Re: [HACKERS] Tracking wait event for latches

2016-06-07 Thread Michael Paquier
On Sat, Jun 4, 2016 at 2:41 AM, Robert Haas wrote: > On Thu, Jun 2, 2016 at 1:34 AM, Michael Paquier > wrote: >> This patch is shaped this way intentionally based on the feedback I >> received at PGCon (Robert and others). We could provide a

Re: [HACKERS] Tracking wait event for latches

2016-06-03 Thread Robert Haas
On Thu, Jun 2, 2016 at 1:34 AM, Michael Paquier wrote: > This patch is shaped this way intentionally based on the feedback I > received at PGCon (Robert and others). We could provide a routine that > extensions call in _PG_init to register a new latch event name in >

Re: [HACKERS] Tracking wait event for latches

2016-06-01 Thread Michael Paquier
On Thu, Jun 2, 2016 at 12:25 PM, Thomas Munro wrote: > This patch allows identifiers to be specified by the WaitLatch and > WaitLatchOrSocket calls, but not for WaitEventSetWait, which is the > more general waiting primitive. I think it should be done by >

Re: [HACKERS] Tracking wait event for latches

2016-06-01 Thread Thomas Munro
On Fri, May 20, 2016 at 8:14 AM, Michael Paquier wrote: > Hi all, > > As I mentioned $subject a couple of months back after looking at the > wait event facility here: >

Re: [HACKERS] Tracking wait event for latches

2016-05-19 Thread Michael Paquier
On Thu, May 19, 2016 at 4:14 PM, Michael Paquier wrote: > Comments are welcome, I am adding that in the 9.7 queue. Take that as 10.0 as things are going. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

[HACKERS] Tracking wait event for latches

2016-05-19 Thread Michael Paquier
Hi all, As I mentioned $subject a couple of months back after looking at the wait event facility here: http://www.postgresql.org/message-id/CAB7nPqTJpgAvOK4qSC96Fpm5W+aCtJ9D=3Vn9AfiEYsur=-j...@mail.gmail.com I have actually taken some time to implement this idea. The particular case that I had