Re: [HACKERS] [COMMITTERS] pgsql: Use asynchronous connect API in libpqwalreceiver

2017-03-17 Thread Tom Lane
I wrote: > Petr Jelinek writes: >> Thanks, now that I look at it again I think we might need to do cycle >> with the occurred_events and returned_events and not return on first >> find since theoretically there can be multiple sockets if this gets >> called directly and not via WaitLatchOrSocket()

Re: [HACKERS] [COMMITTERS] pgsql: Use asynchronous connect API in libpqwalreceiver

2017-03-17 Thread Tom Lane
Andrew Dunstan writes: > On 03/17/2017 12:28 PM, Tom Lane wrote: >> I'll add some comments and push this. Does everyone agree that >> this had better get back-patched, too? > Confirmed that this fixes the problem on jacana. > +1 for backpatching. I've pushed this into 9.6, but the WaitEventSet

Re: [HACKERS] [COMMITTERS] pgsql: Use asynchronous connect API in libpqwalreceiver

2017-03-17 Thread Andrew Dunstan
On 03/17/2017 12:28 PM, Tom Lane wrote: > Petr Jelinek writes: >> Now the documentation for WSAEventSelect says "The FD_WRITE network >> event is handled slightly differently. An FD_WRITE network event is >> recorded when a socket is first connected with a call to the connect, >> ConnectEx, WSAC

Re: [HACKERS] [COMMITTERS] pgsql: Use asynchronous connect API in libpqwalreceiver

2017-03-17 Thread Tom Lane
Petr Jelinek writes: > On 17/03/17 17:28, Tom Lane wrote: >> Yeah, I'm afraid we had better do something more or less like this. >> It's interesting to speculate about whether WaitEventSet could keep >> additional state that would avoid the need for a dummy send() every >> time, but that seems lik

Re: [HACKERS] [COMMITTERS] pgsql: Use asynchronous connect API in libpqwalreceiver

2017-03-17 Thread Petr Jelinek
On 17/03/17 17:28, Tom Lane wrote: > Petr Jelinek writes: >> Now the documentation for WSAEventSelect says "The FD_WRITE network >> event is handled slightly differently. An FD_WRITE network event is >> recorded when a socket is first connected with a call to the connect, >> ConnectEx, WSAConnect,

Re: [HACKERS] [COMMITTERS] pgsql: Use asynchronous connect API in libpqwalreceiver

2017-03-17 Thread Tom Lane
Petr Jelinek writes: > Now the documentation for WSAEventSelect says "The FD_WRITE network > event is handled slightly differently. An FD_WRITE network event is > recorded when a socket is first connected with a call to the connect, > ConnectEx, WSAConnect, WSAConnectByList, or WSAConnectByName fu

Re: [HACKERS] [COMMITTERS] pgsql: Use asynchronous connect API in libpqwalreceiver

2017-03-17 Thread Petr Jelinek
On 17/03/17 16:07, Tom Lane wrote: > Andrew Dunstan writes: >> I have confirmed on jacana Petr's observation that adding a timeout to >> the WaitLatchOrSocket cures the problem. > > Does anyone have a theory as to why that cures the problem? > I now have theory and even PoC patch for it. The l

Re: [HACKERS] [COMMITTERS] pgsql: Use asynchronous connect API in libpqwalreceiver

2017-03-17 Thread Tom Lane
Andrew Dunstan writes: > I have confirmed on jacana Petr's observation that adding a timeout to > the WaitLatchOrSocket cures the problem. Does anyone have a theory as to why that cures the problem? What length of timeout is being suggested here? Would a long timeout perhaps create a performanc

Re: [HACKERS] [COMMITTERS] pgsql: Use asynchronous connect API in libpqwalreceiver

2017-03-17 Thread Andrew Dunstan
On 03/08/2017 08:29 PM, Andrew Dunstan wrote: > > On 03/08/2017 08:33 AM, Peter Eisentraut wrote: >> On 3/6/17 09:38, Peter Eisentraut wrote: >>> On 3/4/17 01:45, Petr Jelinek wrote: If that's the case, the attached should fix it, but I have no way of testing it on windows, I can only s

Re: [HACKERS] [COMMITTERS] pgsql: Use asynchronous connect API in libpqwalreceiver

2017-03-16 Thread Tom Lane
Andres Freund writes: > On 2017-03-16 13:00:54 +0100, Petr Jelinek wrote: >> Looks like that didn't help either. >> >> I setup my own Windows machine and can reproduce the issue. I played >> around a bit and could not really find a fix other than adding >> WL_TIMEOUT and short timeout to WaitLatc

Re: [HACKERS] [COMMITTERS] pgsql: Use asynchronous connect API in libpqwalreceiver

2017-03-16 Thread Andres Freund
On 2017-03-16 13:00:54 +0100, Petr Jelinek wrote: > On 15/03/17 17:55, Tom Lane wrote: > > Andrew Dunstan writes: > >> On 03/03/2017 11:11 PM, Tom Lane wrote: > >>> Yeah, I was wondering if this is just exposing a pre-existing bug. > >>> However, the "normal" path operates by repeatedly invoking P

Re: [HACKERS] [COMMITTERS] pgsql: Use asynchronous connect API in libpqwalreceiver

2017-03-16 Thread Petr Jelinek
On 15/03/17 17:55, Tom Lane wrote: > Andrew Dunstan writes: >> On 03/03/2017 11:11 PM, Tom Lane wrote: >>> Yeah, I was wondering if this is just exposing a pre-existing bug. >>> However, the "normal" path operates by repeatedly invoking PQconnectPoll >>> (cf. connectDBComplete) so it's not immedia

Re: [HACKERS] [COMMITTERS] pgsql: Use asynchronous connect API in libpqwalreceiver

2017-03-15 Thread Tom Lane
Andrew Dunstan writes: > On 03/03/2017 11:11 PM, Tom Lane wrote: >> Yeah, I was wondering if this is just exposing a pre-existing bug. >> However, the "normal" path operates by repeatedly invoking PQconnectPoll >> (cf. connectDBComplete) so it's not immediately obvious how such a bug >> would've e

Re: [HACKERS] [COMMITTERS] pgsql: Use asynchronous connect API in libpqwalreceiver

2017-03-14 Thread Andrew Dunstan
On 03/03/2017 11:11 PM, Tom Lane wrote: > Peter Eisentraut writes: >> On 3/3/17 19:16, Tom Lane wrote: >>> Peter Eisentraut writes: Use asynchronous connect API in libpqwalreceiver >>> Buildfarm member bowerbird has been failing in the pg_rewind test since >>> this patch went in. It looks

Re: [HACKERS] [COMMITTERS] pgsql: Use asynchronous connect API in libpqwalreceiver

2017-03-08 Thread Andrew Dunstan
On 03/08/2017 08:33 AM, Peter Eisentraut wrote: > On 3/6/17 09:38, Peter Eisentraut wrote: >> On 3/4/17 01:45, Petr Jelinek wrote: >>> If that's the case, the attached should fix it, but I have no way of >>> testing it on windows, I can only say that it still works on my machine >>> so at least i

Re: [HACKERS] [COMMITTERS] pgsql: Use asynchronous connect API in libpqwalreceiver

2017-03-08 Thread Peter Eisentraut
On 3/6/17 09:38, Peter Eisentraut wrote: > On 3/4/17 01:45, Petr Jelinek wrote: >> If that's the case, the attached should fix it, but I have no way of >> testing it on windows, I can only say that it still works on my machine >> so at least it hopefully does not make things worse. > > Committed t

Re: [HACKERS] [COMMITTERS] pgsql: Use asynchronous connect API in libpqwalreceiver

2017-03-06 Thread Peter Eisentraut
On 3/4/17 01:45, Petr Jelinek wrote: > I can see one difference though (I didn't see this code before) and that > is, the connectDBComplete starts with waiting for socket to become > writable and only then calls PQconnectPoll, while my patch starts with > PQconnectPoll call. And I see following com

Re: [HACKERS] [COMMITTERS] pgsql: Use asynchronous connect API in libpqwalreceiver

2017-03-03 Thread Petr Jelinek
On 04/03/17 05:11, Tom Lane wrote: > Peter Eisentraut writes: >> On 3/3/17 19:16, Tom Lane wrote: >>> Peter Eisentraut writes: Use asynchronous connect API in libpqwalreceiver > >>> Buildfarm member bowerbird has been failing in the pg_rewind test since >>> this patch went in. It looks lik

Re: [HACKERS] [COMMITTERS] pgsql: Use asynchronous connect API in libpqwalreceiver

2017-03-03 Thread Tom Lane
Peter Eisentraut writes: > On 3/3/17 19:16, Tom Lane wrote: >> Peter Eisentraut writes: >>> Use asynchronous connect API in libpqwalreceiver >> Buildfarm member bowerbird has been failing in the pg_rewind test since >> this patch went in. It looks like it's failing to complete connections >> fr

Re: [HACKERS] [COMMITTERS] pgsql: Use asynchronous connect API in libpqwalreceiver

2017-03-03 Thread Peter Eisentraut
On 3/3/17 19:16, Tom Lane wrote: > Peter Eisentraut writes: >> Use asynchronous connect API in libpqwalreceiver > > Buildfarm member bowerbird has been failing in the pg_rewind test since > this patch went in. It looks like it's failing to complete connections > from the standby; which suggests

Re: [HACKERS] [COMMITTERS] pgsql: Use asynchronous connect API in libpqwalreceiver

2017-03-03 Thread Tom Lane
Peter Eisentraut writes: > Use asynchronous connect API in libpqwalreceiver Buildfarm member bowerbird has been failing in the pg_rewind test since this patch went in. It looks like it's failing to complete connections from the standby; which suggests that something platform-specific is missing