Re: [HACKERS] logical replication busy-waiting on a lock

On 14/06/17 20:57, Andres Freund wrote: > Hi, > > On 2017-06-13 00:50:20 -0700, Andres Freund wrote: >> Just to be clear: The patch, after the first point above (which I did), >> looks ok. I'm just looking for comments. > > And pushed. > Thanks! -- Petr Jelinek

Re: [HACKERS] logical replication busy-waiting on a lock

Hi, On 2017-06-13 00:50:20 -0700, Andres Freund wrote: > Just to be clear: The patch, after the first point above (which I did), > looks ok. I'm just looking for comments. And pushed. - Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] logical replication busy-waiting on a lock

On 2017-06-13 00:32:40 -0700, Andres Freund wrote: > On 2017-06-09 22:28:00 +0200, Petr Jelinek wrote: > > On 09/06/17 03:20, Petr Jelinek wrote: > > > On 09/06/17 01:06, Andres Freund wrote: > > >> Hi, > > >> > > >> On 2017-06-03 04:50:00 +0200, Petr Jelinek wrote: > > >>> One thing I don't like

Re: [HACKERS] logical replication busy-waiting on a lock

On 2017-06-09 22:28:00 +0200, Petr Jelinek wrote: > On 09/06/17 03:20, Petr Jelinek wrote: > > On 09/06/17 01:06, Andres Freund wrote: > >> Hi, > >> > >> On 2017-06-03 04:50:00 +0200, Petr Jelinek wrote: > >>> One thing I don't like is the GetLastLocalTransactionId() that I had to > >>> add

Re: [HACKERS] logical replication busy-waiting on a lock

On 2017-06-09 22:28:00 +0200, Petr Jelinek wrote: > And here it is, seems better (the 0002 is same as before). Cool, looks good on a quick scan. > /* Define pathname of exported-snapshot files */ > #define SNAPSHOT_EXPORT_DIR "pg_snapshots" > -#define XactExportFilePath(path, xid, num,

Re: [HACKERS] logical replication busy-waiting on a lock

On 09/06/17 03:20, Petr Jelinek wrote: > On 09/06/17 01:06, Andres Freund wrote: >> Hi, >> >> On 2017-06-03 04:50:00 +0200, Petr Jelinek wrote: >>> One thing I don't like is the GetLastLocalTransactionId() that I had to >>> add because we clear the proc->lxid before we get to AtEOXact_Snapshot()

Re: [HACKERS] logical replication busy-waiting on a lock

On 09/06/17 01:06, Andres Freund wrote: > Hi, > > On 2017-06-03 04:50:00 +0200, Petr Jelinek wrote: >> One thing I don't like is the GetLastLocalTransactionId() that I had to >> add because we clear the proc->lxid before we get to AtEOXact_Snapshot() >> but I don't have better solution there. >

Re: [HACKERS] logical replication busy-waiting on a lock

Hi, On 2017-06-03 04:50:00 +0200, Petr Jelinek wrote: > One thing I don't like is the GetLastLocalTransactionId() that I had to > add because we clear the proc->lxid before we get to AtEOXact_Snapshot() > but I don't have better solution there. I dislike that quite a bit - how about we instead

Re: [HACKERS] logical replication busy-waiting on a lock

On 03/06/17 03:25, Andres Freund wrote: > >> That should solve the original problem reported here. > > Did you verify that? > Yes, I tried to manually create multiple exported logical decoding snapshots in parallel and read data from them and it worked fine, while it blocked before. > >> @@

Re: [HACKERS] logical replication busy-waiting on a lock

Hi, On 2017-06-03 03:03:20 +0200, Petr Jelinek wrote: > All right, here is my rough attempt on doing what Andres has suggested, > going straight from xid to vxid, seems to work fine in my tests and > parallel pg_dump seems happy with it as well. Cool! > The second patch removes the xid

Re: [HACKERS] logical replication busy-waiting on a lock

On 02/06/17 03:38, Robert Haas wrote: > On Thu, Jun 1, 2017 at 2:28 PM, Andres Freund wrote: >> On 2017-06-01 14:17:44 +0200, Petr Jelinek wrote: >>> Thinking more about this, I am not convinced it's a good idea to change >>> exports this late in the cycle. I still think it's

Re: [HACKERS] logical replication busy-waiting on a lock

On Thu, Jun 1, 2017 at 2:28 PM, Andres Freund wrote: > On 2017-06-01 14:17:44 +0200, Petr Jelinek wrote: >> Thinking more about this, I am not convinced it's a good idea to change >> exports this late in the cycle. I still think it's best to do the xid >> assignment only when

Re: [HACKERS] logical replication busy-waiting on a lock

On 2017-06-01 14:17:44 +0200, Petr Jelinek wrote: > Thinking more about this, I am not convinced it's a good idea to change > exports this late in the cycle. I still think it's best to do the xid > assignment only when the snapshot is actually exported but don't assign > xid when the export is

Re: [HACKERS] logical replication busy-waiting on a lock

On 31/05/17 11:21, Petr Jelinek wrote: > On 31/05/17 09:24, Andres Freund wrote: >> On 2017-05-29 23:49:33 +0200, Petr Jelinek wrote: >>> I am not quite sure I understand (both the vxid suggestion and for the >>> session dying badly). Maybe we can discuss bit more when you get to >>> computer so

Re: [HACKERS] logical replication busy-waiting on a lock

On 31/05/17 09:24, Andres Freund wrote: > On 2017-05-29 23:49:33 +0200, Petr Jelinek wrote: >> I am not quite sure I understand (both the vxid suggestion and for the >> session dying badly). Maybe we can discuss bit more when you get to >> computer so it's easier for you to expand on what you

Re: [HACKERS] logical replication busy-waiting on a lock

On 2017-05-29 23:49:33 +0200, Petr Jelinek wrote: > I am not quite sure I understand (both the vxid suggestion and for the > session dying badly). Maybe we can discuss bit more when you get to > computer so it's easier for you to expand on what you mean. The xid interlock when exporting a

Re: [HACKERS] logical replication busy-waiting on a lock

On 29/05/17 21:44, Andres Freund wrote: > > > On May 29, 2017 12:41:26 PM PDT, Petr Jelinek > wrote: >> On 29/05/17 21:28, Andres Freund wrote: >>> >>> On May 29, 2017 12:25:35 PM PDT, Petr Jelinek >> wrote: Looking at the

Re: [HACKERS] logical replication busy-waiting on a lock

On May 29, 2017 12:41:26 PM PDT, Petr Jelinek wrote: >On 29/05/17 21:28, Andres Freund wrote: >> >> On May 29, 2017 12:25:35 PM PDT, Petr Jelinek > wrote: >>> >>> Looking at the code more, the xid is only used as parameter for >>>

Re: [HACKERS] logical replication busy-waiting on a lock

On 29/05/17 21:28, Andres Freund wrote: > > On May 29, 2017 12:25:35 PM PDT, Petr Jelinek > wrote: >> >> Looking at the code more, the xid is only used as parameter for >> SnapBuildBuildSnapshot() which never does anything with that parameter, >> I wonder if it's

Re: [HACKERS] logical replication busy-waiting on a lock

On 29/05/17 21:23, Andres Freund wrote: > > > On May 29, 2017 12:21:50 PM PDT, Petr Jelinek > wrote: >> On 29/05/17 20:59, Andres Freund wrote: >>> >>> >>> On May 29, 2017 11:58:05 AM PDT, Petr Jelinek >> wrote: On 27/05/17

Re: [HACKERS] logical replication busy-waiting on a lock

On May 29, 2017 12:25:35 PM PDT, Petr Jelinek wrote: >On 29/05/17 21:21, Petr Jelinek wrote: >> On 29/05/17 20:59, Andres Freund wrote: >>> >>> >>> On May 29, 2017 11:58:05 AM PDT, Petr Jelinek > wrote: On 27/05/17 17:17, Andres

Re: [HACKERS] logical replication busy-waiting on a lock

On 29/05/17 21:21, Petr Jelinek wrote: > On 29/05/17 20:59, Andres Freund wrote: >> >> >> On May 29, 2017 11:58:05 AM PDT, Petr Jelinek >> wrote: >>> On 27/05/17 17:17, Andres Freund wrote: On May 27, 2017 9:48:22 AM EDT, Petr Jelinek >>>

Re: [HACKERS] logical replication busy-waiting on a lock

On May 29, 2017 12:21:50 PM PDT, Petr Jelinek wrote: >On 29/05/17 20:59, Andres Freund wrote: >> >> >> On May 29, 2017 11:58:05 AM PDT, Petr Jelinek > wrote: >>> On 27/05/17 17:17, Andres Freund wrote: On May 27, 2017

Re: [HACKERS] logical replication busy-waiting on a lock

On 29/05/17 20:59, Andres Freund wrote: > > > On May 29, 2017 11:58:05 AM PDT, Petr Jelinek > wrote: >> On 27/05/17 17:17, Andres Freund wrote: >>> >>> >>> On May 27, 2017 9:48:22 AM EDT, Petr Jelinek >> wrote: Actually, I guess

Re: [HACKERS] logical replication busy-waiting on a lock

On May 29, 2017 11:58:05 AM PDT, Petr Jelinek wrote: >On 27/05/17 17:17, Andres Freund wrote: >> >> >> On May 27, 2017 9:48:22 AM EDT, Petr Jelinek > wrote: >>> Actually, I guess it's the pid 47457 (COPY process) who is actually >>>

Re: [HACKERS] logical replication busy-waiting on a lock

On 27/05/17 17:17, Andres Freund wrote: > > > On May 27, 2017 9:48:22 AM EDT, Petr Jelinek > wrote: >> Actually, I guess it's the pid 47457 (COPY process) who is actually >> running the xid 73322726. In that case that's the same thing Masahiko >> Sawada reported

Re: [HACKERS] logical replication busy-waiting on a lock

On 2017-05-29 11:38:20 -0700, Jeff Janes wrote: > Related, but not the same. It would be nice if they didn't block, but if > they do have to block, shouldn't it wait on a semaphore, rather than doing > a tight loop? It looks like maybe a latch didn't get reset when it should > have or something.

Re: [HACKERS] logical replication busy-waiting on a lock

On Sat, May 27, 2017 at 6:48 AM, Petr Jelinek wrote: > > > Actually, I guess it's the pid 47457 (COPY process) who is actually > running the xid 73322726. In that case that's the same thing Masahiko > Sawada reported [1]. Related, but not the same. It would be

Re: [HACKERS] logical replication busy-waiting on a lock

On May 27, 2017 9:48:22 AM EDT, Petr Jelinek wrote: >Actually, I guess it's the pid 47457 (COPY process) who is actually >running the xid 73322726. In that case that's the same thing Masahiko >Sawada reported [1]. Which basically is result of snapshot builder

Re: [HACKERS] logical replication busy-waiting on a lock

On 27/05/17 15:44, Petr Jelinek wrote: > On 27/05/17 01:25, Jeff Janes wrote: >> When I create a subscription in the disabled state, and then later doing >> "alter subscription sub enable;", on the master I sometimes get a tight >> loop of the deadlock detector: >> >> (log_lock_waits is on, of

Re: [HACKERS] logical replication busy-waiting on a lock

On 27/05/17 01:25, Jeff Janes wrote: > When I create a subscription in the disabled state, and then later doing > "alter subscription sub enable;", on the master I sometimes get a tight > loop of the deadlock detector: > > (log_lock_waits is on, of course) > > deadlock_timeout is set to 1s, so I