Re: add checkpoint stats of snapshot and mapping files of pg_logical dir

2022-03-20 Thread Bharath Rupireddy
On Sat, Mar 19, 2022 at 4:39 AM Andres Freund wrote: > > before we further change this (as done in this patch) we should deduplicate > these huge statements in a separate patch / commit. Something like attached v6-0001-Deduplicate-checkpoint-restartpoint-complete-mess.patch? > As discussed in a

Re: Logical replication timeout problem

2022-03-20 Thread Amit Kapila
On Fri, Mar 18, 2022 at 4:20 PM Amit Kapila wrote: > > On Fri, Mar 18, 2022 at 10:43 AM wangw.f...@fujitsu.com > wrote: > > > > On Thu, Mar 17, 2022 at 7:52 PM Masahiko Sawada > > wrote: > > > > > > > Attach the new patch. > > > > * > case REORDER_BUFFER_CHANGE_INVALIDATION: > - /* Execute

Re: [PATCH] add relation and block-level filtering to pg_waldump

2022-03-20 Thread Thomas Munro
On Mon, Mar 21, 2022 at 4:36 PM Thomas Munro wrote: > On Sat, Feb 26, 2022 at 7:58 AM David Christensen > wrote: > > Attached is V2 with additional feedback from this email, as well as the > > specification of the > > ForkNumber and FPW as specifiable options. > > Trivial fixup needed after

Re: [PATCH] add relation and block-level filtering to pg_waldump

2022-03-20 Thread Thomas Munro
On Sat, Feb 26, 2022 at 7:58 AM David Christensen wrote: > Attached is V2 with additional feedback from this email, as well as the > specification of the > ForkNumber and FPW as specifiable options. Trivial fixup needed after commit 3f1ce973. From 39e683756e718904b3a8651508cb1e2198a852e4 Mon

Summary Sort workers Stats in EXPLAIN ANALYZE

2022-03-20 Thread Jian Guo
In current EXPLAIN ANALYZE implementation, the Sort Node stats from each workers are not summarized: https://github.com/postgres/postgres/blob/d4ba8b51c76300f06cc23f4d8a41d9f7210c4866/src/backend/commands/explain.c#L2762 When the worker number is large, it will print out huge amount of node

Re: Skipping logical replication transactions on subscriber side

2022-03-20 Thread Amit Kapila
On Mon, Mar 21, 2022 at 7:09 AM Euler Taveira wrote: > > On Thu, Mar 17, 2022, at 3:03 AM, Amit Kapila wrote: > > On Wed, Mar 16, 2022 at 1:53 PM Masahiko Sawada wrote: > > > > I've attached an updated version patch. > > > > The patch LGTM. I have made minor changes in comments and docs in the >

Re: Probable CF bot degradation

2022-03-20 Thread Thomas Munro
On Mon, Mar 21, 2022 at 3:11 PM Andres Freund wrote: > On 2022-03-21 14:44:55 +1300, Thomas Munro wrote: > > Those tooltips come from the "name" elements of the .cirrus.yml file > > where tasks are defined, with Cirrus's task status appended. If we > > had a set of monochrome green and red icons

Re: Skipping logical replication transactions on subscriber side

2022-03-20 Thread Amit Kapila
On Mon, Mar 21, 2022 at 7:09 AM Euler Taveira wrote: > > src/test/subscription/t/029_disable_on_error.pl | 94 -- > src/test/subscription/t/029_on_error.pl | 183 +++ > > It seems you are removing a test for 705e20f8550c0e8e47c0b6b20b5f5ffd6ffd9e33. > We have

Re: Probable CF bot degradation

2022-03-20 Thread Andres Freund
Hi, On 2022-03-21 14:44:55 +1300, Thomas Munro wrote: > Those tooltips come from the "name" elements of the .cirrus.yml file > where tasks are defined, with Cirrus's task status appended. If we > had a set of monochrome green and red icons with a Linux penguin, > FreeBSD daemon, Windows logo and

Re: refactoring basebackup.c (zstd workers)

2022-03-20 Thread Robert Haas
On Sun, Mar 20, 2022 at 9:32 PM Tom Lane wrote: > Robert Haas writes: > > I think I'm guilty of verbal inexactitude here but not bad coding. > > Checking for *endptr != '\0', as I did, is not sufficient to detect > > "whether an error occurred," as I alleged. But, in the part of my > > response

Re: Probable CF bot degradation

2022-03-20 Thread Peter Geoghegan
On Sun, Mar 20, 2022 at 6:45 PM Thomas Munro wrote: > Nice idea, if someone with graphics skills is interested in looking into it... The logo thing wasn't really the point for me. I'd just like to have the information be more visible, sooner. I was hoping that there might be a very simple

Re: Probable CF bot degradation

2022-03-20 Thread Thomas Munro
On Mon, Mar 21, 2022 at 1:41 PM Peter Geoghegan wrote: > BTW, I think that the usability of the CFBot website would be improved > if there was a better visual indicator of what each "green tick inside > a circle" link actually indicates -- what are we testing for each > green tick/red X shown? >

Re: Skipping logical replication transactions on subscriber side

2022-03-20 Thread Euler Taveira
On Thu, Mar 17, 2022, at 3:03 AM, Amit Kapila wrote: > On Wed, Mar 16, 2022 at 1:53 PM Masahiko Sawada wrote: > > > > I've attached an updated version patch. > > > > The patch LGTM. I have made minor changes in comments and docs in the > attached patch. Kindly let me know what you think of the

Re: refactoring basebackup.c (zstd workers)

2022-03-20 Thread Robert Haas
On Sun, Mar 20, 2022 at 3:40 PM Justin Pryzby wrote: > The user-facing docs are already standardized using "compression method", with > 2 exceptions, of which one is contrib/ and the other is what I'm suggesting to > make consistent here. > > $ git grep 'compression algorithm' doc >

Re: refactoring basebackup.c (zstd workers)

2022-03-20 Thread Tom Lane
Robert Haas writes: > I think I'm guilty of verbal inexactitude here but not bad coding. > Checking for *endptr != '\0', as I did, is not sufficient to detect > "whether an error occurred," as I alleged. But, in the part of my > response you didn't quote, I believe I made it clear that I only

Re: refactoring basebackup.c (zstd workers)

2022-03-20 Thread Robert Haas
On Sun, Mar 20, 2022 at 3:11 PM Tom Lane wrote: > > Even after reading the man page for strtol, it's not clear to me that > > this is needed. That page represents checking *endptr != '\0' as > > sufficient to tell whether an error occurred. > > I'm not sure whose man page you looked at, but the

Re: Probable CF bot degradation

2022-03-20 Thread Peter Geoghegan
On Sun, Mar 20, 2022 at 4:23 PM Thomas Munro wrote: > On Mon, Mar 21, 2022 at 1:58 AM Matthias van de Meent > wrote: > > Would you know how long the expected bitrot re-check period for CF > > entries that haven't been updated is, or could the bitrot-checking > > queue be displayed somewhere to

Re: logical replication restrictions

2022-03-20 Thread Euler Taveira
On Mon, Feb 28, 2022, at 9:18 PM, Euler Taveira wrote: > Long time, no patch. Here it is. I will provide documentation in the next > version. I would appreciate some feedback. This patch is broken since commit 705e20f8550c0e8e47c0b6b20b5f5ffd6ffd9e33. I rebased it. I added documentation that

Re: Probable CF bot degradation

2022-03-20 Thread Andres Freund
Hi, On 2022-03-21 12:23:02 +1300, Thomas Munro wrote: > or actually a shade fewer because it's pretty dumb and only wakes up once a > minute to decide what to do Might be worth using https://cirrus-ci.org/api/#webhooks to trigger a run of the scheduler. Probably still want to have the timeout

Re: Probable CF bot degradation

2022-03-20 Thread Andres Freund
Hi, On 2022-03-21 12:23:02 +1300, Thomas Munro wrote: > It was set to try to recheck every ~48 hours, though it couldn't quite > always achieve that when the total number of eligible submissions is > too large. In this case it had stalled for too long after the github > outage, which I'm going

Re: Tab completion for SET TimeZone

2022-03-20 Thread Dagfinn Ilmari Mannsåker
Tom Lane writes: > After thinking a bit harder, I realized that the SchemaQuery > infrastructure has no way to deal with the case of the input text not > being a prefix of what we want the output to be, so it can't do something > comparable to Query_for_list_of_timezone_names_quoted_out. Maybe

Re: Probable CF bot degradation

2022-03-20 Thread Thomas Munro
On Mon, Mar 21, 2022 at 12:23 PM Thomas Munro wrote: > On Mon, Mar 21, 2022 at 1:58 AM Matthias van de Meent > wrote: > > Would you know how long the expected bitrot re-check period for CF > > entries that haven't been updated is, or could the bitrot-checking > > queue be displayed somewhere to

Re: Probable CF bot degradation

2022-03-20 Thread Thomas Munro
On Mon, Mar 21, 2022 at 1:58 AM Matthias van de Meent wrote: > Would you know how long the expected bitrot re-check period for CF > entries that haven't been updated is, or could the bitrot-checking > queue be displayed somewhere to indicate the position of a patch in > this queue? I see that

Re: [PATCH] pg_statio_all_tables: several rows per table due to invalid TOAST index

2022-03-20 Thread Tom Lane
I wrote: > ... We need to sum separately over the > table indexes and toast indexes, and I don't immediately see how > to do that without creating an optimization fence. After a bit of further fooling, I found that we could make that work with LEFT JOIN LATERAL. This formulation has a different

Re: [PATCH] pg_statio_all_tables: several rows per table due to invalid TOAST index

2022-03-20 Thread Tom Lane
Andrei Zubkov writes: > On Tue, 2021-11-30 at 17:29 +0900, Michael Paquier wrote: >> Hmm.  Why should we care about invalid indexes at all, including >> pg_statio_all_indexes? > I think we should care about them at least because they are exists and > can consume resources. For example, invalid

Re: shared-memory based stats collector - v66

2022-03-20 Thread Melanie Plageman
On Sun, Mar 20, 2022 at 12:58 PM Andres Freund wrote: > > Hi, > > On 2022-03-20 12:32:39 -0400, Melanie Plageman wrote: > > Attached is a TAP test to check that stats are cleaned up on a physical > > replica after the objects they concern are dropped on the primary. > > Thanks! > > > > I'm not

Re: A test for replay of regression tests

2022-03-20 Thread Thomas Munro
On Mon, Mar 21, 2022 at 2:34 AM Andrew Dunstan wrote: > On 3/20/22 05:36, Thomas Munro wrote: > > On Sun, Mar 20, 2022 at 5:20 PM Thomas Munro wrote: > >> I'll try to come up with the perl needed to see the regression.diffs > >> next time... > > Here's my proposed change to achieve that. > > I

Re: Patch proposal - parameter to limit amount of FPW because of hint bits per second

2022-03-20 Thread Peter Geoghegan
On Sun, Mar 20, 2022 at 12:44 PM Michail Nikolaev wrote: > So, I was thinking about a way to avoid such downtimes. What is about > a patch to add parameters to limit the number of FPW caused by LP_DEAD > bits per second? It is always possible to skip the setting of LP_DEAD > for future time. Such

Re: Hardening heap pruning code (was: BUG #17255: Server crashes in index_delete_sort_cmp() due to race condition with vacuum)

2022-03-20 Thread Matthias van de Meent
On Sun, 20 Mar 2022 at 04:48, Peter Geoghegan wrote: > > Attached is v6, which goes a bit further than v5 in using local state > that we build up-front to describe the state of the page being pruned > (rather than rereading the page itself). I didn't test the code; so these comments are my

Re: Tab completion for SET TimeZone

2022-03-20 Thread Tom Lane
=?utf-8?Q?Dagfinn_Ilmari_Manns=C3=A5ker?= writes: > Tom Lane writes: >> I think the reason the COMPLETE_WITH_ENUM_VALUE macro doesn't look >> similar is that it hasn't made an attempt to work with input that >> the user didn't quote --- that is, if you type >> alter type planets rename value ur

Re: Tab completion for SET TimeZone

2022-03-20 Thread Dagfinn Ilmari Mannsåker
Tom Lane writes: > =?utf-8?Q?Dagfinn_Ilmari_Manns=C3=A5ker?= writes: >> I just realised there's no point in the subselect when I'm not applying >> the same function in the WHERE and the SELECT, so here's an updated >> version that simplifies that. It also fixes a typo in the commit >> message.

Re: refactoring basebackup.c (zstd workers)

2022-03-20 Thread Justin Pryzby
On Sun, Mar 20, 2022 at 03:05:28PM -0400, Robert Haas wrote: > On Thu, Mar 17, 2022 at 3:41 PM Justin Pryzby wrote: > > -errmsg("unrecognized > > compression algorithm: \"%s\"", > > +

Re: refactoring basebackup.c (zstd workers)

2022-03-20 Thread Tom Lane
Robert Haas writes: >> Should this also set/check errno ? >> And check if value != ivalue_endp ? >> See strtol(3) > Even after reading the man page for strtol, it's not clear to me that > this is needed. That page represents checking *endptr != '\0' as > sufficient to tell whether an error

Re: refactoring basebackup.c (zstd workers)

2022-03-20 Thread Robert Haas
On Thu, Mar 17, 2022 at 3:41 PM Justin Pryzby wrote: > gzip comma I think it's fine the way it's written. If we made that change, then we'd have a comma for gzip and not for the other two algorithms. Also, I'm just moving that sentence, so any change that there is to be made here is a job for

Re: Concurrent deadlock scenario with DROP INDEX on partitioned index

2022-03-20 Thread Tom Lane
Jimmy Yih writes: > Tom Lane wrote: >> Actually though, maybe you *don't* want to do this in >> RangeVarCallbackForDropRelation. Because of point 2, it might be >> better to run find_all_inheritors after we've successfully >> identified and locked the direct target relation, ie do it back >> in

Re: a misbehavior of partition row movement (?)

2022-03-20 Thread Alvaro Herrera
On 2022-Mar-20, Amit Langote wrote: > On Sun, Mar 20, 2022 at 5:13 AM Alvaro Herrera > wrote: > > On 2022-Mar-18, Zhihong Yu wrote: > > > + if (!partRel->rd_rel->relispartition) > > > + elog(ERROR, "cannot find ancestors of a non-partition result > > > relation"); > > > > > > It would

Re: Possible fails in pg_stat_statements test

2022-03-20 Thread Andres Freund
Hi, On 2022-01-14 11:11:07 +0300, Anton A. Melnikov wrote: > This patch introduces an additional counter of wal records not related to > the query being executed. They're not unrelated though. > Due to this counter pg_stat_statement finds out the number of wal records > that are not relevant

Re: Possible fails in pg_stat_statements test

2022-03-20 Thread Anton A. Melnikov
Hello! Here is the second version of the patch rebased onto the current master. No logical changes. All other attached files from previous letter are actual. With best regards, -- Anton A. Melnikov Postgres Professional: http://www.postgrespro.com The Russian Postgres Companycommit

Re: Problem with moderation of messages with patched attached.

2022-03-20 Thread Pavel Borisov
> > оf course we could get complaints no matter what level we set the limit > at. I think raising it to 2Mb would be a reasonable experiment. If no > observable evil ensues then leave it that way. If it does then roll it > back. I agree that plain uncompressed patches are easier to deal with in >

Re: shared-memory based stats collector - v66

2022-03-20 Thread Andres Freund
Hi, On 2022-03-20 12:32:39 -0400, Melanie Plageman wrote: > Attached is a TAP test to check that stats are cleaned up on a physical > replica after the objects they concern are dropped on the primary. Thanks! > I'm not sure that the extra force next flush on standby is needed after > drop on

Re: [PATCH v2] use has_privs_for_role for predefined roles

2022-03-20 Thread Stephen Frost
Greetings, On Sun, Mar 20, 2022 at 18:31 Joshua Brindle wrote: > On Sun, Mar 20, 2022 at 12:27 PM Joe Conway wrote: > > > > On 3/3/22 11:26, Joshua Brindle wrote: > > > On Thu, Feb 10, 2022 at 2:37 PM Joe Conway wrote: > > >> > > >> On 2/10/22 14:28, Nathan Bossart wrote: > > >> > On Wed, Feb

Re: [PATCH v2] use has_privs_for_role for predefined roles

2022-03-20 Thread Joshua Brindle
On Sun, Mar 20, 2022 at 12:34 PM Joe Conway wrote: > > On 3/20/22 12:31, Joshua Brindle wrote: > > On Sun, Mar 20, 2022 at 12:27 PM Joe Conway wrote: > >> > >> On 3/3/22 11:26, Joshua Brindle wrote: > >> > On Thu, Feb 10, 2022 at 2:37 PM Joe Conway wrote: > >> >> > >> >> On 2/10/22 14:28,

Re: [PATCH v2] use has_privs_for_role for predefined roles

2022-03-20 Thread Joe Conway
On 3/20/22 12:31, Joshua Brindle wrote: On Sun, Mar 20, 2022 at 12:27 PM Joe Conway wrote: On 3/3/22 11:26, Joshua Brindle wrote: > On Thu, Feb 10, 2022 at 2:37 PM Joe Conway wrote: >> >> On 2/10/22 14:28, Nathan Bossart wrote: >> > On Wed, Feb 09, 2022 at 04:39:11PM -0500, Joe Conway wrote:

Re: shared-memory based stats collector - v66

2022-03-20 Thread Melanie Plageman
On Thu, Mar 17, 2022 at 3:36 AM Andres Freund wrote: > > Starting to feel more optimistic about this! There's loads more to do, but now > the TODOs just seem to require elbow grease, rather than deep thinking. > > The biggest todos are: > - Address all the remaining AFIXMEs and XXXs > > - add

Re: pgsql: Add option to use ICU as global locale provider

2022-03-20 Thread Andres Freund
Hi, On 2022-03-20 11:03:38 +0100, Peter Eisentraut wrote: > committed Thanks. Rebasing over that fixed the meson Centos 7 build in my meson tree. https://cirrus-ci.com/build/5265480968568832 Greetings, Andres Freund

Re: [PATCH v2] use has_privs_for_role for predefined roles

2022-03-20 Thread Joshua Brindle
On Sun, Mar 20, 2022 at 12:27 PM Joe Conway wrote: > > On 3/3/22 11:26, Joshua Brindle wrote: > > On Thu, Feb 10, 2022 at 2:37 PM Joe Conway wrote: > >> > >> On 2/10/22 14:28, Nathan Bossart wrote: > >> > On Wed, Feb 09, 2022 at 04:39:11PM -0500, Joe Conway wrote: > >> >> On 2/9/22 13:13, Nathan

Re: [PATCH v2] use has_privs_for_role for predefined roles

2022-03-20 Thread Joe Conway
On 3/3/22 11:26, Joshua Brindle wrote: On Thu, Feb 10, 2022 at 2:37 PM Joe Conway wrote: On 2/10/22 14:28, Nathan Bossart wrote: > On Wed, Feb 09, 2022 at 04:39:11PM -0500, Joe Conway wrote: >> On 2/9/22 13:13, Nathan Bossart wrote: >>> I do wonder if users find the differences between

Re: Problem with moderation of messages with patched attached.

2022-03-20 Thread Andrew Dunstan
On 3/19/22 14:48, Andres Freund wrote: > Hi, > > On 2022-03-03 13:37:35 +, Dave Page wrote: >> On Thu, 3 Mar 2022 at 13:28, Pavel Borisov wrote: >> >>> The mail system doesn't have the capability to apply different moderation rules for people in that way I'm afraid. >>> Maybe then

Re: A test for replay of regression tests

2022-03-20 Thread Andrew Dunstan
On 3/20/22 05:36, Thomas Munro wrote: > On Sun, Mar 20, 2022 at 5:20 PM Thomas Munro wrote: >> Another failure under 027_stream_regress.pl: >> >> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=prion=2022-03-16%2005%3A58%3A05 >> >> vacuum ... FAILED 3463

Re: Probable CF bot degradation

2022-03-20 Thread Julien Rouhaud
On Sun, Mar 20, 2022 at 01:58:01PM +0100, Matthias van de Meent wrote: > > I noticed that two of my patches (37/3543 and 37/3542) both failed due > to a bad commit on master (076f4d9). The issue was fixed an hour later > with b61e6214; but the pipeline for these patches hasn't run since. > Because

Re: Probable CF bot degradation

2022-03-20 Thread Matthias van de Meent
On Fri, 18 Mar 2022 at 19:52, Thomas Munro wrote: > Unfortunately cfbot didn't handle that failure very well and it was > waiting for a long timeout before scheduling more jobs. It's going > again now, and I'll try to make it more resilient against that type of > failure... I noticed that two

Re: pgsql: Add option to use ICU as global locale provider

2022-03-20 Thread Julien Rouhaud
On Sun, Mar 20, 2022 at 11:03:38AM +0100, Peter Eisentraut wrote: > On 19.03.22 05:14, Julien Rouhaud wrote: > > On Fri, Mar 18, 2022 at 03:09:59PM -0700, Andres Freund wrote: > > > Hi, > > > > > > On 2022-03-18 20:28:58 +0100, Peter Eisentraut wrote: > > > > Why does your patch introduce a

Re: Column Filtering in Logical Replication

2022-03-20 Thread Tomas Vondra
On 3/20/22 07:23, Amit Kapila wrote: > On Sun, Mar 20, 2022 at 8:41 AM Amit Kapila wrote: >> >> On Fri, Mar 18, 2022 at 10:42 PM Tomas Vondra >> wrote: >> >>> So the question is why those two sync workers never complete - I guess >>> there's some sort of lock wait (deadlock?) or infinite

Re: pgsql: Add option to use ICU as global locale provider

2022-03-20 Thread Peter Eisentraut
On 19.03.22 18:53, Christoph Berg wrote: Re: Peter Eisentraut Since some (legacy) code still uses the libc locale facilities directly, we still need to set the libc global locale settings even if ICU is otherwise selected. So pg_database now has three locale-related fields: the existing

Re: pgsql: Add option to use ICU as global locale provider

2022-03-20 Thread Peter Eisentraut
On 19.03.22 05:14, Julien Rouhaud wrote: On Fri, Mar 18, 2022 at 03:09:59PM -0700, Andres Freund wrote: Hi, On 2022-03-18 20:28:58 +0100, Peter Eisentraut wrote: Why does your patch introduce a function check_icu_locale() that is only called once? Did you have further plans for that? I

Re: BufferAlloc: don't take two simultaneous locks

2022-03-20 Thread Yura Sokolov
В Чт, 17/03/2022 в 12:02 +0900, Kyotaro Horiguchi пишет: > At Wed, 16 Mar 2022 14:11:58 +0300, Yura Sokolov > wrote in > > В Ср, 16/03/2022 в 12:07 +0900, Kyotaro Horiguchi пишет: > > > At Tue, 15 Mar 2022 13:47:17 +0300, Yura Sokolov > > > wrote in > > > In v7, HASH_ENTER returns the

Re: A test for replay of regression tests

2022-03-20 Thread Thomas Munro
On Sun, Mar 20, 2022 at 5:20 PM Thomas Munro wrote: > Another failure under 027_stream_regress.pl: > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=prion=2022-03-16%2005%3A58%3A05 > > vacuum ... FAILED 3463 ms > > I'll try to come up with the perl needed

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2022-03-20 Thread Bharath Rupireddy
On Fri, Mar 18, 2022 at 8:07 PM Nitin Jadhav wrote: > > Hi Bharath, > > Due to recent commits on master, the pg_walinpect module is not > compiling. Kindly update the patch. Thanks Nitin. Here's an updated v12 patch-set. I will respond to Andres comments in a while. Regards, Bharath Rupireddy.

Re: WIP: WAL prefetch (another approach)

2022-03-20 Thread Thomas Munro
On Sun, Mar 20, 2022 at 5:36 PM Thomas Munro wrote: > Clearly 018_wal_optimize.pl is flapping Correction, 019_replslot_limit.pl, discussed at https://www.postgresql.org/message-id/flat/83b46e5f-2a52-86aa-fa6c-8174908174b8%40iki.fi .

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2022-03-20 Thread Tatsuo Ishii
> Hi Yugo, > > I tested with serialization error scenario by setting: > default_transaction_isolation = 'repeatable read' > The result was: > > $ pgbench -t 10 -c 10 --max-tries=10 test > transaction type: > scaling factor: 10 > query mode: simple > number of clients: 10 > number of threads: 1

Re: Column Filtering in Logical Replication

2022-03-20 Thread Amit Kapila
On Sun, Mar 20, 2022 at 8:41 AM Amit Kapila wrote: > > On Fri, Mar 18, 2022 at 10:42 PM Tomas Vondra > wrote: > > > So the question is why those two sync workers never complete - I guess > > there's some sort of lock wait (deadlock?) or infinite loop. > > > > It would be a bit tricky to