Re: [HACKERS] walsender & parallelism

On 09/06/17 20:56, Andres Freund wrote: > On 2017-06-08 23:04:31 -0700, Noah Misch wrote: >> On Wed, Jun 07, 2017 at 10:54:57PM -0700, Noah Misch wrote: >>> On Fri, Jun 02, 2017 at 11:06:29PM -0700, Andres Freund wrote: On 2017-06-02 22:12:46 -0700, Noah Misch wrote: > On Fri, Jun 02,

Re: [HACKERS] walsender & parallelism

On 2017-06-08 23:04:31 -0700, Noah Misch wrote: > On Wed, Jun 07, 2017 at 10:54:57PM -0700, Noah Misch wrote: > > On Fri, Jun 02, 2017 at 11:06:29PM -0700, Andres Freund wrote: > > > On 2017-06-02 22:12:46 -0700, Noah Misch wrote: > > > > On Fri, Jun 02, 2017 at 11:27:55PM -0400, Peter Eisentraut

Re: [HACKERS] walsender & parallelism

On Wed, Jun 07, 2017 at 10:54:57PM -0700, Noah Misch wrote: > On Fri, Jun 02, 2017 at 11:06:29PM -0700, Andres Freund wrote: > > On 2017-06-02 22:12:46 -0700, Noah Misch wrote: > > > On Fri, Jun 02, 2017 at 11:27:55PM -0400, Peter Eisentraut wrote: > > > > On 5/31/17 23:54, Peter Eisentraut wrote:

Re: [HACKERS] walsender & parallelism

On Fri, Jun 02, 2017 at 11:06:29PM -0700, Andres Freund wrote: > On 2017-06-02 22:12:46 -0700, Noah Misch wrote: > > On Fri, Jun 02, 2017 at 11:27:55PM -0400, Peter Eisentraut wrote: > > > On 5/31/17 23:54, Peter Eisentraut wrote: > > > > On 5/29/17 22:01, Noah Misch wrote: > > > >> On Tue, May

Re: [HACKERS] walsender & parallelism

On 2017-06-02 22:12:46 -0700, Noah Misch wrote: > On Fri, Jun 02, 2017 at 11:27:55PM -0400, Peter Eisentraut wrote: > > On 5/31/17 23:54, Peter Eisentraut wrote: > > > On 5/29/17 22:01, Noah Misch wrote: > > >> On Tue, May 23, 2017 at 01:45:59PM -0400, Andres Freund wrote: > > >>> On May 23, 2017

Re: [HACKERS] walsender & parallelism

On Fri, Jun 02, 2017 at 11:27:55PM -0400, Peter Eisentraut wrote: > On 5/31/17 23:54, Peter Eisentraut wrote: > > On 5/29/17 22:01, Noah Misch wrote: > >> On Tue, May 23, 2017 at 01:45:59PM -0400, Andres Freund wrote: > >>> On May 23, 2017 1:42:41 PM EDT, Petr Jelinek > >>>

Re: [HACKERS] walsender & parallelism

On 2017-06-02 23:27:55 -0400, Peter Eisentraut wrote: > On 5/31/17 23:54, Peter Eisentraut wrote: > > On 5/29/17 22:01, Noah Misch wrote: > >> On Tue, May 23, 2017 at 01:45:59PM -0400, Andres Freund wrote: > >>> On May 23, 2017 1:42:41 PM EDT, Petr Jelinek > >>>

Re: [HACKERS] walsender & parallelism

On 5/31/17 23:54, Peter Eisentraut wrote: > On 5/29/17 22:01, Noah Misch wrote: >> On Tue, May 23, 2017 at 01:45:59PM -0400, Andres Freund wrote: >>> On May 23, 2017 1:42:41 PM EDT, Petr Jelinek >>> wrote: Hi, so this didn't really move anywhere

Re: [HACKERS] walsender & parallelism

On 2017-06-01 22:17:57 -0400, Peter Eisentraut wrote: > On 6/1/17 00:06, Andres Freund wrote: > > On 2017-05-31 23:51:08 -0400, Peter Eisentraut wrote: > >> I think the easiest and safest thing to do now is to just prevent > >> parallel plans in the walsender. See attached patch. This prevents

Re: [HACKERS] walsender & parallelism

On 6/1/17 00:06, Andres Freund wrote: > On 2017-05-31 23:51:08 -0400, Peter Eisentraut wrote: >> I think the easiest and safest thing to do now is to just prevent >> parallel plans in the walsender. See attached patch. This prevents the >> hang in the select_parallel tests run under your new

Re: [HACKERS] walsender & parallelism

On 01/06/17 06:06, Andres Freund wrote: > On 2017-05-31 23:51:08 -0400, Peter Eisentraut wrote: >> I think the easiest and safest thing to do now is to just prevent >> parallel plans in the walsender. See attached patch. This prevents the >> hang in the select_parallel tests run under your new

Re: [HACKERS] walsender & parallelism

On 2017-05-31 23:51:08 -0400, Peter Eisentraut wrote: > I think the easiest and safest thing to do now is to just prevent > parallel plans in the walsender. See attached patch. This prevents the > hang in the select_parallel tests run under your new test setup. I'm not quite sure I can buy

Re: [HACKERS] walsender & parallelism

On 1 June 2017 at 11:51, Peter Eisentraut wrote: > Unifying the signal handling and query processing further seems like a > good idea, but the patches are pretty involved, so I suggest to put them > into the next commit fest. I had a quick look a the idea of

Re: [HACKERS] walsender & parallelism

On 5/29/17 22:01, Noah Misch wrote: > On Tue, May 23, 2017 at 01:45:59PM -0400, Andres Freund wrote: >> On May 23, 2017 1:42:41 PM EDT, Petr Jelinek >> wrote: >>> Hi, >>> >>> so this didn't really move anywhere AFAICS, do we think the approach >>> I've chosen is

Re: [HACKERS] walsender & parallelism

On 5/23/17 13:57, Petr Jelinek wrote: > On 23/05/17 19:45, Andres Freund wrote: >> >> >> On May 23, 2017 1:42:41 PM EDT, Petr Jelinek >> wrote: >>> Hi, >>> >>> so this didn't really move anywhere AFAICS, do we think the approach >>> I've chosen is good or do we want

Re: [HACKERS] walsender & parallelism

On Tue, May 23, 2017 at 01:45:59PM -0400, Andres Freund wrote: > On May 23, 2017 1:42:41 PM EDT, Petr Jelinek > wrote: > >Hi, > > > >so this didn't really move anywhere AFAICS, do we think the approach > >I've chosen is good or do we want to do something else here?

Re: [HACKERS] walsender & parallelism

On 23/05/17 19:45, Andres Freund wrote: > > > On May 23, 2017 1:42:41 PM EDT, Petr Jelinek > wrote: >> Hi, >> >> so this didn't really move anywhere AFAICS, do we think the approach >> I've chosen is good or do we want to do something else here? > > Can you add

Re: [HACKERS] walsender & parallelism

On May 23, 2017 1:42:41 PM EDT, Petr Jelinek wrote: >Hi, > >so this didn't really move anywhere AFAICS, do we think the approach >I've chosen is good or do we want to do something else here? Can you add it to the open items list? Andres -- Sent from my Android

Re: [HACKERS] walsender & parallelism

Hi, so this didn't really move anywhere AFAICS, do we think the approach I've chosen is good or do we want to do something else here? -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing

Re: [HACKERS] walsender & parallelism

On 2017-04-25 11:18:35 -0400, Robert Haas wrote: > On Thu, Apr 20, 2017 at 9:40 PM, Andres Freund wrote: > > This'd be easier to look at if the fairly separate facility of allowing > > walsender to execute SQL commands had been committed separately, rather > > than as part of

Re: [HACKERS] walsender & parallelism

On Thu, Apr 20, 2017 at 9:40 PM, Andres Freund wrote: > This'd be easier to look at if the fairly separate facility of allowing > walsender to execute SQL commands had been committed separately, rather > than as part of a fairly large commit of largely unrelated things. Good

Re: [HACKERS] walsender & parallelism

On 25/04/17 01:25, Andres Freund wrote: > Hi, > > On 2017-04-24 07:31:18 +0200, Petr Jelinek wrote: >> The previous coding tried to run the unknown string throur lexer which >> could fail for some valid SQL statements as the replication command >> parser is too simple to handle all the

Re: [HACKERS] walsender & parallelism

Hi, On 2017-04-24 07:31:18 +0200, Petr Jelinek wrote: > The previous coding tried to run the unknown string throur lexer which > could fail for some valid SQL statements as the replication command > parser is too simple to handle all the complexities of SQL language. > > Instead just fall back

Re: [HACKERS] walsender & parallelism

On 24/04/17 20:00, Andres Freund wrote: > On 2017-04-24 18:29:51 +0200, Petr Jelinek wrote: >> On 24/04/17 07:42, Andres Freund wrote: >>> >>> >>> On April 23, 2017 10:31:18 PM PDT, Petr Jelinek >>> wrote: On 24/04/17 04:31, Petr Jelinek wrote: So actually

Re: [HACKERS] walsender & parallelism

On 2017-04-24 18:29:51 +0200, Petr Jelinek wrote: > On 24/04/17 07:42, Andres Freund wrote: > > > > > > On April 23, 2017 10:31:18 PM PDT, Petr Jelinek > > wrote: > >> On 24/04/17 04:31, Petr Jelinek wrote: > >> So actually maybe running regression tests through

Re: [HACKERS] walsender & parallelism

On 24/04/17 07:42, Andres Freund wrote: > > > On April 23, 2017 10:31:18 PM PDT, Petr Jelinek > wrote: >> On 24/04/17 04:31, Petr Jelinek wrote: >> So actually maybe running regression tests through it might be >> reasonable approach if we add new make target for

Re: [HACKERS] walsender & parallelism

On April 23, 2017 10:31:18 PM PDT, Petr Jelinek wrote: >On 24/04/17 04:31, Petr Jelinek wrote: >So actually maybe running regression tests through it might be >reasonable approach if we add new make target for it. That sounds like a good plan. >Note that the

Re: [HACKERS] walsender & parallelism

On 2017-04-24 04:27:58 +0200, Petr Jelinek wrote: > On 24/04/17 01:43, Andres Freund wrote: > > > >> BTW while looking at the code, I don't understand why we call > >> latch_sigusr1_handler after calling SetLatch(MyLatch), shouldn't just > >> the SetLatch be enough (they both end up calling

Re: [HACKERS] walsender & parallelism

On 2017-04-24 04:26:16 +0200, Petr Jelinek wrote: > > WalSndLastCycleHandler is genuinely different. WalSndSigHupHandler > > currently sets a different variable from postgres.c, but that seems like > > a bad idea, because afaics we'll plainly ignore SIGHUPS unless in > > WalSndLoop,

Re: [HACKERS] walsender & parallelism

On 24/04/17 02:04, Andres Freund wrote: > Hi, > > Oh, and one more point: There desperately need to be tests exercising > "SQL via walsender". Including the issue of parallelism here, the missed > cancel handler, timeouts, and such. One way to do so is to use > pg_regress with an adjusted

Re: [HACKERS] walsender & parallelism

On 24/04/17 01:43, Andres Freund wrote: > >> BTW while looking at the code, I don't understand why we call >> latch_sigusr1_handler after calling SetLatch(MyLatch), shouldn't just >> the SetLatch be enough (they both end up calling sendSelfPipeByte() >> eventually)? > > Historic raisins - there

Re: [HACKERS] walsender & parallelism

On 24/04/17 01:59, Andres Freund wrote: > Hi, > > On 2017-04-22 17:53:19 +0200, Petr Jelinek wrote: >> Here is patch. I changed both SIGINT and SIGUSR1 handlers, afaics it >> does not break anything for existing walsender usage. > >> diff --git a/src/backend/replication/logical/launcher.c >>

Re: [HACKERS] walsender & parallelism

Hi, On 2017-04-23 16:59:41 -0700, Andres Freund wrote: > Hi, > > On 2017-04-22 17:53:19 +0200, Petr Jelinek wrote: > > Here is patch. I changed both SIGINT and SIGUSR1 handlers, afaics it > > does not break anything for existing walsender usage. > > > diff --git

Re: [HACKERS] walsender & parallelism

Hi, On 2017-04-22 17:53:19 +0200, Petr Jelinek wrote: > Here is patch. I changed both SIGINT and SIGUSR1 handlers, afaics it > does not break anything for existing walsender usage. > diff --git a/src/backend/replication/logical/launcher.c > b/src/backend/replication/logical/launcher.c > index

Re: [HACKERS] walsender & parallelism

On 2017-04-21 04:20:26 +0200, Petr Jelinek wrote: > Looks like SIGUSR1 being different is problem here - it's normally used > to . I also noticed that we don't handle SIGINT (query cancel). I think we really need to unify the paths between walsender and normal backends to a much larger degree.

Re: [HACKERS] walsender & parallelism

On 21/04/17 04:37, Petr Jelinek wrote: > On 21/04/17 04:32, Craig Ringer wrote: >> On 21 April 2017 at 10:20, Petr Jelinek wrote: >>> On 21/04/17 03:40, Andres Freund wrote: Since [1] walsender (not receiver as commit message says) can execute SQL

Re: [HACKERS] walsender & parallelism

On 21/04/17 04:32, Craig Ringer wrote: > On 21 April 2017 at 10:20, Petr Jelinek wrote: >> On 21/04/17 03:40, Andres Freund wrote: >>> >>> Since [1] walsender (not receiver as commit message says) can execute >>> SQL queries. While doing some testing of [2] I

Re: [HACKERS] walsender & parallelism

On 21 April 2017 at 10:20, Petr Jelinek wrote: > On 21/04/17 03:40, Andres Freund wrote: >> >> Since [1] walsender (not receiver as commit message says) can execute >> SQL queries. While doing some testing of [2] I noticed that SQL queries >> in walsender get stuck

Re: [HACKERS] walsender & parallelism

On 21/04/17 03:40, Andres Freund wrote: > > Since [1] walsender (not receiver as commit message says) can execute > SQL queries. While doing some testing of [2] I noticed that SQL queries > in walsender get stuck if parallelism is used - I have not investigated > why that is yet, but it surely