Re: [HACKERS] why not parallel seq scan for slow functions

On Thu, Nov 9, 2017 at 3:47 AM, Amit Kapila wrote: > I think I understood your concern after some offlist discussion and it > is primarily due to the inheritance related check which can skip the > generation of gather paths when it shouldn't. So what might fit > better

Re: [HACKERS] why not parallel seq scan for slow functions

On Wed, Nov 8, 2017 at 6:48 PM, Robert Haas wrote: > On Wed, Nov 8, 2017 at 7:26 AM, Amit Kapila wrote: >> We do want to generate it later when there isn't inheritance involved, >> but only if there is a single rel involved (simple_rel_array_size

Re: [HACKERS] why not parallel seq scan for slow functions

On Wed, Nov 8, 2017 at 7:26 AM, Amit Kapila wrote: > We do want to generate it later when there isn't inheritance involved, > but only if there is a single rel involved (simple_rel_array_size > <=2). The rule is something like this, we will generate the gather > paths at

Re: [HACKERS] why not parallel seq scan for slow functions

On Wed, Nov 8, 2017 at 4:34 PM, Robert Haas wrote: > On Tue, Nov 7, 2017 at 9:41 PM, Amit Kapila wrote: >> This is required to prohibit generating gather path for top rel in >> case of inheritence (Append node) at this place (we want to generate >>

Re: [HACKERS] why not parallel seq scan for slow functions

On Tue, Nov 7, 2017 at 9:41 PM, Amit Kapila wrote: > This is required to prohibit generating gather path for top rel in > case of inheritence (Append node) at this place (we want to generate > it later when scan/join target is available). OK, but why don't we want to

Re: [HACKERS] why not parallel seq scan for slow functions

On Wed, Nov 8, 2017 at 2:51 AM, Robert Haas wrote: > On Mon, Nov 6, 2017 at 9:57 PM, Amit Kapila wrote: > >>> Also, even if inheritance is used, we might still be the >>> topmost scan/join target. >> >> Sure, but in that case, it won't generate

Re: [HACKERS] why not parallel seq scan for slow functions

On Mon, Nov 6, 2017 at 9:57 PM, Amit Kapila wrote: >> Well, I suppose that test will fire for a baserel when the total >> number of baserels is at least 3 and there's no inheritance involved. >> But if there are 2 baserels, we're still not the topmost scan/join >> target.

Re: [HACKERS] why not parallel seq scan for slow functions

On Mon, Nov 6, 2017 at 7:05 PM, Robert Haas wrote: > On Mon, Nov 6, 2017 at 11:20 AM, Amit Kapila wrote: >> On Mon, Nov 6, 2017 at 3:51 AM, Robert Haas wrote: >>> This looks like it's on the right track to me. I hope Tom

Re: [HACKERS] why not parallel seq scan for slow functions

On Mon, Nov 6, 2017 at 11:20 AM, Amit Kapila wrote: > On Mon, Nov 6, 2017 at 3:51 AM, Robert Haas wrote: >> This looks like it's on the right track to me. I hope Tom will look >> into it, but if he doesn't I may try to get it committed myself. >>

Re: [HACKERS] why not parallel seq scan for slow functions

On Mon, Nov 6, 2017 at 3:51 AM, Robert Haas wrote: > On Sun, Nov 5, 2017 at 12:57 AM, Amit Kapila wrote: >> Thanks for the confirmation. Find rebased patch attached. > > This looks like it's on the right track to me. I hope Tom will look > into

Re: [HACKERS] why not parallel seq scan for slow functions

Robert Haas writes: > This looks like it's on the right track to me. I hope Tom will look > into it, but if he doesn't I may try to get it committed myself. I do plan to take a look at it during this CF. > +/* Set or update cheapest_total_path and related fields

Re: [HACKERS] why not parallel seq scan for slow functions

On Sun, Nov 5, 2017 at 12:57 AM, Amit Kapila wrote: > Thanks for the confirmation. Find rebased patch attached. This looks like it's on the right track to me. I hope Tom will look into it, but if he doesn't I may try to get it committed myself. -if

Re: [HACKERS] why not parallel seq scan for slow functions

On Thu, Sep 21, 2017 at 2:35 AM, Jeff Janes wrote: > On Tue, Sep 19, 2017 at 9:15 PM, Amit Kapila > wrote: >> >> On Wed, Sep 20, 2017 at 3:05 AM, Jeff Janes wrote: >> > On Tue, Sep 19, 2017 at 1:17 PM, Thomas Munro >> >

Re: [HACKERS] why not parallel seq scan for slow functions

On Tue, Sep 19, 2017 at 9:15 PM, Amit Kapila wrote: > On Wed, Sep 20, 2017 at 3:05 AM, Jeff Janes wrote: > > On Tue, Sep 19, 2017 at 1:17 PM, Thomas Munro > > wrote: > >> > >> On Thu, Sep 14, 2017 at 3:19 PM, Amit

Re: [HACKERS] why not parallel seq scan for slow functions

On Wed, Sep 20, 2017 at 3:05 AM, Jeff Janes wrote: > On Tue, Sep 19, 2017 at 1:17 PM, Thomas Munro > wrote: >> >> On Thu, Sep 14, 2017 at 3:19 PM, Amit Kapila >> wrote: >> > The attached patch fixes both the review

Re: [HACKERS] why not parallel seq scan for slow functions

On Tue, Sep 19, 2017 at 1:17 PM, Thomas Munro wrote: > On Thu, Sep 14, 2017 at 3:19 PM, Amit Kapila > wrote: > > The attached patch fixes both the review comments as discussed above. > > This cost stuff looks unstable: > > test

Re: [HACKERS] why not parallel seq scan for slow functions

On Thu, Sep 14, 2017 at 3:19 PM, Amit Kapila wrote: > The attached patch fixes both the review comments as discussed above. This cost stuff looks unstable: test select_parallel ... FAILED ! Gather (cost=0.00..623882.94 rows=9976 width=8) Workers

Re: [HACKERS] why not parallel seq scan for slow functions

On Wed, Sep 13, 2017 at 9:39 AM, Amit Kapila wrote: > On Tue, Sep 12, 2017 at 5:47 PM, Amit Khandekar > wrote: >> On 5 September 2017 at 14:04, Amit Kapila wrote: >> >> I started with a quick review ... a couple of

Re: [HACKERS] why not parallel seq scan for slow functions

On Tue, Sep 12, 2017 at 5:47 PM, Amit Khandekar wrote: > On 5 September 2017 at 14:04, Amit Kapila wrote: > > I started with a quick review ... a couple of comments below : > > - * If this is a baserel, consider gathering any partial paths we may

Re: [HACKERS] why not parallel seq scan for slow functions

On 5 September 2017 at 14:04, Amit Kapila wrote: > On Fri, Aug 25, 2017 at 10:08 PM, Robert Haas wrote: >> On Mon, Aug 21, 2017 at 5:08 AM, Amit Kapila wrote: >>> (b) I have changed the costing of gather path for path

Re: [HACKERS] why not parallel seq scan for slow functions

Robert Haas writes: > On Wed, Sep 6, 2017 at 3:41 PM, Tom Lane wrote: >> I'm not entirely following. I thought that add_path was set up to treat >> "can be parallelized" as an independent dimension of merit, so that >> parallel paths would always

Re: [HACKERS] why not parallel seq scan for slow functions

On Wed, Sep 6, 2017 at 3:41 PM, Tom Lane wrote: > Robert Haas writes: >> In particular, as Jeff and Amit point out, it >> may well be that (a) before apply_projection_to_path(), the cheapest >> plan is non-parallel and (b) after

Re: [HACKERS] why not parallel seq scan for slow functions

Robert Haas writes: > In particular, as Jeff and Amit point out, it > may well be that (a) before apply_projection_to_path(), the cheapest > plan is non-parallel and (b) after apply_projection_to_path(), the > cheapest plan would be a Gather plan, except that it's too late

Re: [HACKERS] why not parallel seq scan for slow functions

On Wed, Sep 6, 2017 at 3:18 PM, Tom Lane wrote: > Robert Haas writes: >> On Wed, Sep 6, 2017 at 1:47 PM, Tom Lane wrote: >>> If somebody's applying apply_projection_to_path to a path that's already >>> been add_path'd, that's a

Re: [HACKERS] why not parallel seq scan for slow functions

Robert Haas writes: > On Wed, Sep 6, 2017 at 1:47 PM, Tom Lane wrote: >> If somebody's applying apply_projection_to_path to a path that's already >> been add_path'd, that's a violation of the documented restriction. > /me is confused. Isn't that

Re: [HACKERS] why not parallel seq scan for slow functions

On Wed, Sep 6, 2017 at 1:47 PM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Sep 5, 2017 at 4:34 AM, Amit Kapila wrote: >>> Okay, now I understand your point, but I think we already change the >>> cost of paths in

Re: [HACKERS] why not parallel seq scan for slow functions

Robert Haas writes: > On Tue, Sep 5, 2017 at 4:34 AM, Amit Kapila wrote: >> Okay, now I understand your point, but I think we already change the >> cost of paths in apply_projection_to_path which is done after add_path >> for top level scan/join

Re: [HACKERS] why not parallel seq scan for slow functions

On Tue, Sep 5, 2017 at 4:34 AM, Amit Kapila wrote: > Okay, now I understand your point, but I think we already change the > cost of paths in apply_projection_to_path which is done after add_path > for top level scan/join paths. Yeah. I think that's a nasty hack, and I

Re: [HACKERS] why not parallel seq scan for slow functions

On Fri, Aug 25, 2017 at 10:08 PM, Robert Haas wrote: > On Mon, Aug 21, 2017 at 5:08 AM, Amit Kapila wrote: >> (b) I have changed the costing of gather path for path target in >> generate_gather_paths which I am not sure is the best way. Another >>

Re: [HACKERS] why not parallel seq scan for slow functions

Robert Haas writes: > The point I was trying to make is that if you retroactively change the > cost of a path after you've already done add_path(), it's too late to > change your mind about whether to keep the path. At least according > to my current understanding, that's

Re: [HACKERS] why not parallel seq scan for slow functions

On Mon, Aug 21, 2017 at 5:08 AM, Amit Kapila wrote: > Thanks for acknowledging the idea. I have written a patch which > implements the above idea. At this stage, it is merely to move the > discussion forward. Few things which I am not entirely happy about > this patch

Re: [HACKERS] why not parallel seq scan for slow functions

On 21 August 2017 at 11:42, Amit Kapila wrote: >> or of 2) >> treating cost = speed, so we actually reduce the cost of a parallel >> plan rather than increasing it so it is more likely to be picked. >> > > Yeah, this is what is being currently followed for costing of

Re: [HACKERS] why not parallel seq scan for slow functions

On Mon, Aug 21, 2017 at 3:15 PM, Simon Riggs wrote: > On 21 August 2017 at 10:08, Amit Kapila wrote: > >> Thoughts? > > This seems like a very basic problem for parallel queries. > > The problem seems to be that we are calculating the cost of the

Re: [HACKERS] why not parallel seq scan for slow functions

On 21 August 2017 at 10:08, Amit Kapila wrote: > Thoughts? This seems like a very basic problem for parallel queries. The problem seems to be that we are calculating the cost of the plan rather than the speed of the plan. Clearly, a parallel task has a higher overall

Re: [HACKERS] why not parallel seq scan for slow functions

On Wed, Aug 16, 2017 at 5:04 PM, Robert Haas wrote: > On Wed, Aug 16, 2017 at 7:23 AM, Amit Kapila wrote: >> On Tue, Aug 15, 2017 at 7:15 PM, Robert Haas wrote: >>> On Sat, Aug 12, 2017 at 9:18 AM, Amit Kapila

Re: [HACKERS] why not parallel seq scan for slow functions

On Fri, 18 Aug 2017 at 4:48 PM, Amit Kapila wrote: > On Thu, Aug 17, 2017 at 2:45 PM, Dilip Kumar > wrote: > > On Thu, Aug 17, 2017 at 2:09 PM, Dilip Kumar > wrote: > >> > >> Either we can pass "num_gene" to merge_clump or

Re: [HACKERS] why not parallel seq scan for slow functions

On Thu, Aug 17, 2017 at 2:45 PM, Dilip Kumar wrote: > On Thu, Aug 17, 2017 at 2:09 PM, Dilip Kumar wrote: >> >> Either we can pass "num_gene" to merge_clump or we can store num_gene >> in the root. And inside merge_clump we can check. Do you see some

Re: [HACKERS] why not parallel seq scan for slow functions

On Thu, Aug 17, 2017 at 2:09 PM, Dilip Kumar wrote: > > Either we can pass "num_gene" to merge_clump or we can store num_gene > in the root. And inside merge_clump we can check. Do you see some more > complexity? > After putting some more thought I see one more problem but

Re: [HACKERS] why not parallel seq scan for slow functions

On Sat, Aug 12, 2017 at 6:48 PM, Amit Kapila wrote: > On Thu, Aug 10, 2017 at 1:07 AM, Robert Haas wrote: >> On Tue, Aug 8, 2017 at 3:50 AM, Amit Kapila wrote: >>> Right. >>> > > I think skipping a generation of gather

Re: [HACKERS] why not parallel seq scan for slow functions

On Wed, Aug 16, 2017 at 7:23 AM, Amit Kapila wrote: > On Tue, Aug 15, 2017 at 7:15 PM, Robert Haas wrote: >> On Sat, Aug 12, 2017 at 9:18 AM, Amit Kapila wrote: >>> I think skipping a generation of gather paths for scan

Re: [HACKERS] why not parallel seq scan for slow functions

On Tue, Aug 15, 2017 at 7:15 PM, Robert Haas wrote: > On Sat, Aug 12, 2017 at 9:18 AM, Amit Kapila wrote: >> I think skipping a generation of gather paths for scan node or top >> level join node generated via standard_join_search seems straight >>

Re: [HACKERS] why not parallel seq scan for slow functions

On Sat, Aug 12, 2017 at 9:18 AM, Amit Kapila wrote: > I think skipping a generation of gather paths for scan node or top > level join node generated via standard_join_search seems straight > forward, but skipping for paths generated via geqo seems to be tricky > (See use

Re: [HACKERS] why not parallel seq scan for slow functions

On Thu, Aug 10, 2017 at 1:07 AM, Robert Haas wrote: > On Tue, Aug 8, 2017 at 3:50 AM, Amit Kapila wrote: >> Right. >> >> I see two ways to include the cost of the target list for parallel >> paths before rejecting them (a) Don't reject parallel

Re: [HACKERS] why not parallel seq scan for slow functions

On Tue, Aug 8, 2017 at 3:50 AM, Amit Kapila wrote: > Right. > > I see two ways to include the cost of the target list for parallel > paths before rejecting them (a) Don't reject parallel paths > (Gather/GatherMerge) during add_path. This has the danger of path >

Re: [HACKERS] why not parallel seq scan for slow functions

On Wed, Aug 2, 2017 at 11:12 PM, Jeff Janes wrote: > On Wed, Jul 12, 2017 at 7:08 PM, Amit Kapila > wrote: >> >> On Wed, Jul 12, 2017 at 11:20 PM, Jeff Janes wrote: >> > On Tue, Jul 11, 2017 at 10:25 PM, Amit Kapila

Re: [HACKERS] why not parallel seq scan for slow functions

On Wed, Jul 12, 2017 at 7:08 PM, Amit Kapila wrote: > On Wed, Jul 12, 2017 at 11:20 PM, Jeff Janes wrote: > > On Tue, Jul 11, 2017 at 10:25 PM, Amit Kapila > > wrote: > >> > >> On Wed, Jul 12, 2017 at 1:50 AM, Jeff Janes

Re: [HACKERS] why not parallel seq scan for slow functions

On Mon, Jul 24, 2017 at 9:21 PM, Jeff Janes wrote: > On Sat, Jul 22, 2017 at 8:53 PM, Amit Kapila > wrote: >> >> On Thu, Jul 13, 2017 at 7:38 AM, Amit Kapila >> wrote: >> > On Wed, Jul 12, 2017 at 11:20 PM, Jeff Janes

Re: [HACKERS] why not parallel seq scan for slow functions

On Sat, Jul 22, 2017 at 8:53 PM, Amit Kapila wrote: > On Thu, Jul 13, 2017 at 7:38 AM, Amit Kapila > wrote: > > On Wed, Jul 12, 2017 at 11:20 PM, Jeff Janes > wrote: > >> > >> > >> > >> Setting parallel_workers to 8

Re: [HACKERS] why not parallel seq scan for slow functions

On Thu, Jul 13, 2017 at 7:38 AM, Amit Kapila wrote: > On Wed, Jul 12, 2017 at 11:20 PM, Jeff Janes wrote: >> >> >> >> Setting parallel_workers to 8 changes the threshold for the parallel to even >> be considered from parellel_tuple_cost <= 0.0049 to

Re: [HACKERS] why not parallel seq scan for slow functions

On Wed, Jul 12, 2017 at 11:20 PM, Jeff Janes wrote: > On Tue, Jul 11, 2017 at 10:25 PM, Amit Kapila > wrote: >> >> On Wed, Jul 12, 2017 at 1:50 AM, Jeff Janes wrote: >> > On Mon, Jul 10, 2017 at 9:51 PM, Dilip Kumar

Re: [HACKERS] why not parallel seq scan for slow functions

On Tue, Jul 11, 2017 at 10:25 PM, Amit Kapila wrote: > On Wed, Jul 12, 2017 at 1:50 AM, Jeff Janes wrote: > > On Mon, Jul 10, 2017 at 9:51 PM, Dilip Kumar > wrote: > >> > >> So because of this high projection cost the

Re: [HACKERS] why not parallel seq scan for slow functions

On Wed, Jul 12, 2017 at 10:55 AM, Amit Kapila wrote: > On Wed, Jul 12, 2017 at 1:50 AM, Jeff Janes wrote: >> On Mon, Jul 10, 2017 at 9:51 PM, Dilip Kumar wrote: >>> >>> So because of this high projection cost the seqpath and

Re: [HACKERS] why not parallel seq scan for slow functions

On Wed, Jul 12, 2017 at 1:50 AM, Jeff Janes wrote: > On Mon, Jul 10, 2017 at 9:51 PM, Dilip Kumar wrote: >> >> So because of this high projection cost the seqpath and parallel path >> both have fuzzily same cost but seqpath is winning because it's >>

Re: [HACKERS] why not parallel seq scan for slow functions

On Mon, Jul 10, 2017 at 9:51 PM, Dilip Kumar wrote: > > > In below function, we always multiply the target->cost.per_tuple with > path->rows, but in case of gather it should multiply this with > subpath->rows > > apply_projection_to_path() > > > path->startup_cost +=

Re: [HACKERS] why not parallel seq scan for slow functions

On Tue, Jul 11, 2017 at 9:02 AM, Jeff Janes wrote: > If I have a slow function which is evaluated in a simple seq scan, I do not > get parallel execution, even though it would be massively useful. Unless > force_parallel_mode=ON, then I get a dummy parallel plan with one

[HACKERS] why not parallel seq scan for slow functions

If I have a slow function which is evaluated in a simple seq scan, I do not get parallel execution, even though it would be massively useful. Unless force_parallel_mode=ON, then I get a dummy parallel plan with one worker. explain select aid,slow(abalance) from pgbench_accounts; CREATE OR