Re: [HACKERS] upper planner path-ification

2015-06-23 Thread Kouhei Kaigai
Subject: Re: [HACKERS] upper planner path-ification > > On Tue, Jun 23, 2015 at 4:41 AM, Kouhei Kaigai wrote: > > So, unless we don't find out a solution around planner, 2-phase aggregation > > is > > like a curry without rice > > Simon and I spoke with Tom ab

Re: [HACKERS] upper planner path-ification

2015-06-23 Thread Robert Haas
On Tue, Jun 23, 2015 at 4:41 AM, Kouhei Kaigai wrote: > So, unless we don't find out a solution around planner, 2-phase aggregation is > like a curry without rice Simon and I spoke with Tom about this upper planner path-ification problem at PGCon, and he indicated that he intended to work on

Re: [HACKERS] upper planner path-ification

2015-06-23 Thread Kouhei Kaigai
> -Original Message- > From: David Rowley [mailto:david.row...@2ndquadrant.com] > Sent: Tuesday, June 23, 2015 2:06 PM > To: Kaigai Kouhei(海外 浩平) > Cc: Robert Haas; pgsql-hackers@postgresql.org; Tom Lane > Subject: Re: [HACKERS] upper planner path-ification > > &g

Re: [HACKERS] upper planner path-ification

2015-06-23 Thread Kouhei Kaigai
> -Original Message- > From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Tom Lane > Sent: Monday, May 18, 2015 1:12 AM > To: Robert Haas > Cc: pgsql-hackers@postgresql.org > Subject: Re: [HACKERS] upper planner path-

Re: [HACKERS] upper planner path-ification

2015-06-22 Thread David Rowley
On 23 June 2015 at 13:55, Kouhei Kaigai wrote: > Once we support to add aggregation path during path consideration, > we need to pay attention morphing of the final target-list according > to the intermediate path combination, tentatively chosen. > For example, if partial-aggregation makes sense

Re: [HACKERS] upper planner path-ification

2015-06-22 Thread Kouhei Kaigai
> -Original Message- > From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Robert Haas > Sent: Thursday, May 14, 2015 10:39 AM > To: pgsql-hackers@postgresql.org; Tom Lane > Subject: [HACKERS] upper planner path-ification

Re: [HACKERS] upper planner path-ification

2015-05-19 Thread Kyotaro HORIGUCHI
At Tue, 19 May 2015 09:04:00 -0400, Robert Haas wrote in > On Tue, May 19, 2015 at 7:19 AM, Andrew Gierth > wrote: > >> "Tom" == Tom Lane writes: > > Tom> Hm. That's a hangover from when query_planner also gave back a > > Tom> Plan (singular) rather than a set of Paths. I don't see any

Re: [HACKERS] upper planner path-ification

2015-05-19 Thread Robert Haas
On Tue, May 19, 2015 at 7:19 AM, Andrew Gierth wrote: >> "Tom" == Tom Lane writes: > Tom> Hm. That's a hangover from when query_planner also gave back a > Tom> Plan (singular) rather than a set of Paths. I don't see any > Tom> fundamental reason why we couldn't generalize it to be a list

Re: [HACKERS] upper planner path-ification

2015-05-19 Thread Andrew Gierth
> "Tom" == Tom Lane writes: Tom> Hm. That's a hangover from when query_planner also gave back a Tom> Plan (singular) rather than a set of Paths. I don't see any Tom> fundamental reason why we couldn't generalize it to be a list of Tom> potentially useful output orderings rather than jus

Re: [HACKERS] upper planner path-ification

2015-05-18 Thread Robert Haas
On Mon, May 18, 2015 at 2:50 PM, Tom Lane wrote: >> I don't know, but it seems like this might be pulling in the opposite >> direction from your previously-stated desire to get subquery_planner >> to output Paths rather than Plans as soon as possible. > > Sorry, I didn't mean to suggest that that

Re: [HACKERS] upper planner path-ification

2015-05-18 Thread Tom Lane
Simon Riggs writes: > On 18 May 2015 at 14:50, Tom Lane wrote: >> So for the moment, let's assume that we still rigidly follow the sequence >> of upper-level steps currently embodied in grouping_planner. (I'm not >> sure if it even makes sense to consider other orderings of those >> processing s

Re: [HACKERS] upper planner path-ification

2015-05-18 Thread Simon Riggs
On 18 May 2015 at 14:50, Tom Lane wrote: > Robert Haas writes: > > On Sun, May 17, 2015 at 12:11 PM, Tom Lane wrote: > >> Rather than adding tlists per se to Paths, I've been vaguely toying with > >> a notion of identifying all the "interesting" subexpressions in a query > >> (expensive functio

Re: [HACKERS] upper planner path-ification

2015-05-18 Thread Tom Lane
Robert Haas writes: > On Sun, May 17, 2015 at 12:11 PM, Tom Lane wrote: >> Rather than adding tlists per se to Paths, I've been vaguely toying with >> a notion of identifying all the "interesting" subexpressions in a query >> (expensive functions, aggregates, etc), giving them indexes 1..n, and t

Re: [HACKERS] upper planner path-ification

2015-05-18 Thread Robert Haas
On Sun, May 17, 2015 at 12:11 PM, Tom Lane wrote: > Rather than adding tlists per se to Paths, I've been vaguely toying with > a notion of identifying all the "interesting" subexpressions in a query > (expensive functions, aggregates, etc), giving them indexes 1..n, and then > marking Paths with b

Re: [HACKERS] upper planner path-ification

2015-05-18 Thread Tom Lane
Andrew Gierth writes: > Incidentally, the most obvious obstacle to better planning of grouping > sets in the sorted cases is not so much how to pick paths in > grouping_planner itself, but rather the fact that query_planner wants to > be given only one sort order. Is there any prospect for improve

Re: [HACKERS] upper planner path-ification

2015-05-18 Thread Andrew Gierth
> "Tom" == Tom Lane writes: >> Hrm, ok. So for the near future, we should leave it more or less >> as-is? We don't have a timescale yet, but it's our intention to >> submit a hashagg support patch for grouping sets as soon as time >> permits. Tom> Well, mumble. I keep saying that I wa

Re: [HACKERS] upper planner path-ification

2015-05-17 Thread Tom Lane
Robert Haas writes: > So, getting back to this part, what's the value of returning a list of > Paths rather than a list of Plans? (1) less work, since we don't have to fill in details not needed for costing purposes; (2) paths carry info that the planner wants but the executor doesn't, no

Re: [HACKERS] upper planner path-ification

2015-05-17 Thread Tom Lane
Andrew Gierth writes: > "Tom" == Tom Lane writes: > Tom> So I'm all for refactoring, but I think it will happen as a natural > Tom> byproduct of path-ification, and otherwise would be rather forced. > Hrm, ok. So for the near future, we should leave it more or less as-is? > We don't have a tim

Re: [HACKERS] upper planner path-ification

2015-05-17 Thread Andrew Gierth
> "Tom" == Tom Lane writes: >> If there's interest, we could do that specific task as part of >> adding hashagg support for grouping sets (which would otherwise make >> it even longer), or as preparatory work for that. Tom> I think that refactoring without changing anything about the way

Re: [HACKERS] upper planner path-ification

2015-05-17 Thread Tom Lane
Andrew Gierth writes: > "Robert" == Robert Haas writes: > Robert> I think grouping_planner() is badly in need of some refactoring > Robert> just to make it shorter. It's over 1000 lines of code, which > Robert> IMHO is a fairly ridiculous length for a single function. > If there's interest,

Re: [HACKERS] upper planner path-ification

2015-05-17 Thread Andrew Gierth
> "Robert" == Robert Haas writes: Robert> I think grouping_planner() is badly in need of some refactoring Robert> just to make it shorter. It's over 1000 lines of code, which Robert> IMHO is a fairly ridiculous length for a single function. If there's interest, we could do that specific

Re: [HACKERS] upper planner path-ification

2015-05-16 Thread Robert Haas
On Wed, May 13, 2015 at 10:27 PM, Tom Lane wrote: > For the reasons I mentioned, I'd like to get to a point where > subquery_planner's output is Paths not Plans as soon as possible. But the > idea of coarse representation of steps that we aren't trying to be smart > about might be useful to save

Re: [HACKERS] upper planner path-ification

2015-05-14 Thread Ashutosh Bapat
On Thu, May 14, 2015 at 8:24 PM, Tom Lane wrote: > Robert Haas writes: > > On Wed, May 13, 2015 at 10:27 PM, Tom Lane wrote: > >> For the reasons I mentioned, I'd like to get to a point where > >> subquery_planner's output is Paths not Plans as soon as possible. But > the > >> idea of coarse r

Re: [HACKERS] upper planner path-ification

2015-05-14 Thread Ashutosh Bapat
On Thu, May 14, 2015 at 7:09 AM, Robert Haas wrote: > Hi, > > I've been pulling over Tom's occasional remarks about redoing > grouping_planner - and maybe further layers of the planner - to work > with Paths instead of Plans. I've had difficulty locating all of the > relevant threads. Here's ev

Re: [HACKERS] upper planner path-ification

2015-05-14 Thread Martijn van Oosterhout
On Thu, May 14, 2015 at 12:19:44PM -0400, Robert Haas wrote: > Well, I'm just shooting from the hip here, but it seems to me that the > basic pipeline as it exists today is Join -> Aggregate -> SetOp -> > Limit -> LockRows. I don't think Limit or LockRows can be moved any > earlier. SetOps have a

Re: [HACKERS] upper planner path-ification

2015-05-14 Thread Robert Haas
On Thu, May 14, 2015 at 10:54 AM, Tom Lane wrote: > In any case, the key question if we're to have Paths representing > higher-level computations is "what do we hang our lists of such Paths > off of?". Yeah, I was wondering about that, too. > If we have say both GROUP BY and LIMIT, it's importan

Re: [HACKERS] upper planner path-ification

2015-05-14 Thread Tom Lane
Robert Haas writes: > On Wed, May 13, 2015 at 10:27 PM, Tom Lane wrote: >> For the reasons I mentioned, I'd like to get to a point where >> subquery_planner's output is Paths not Plans as soon as possible. But the >> idea of coarse representation of steps that we aren't trying to be smart >> abo

Re: [HACKERS] upper planner path-ification

2015-05-14 Thread Kyotaro HORIGUCHI
Hello, this topic lured me on.. At Wed, 13 May 2015 23:43:57 -0400, Robert Haas wrote in > On Wed, May 13, 2015 at 10:27 PM, Tom Lane wrote: > > Both of those are problems all right, but there is more context here. > > Thanks for providing the context. > > >> I'm inclined to think that it wo

Re: [HACKERS] upper planner path-ification

2015-05-13 Thread Robert Haas
On Wed, May 13, 2015 at 10:27 PM, Tom Lane wrote: > Both of those are problems all right, but there is more context here. Thanks for providing the context. >> I'm inclined to think that it would be useful to solve the first >> problem even if we didn't solve the second one right away (but that >

Re: [HACKERS] upper planner path-ification

2015-05-13 Thread Tom Lane
Robert Haas writes: > I've been pulling over Tom's occasional remarks about redoing > grouping_planner - and maybe further layers of the planner - to work > with Paths instead of Plans. ... > I think there are two separate problems here. First, there's the > problem that grouping_planner() is com

[HACKERS] upper planner path-ification

2015-05-13 Thread Robert Haas
Hi, I've been pulling over Tom's occasional remarks about redoing grouping_planner - and maybe further layers of the planner - to work with Paths instead of Plans. I've had difficulty locating all of the relevant threads. Here's everything I've found so far, which I'm pretty sure is not everythi