Re: Window Function "Run Conditions"

2022-04-07 Thread David Rowley
On Fri, 8 Apr 2022 at 02:11, David Rowley wrote: > Barring any objection, I'm planning to push this one in around 10 hours time. Pushed. 9d9c02ccd Thank you all for the reviews. David

Re: Window Function "Run Conditions"

2022-04-07 Thread Zhihong Yu
On Thu, Apr 7, 2022 at 7:11 AM David Rowley wrote: > On Thu, 7 Apr 2022 at 19:01, David Rowley wrote: > > > > On Thu, 7 Apr 2022 at 15:41, Zhihong Yu wrote: > > > +* We must keep the original qual in place if there is > a > > > +* PARTITION BY clause as the

Re: Window Function "Run Conditions"

2022-04-07 Thread David Rowley
On Thu, 7 Apr 2022 at 19:01, David Rowley wrote: > > On Thu, 7 Apr 2022 at 15:41, Zhihong Yu wrote: > > +* We must keep the original qual in place if there is a > > +* PARTITION BY clause as the top-level WindowAgg remains in > > +* pass-through

Re: Window Function "Run Conditions"

2022-04-07 Thread David Rowley
On Thu, 7 Apr 2022 at 15:41, Zhihong Yu wrote: > +* We must keep the original qual in place if there is a > +* PARTITION BY clause as the top-level WindowAgg remains in > +* pass-through mode and does nothing to filter out unwanted > +

Re: Window Function "Run Conditions"

2022-04-06 Thread Zhihong Yu
On Wed, Apr 6, 2022 at 7:36 PM David Rowley wrote: > On Wed, 6 Apr 2022 at 00:59, Andy Fan wrote: > > > > On Tue, Apr 5, 2022 at 7:49 PM David Rowley > wrote: > >> Yeah, there is more performance to be had than even what you've done > >> there. There's no reason really for spool_tuples() to

Re: Window Function "Run Conditions"

2022-04-06 Thread David Rowley
On Wed, 6 Apr 2022 at 00:59, Andy Fan wrote: > > On Tue, Apr 5, 2022 at 7:49 PM David Rowley wrote: >> Yeah, there is more performance to be had than even what you've done >> there. There's no reason really for spool_tuples() to do >> tuplestore_puttupleslot() when we're not in run mode. > > >

Re: Window Function "Run Conditions"

2022-04-05 Thread Andy Fan
On Tue, Apr 5, 2022 at 7:49 PM David Rowley wrote: > On Tue, 5 Apr 2022 at 19:38, Andy Fan wrote: > > 1. We can do more on PASSTHROUGH, we just bypass the window function > > currently, but IIUC we can ignore all of the following tuples in > current partition > > once we go into this mode.

Re: Window Function "Run Conditions"

2022-04-05 Thread David Rowley
On Tue, 5 Apr 2022 at 19:38, Andy Fan wrote: > 1. We can do more on PASSTHROUGH, we just bypass the window function > currently, but IIUC we can ignore all of the following tuples in current > partition > once we go into this mode. patch 0001 shows what I mean. Yeah, there is more performance

Re: Window Function "Run Conditions"

2022-04-05 Thread Andy Fan
> > The root cause is even ExecQual(winstate->runcondition, econtext) return > false, we > still return the slot to the upper node. A simple hack can avoid it. > Forget to say 0002 shows what I mean. -- Best Regards Andy Fan

Re: Window Function "Run Conditions"

2022-04-05 Thread Andy Fan
Hi David: I just came to the office today to double check this patch. I probably can > finish it very soon. > I would share my current review result first and more review is still in progress. There is a lot of amazing stuff there but I'd save the simple +1 and just share something I'm not

Re: Window Function "Run Conditions"

2022-04-04 Thread Andres Freund
On 2022-04-05 12:04:18 +1200, David Rowley wrote: > > This is afaics slightly cheaper than referencing a variable in a slot. > > I guess you must mean cheaper because it means there will be no > EEOP_*_FETCHSOME step? Otherwise it seems a fairly similar amount of > work. That, and slightly

Re: Window Function "Run Conditions"

2022-04-04 Thread Andy Fan
> > > I'm pretty happy with this now. If anyone wants to have a look at > this, can they do so or let me know they're going to within the next > 24 hours. Otherwise I plan to move into commit mode with it. > > I just came to the office today to double check this patch. I probably can finish it

Re: Window Function "Run Conditions"

2022-04-04 Thread David Rowley
Thanks for having a look at this. On Wed, 30 Mar 2022 at 11:16, Andres Freund wrote: > On 2022-03-29 15:11:52 +1300, David Rowley wrote: > > One thing which I'm not sure about with the patch is how I'm handling > > the evaluation of the runcondition in nodeWindowAgg.c. Instead of > > having

Re: Window Function "Run Conditions"

2022-03-29 Thread Andres Freund
Hi, On 2022-03-29 15:11:52 +1300, David Rowley wrote: > One thing which I'm not sure about with the patch is how I'm handling > the evaluation of the runcondition in nodeWindowAgg.c. Instead of > having ExecQual() evaluate an OpExpr such as "row_number() over (...) > <= 10", I'm replacing the

Re: Window Function "Run Conditions"

2022-03-28 Thread David Rowley
On Wed, 23 Mar 2022 at 16:30, David Rowley wrote: > > On Wed, 23 Mar 2022 at 11:24, David Rowley wrote: > > I think it's safer to just disable the optimisation when there are > > multiple window clauses. Multiple matching clauses are merged > > already, so it's perfectly valid to have multiple

Re: Window Function "Run Conditions"

2022-03-22 Thread David G. Johnston
On Tue, Mar 22, 2022 at 3:39 PM David Rowley wrote: > On Thu, 17 Mar 2022 at 17:04, Corey Huinker > wrote: > > It seems like this effort would aid in implementing what some other > databases implement via the QUALIFY clause, which is to window functions > what HAVING is to aggregate functions.

Re: Window Function "Run Conditions"

2022-03-22 Thread David Rowley
On Wed, 23 Mar 2022 at 11:24, David Rowley wrote: > I think it's safer to just disable the optimisation when there are > multiple window clauses. Multiple matching clauses are merged > already, so it's perfectly valid to have multiple window functions, > it's just they must share the same window

Re: Window Function "Run Conditions"

2022-03-22 Thread David Rowley
Node *clause) +{ + OpExpr *opexpr = (OpExpr *) clause; + boolkeep_original = true; + Var*var1; + Var*var2; + + /* We're only able to use OpExprs with 2 operands */ + if (!IsA(opexpr, OpExpr)) +

Re: Window Function "Run Conditions"

2022-03-22 Thread Zhihong Yu
On Tue, Mar 22, 2022 at 3:24 PM David Rowley wrote: > On Thu, 26 Aug 2021 at 14:54, Andy Fan wrote: > > > > On Thu, Aug 19, 2021 at 2:35 PM David Rowley > wrote: > > > > > > On Thu, 19 Aug 2021 at 00:20, Andy Fan > wrote: > > > > In the current master, the result is: > > > > > > > > empno |

Re: Window Function "Run Conditions"

2022-03-22 Thread David Rowley
On Thu, 17 Mar 2022 at 17:04, Corey Huinker wrote: > It seems like this effort would aid in implementing what some other databases > implement via the QUALIFY clause, which is to window functions what HAVING is > to aggregate functions. > example: >

Re: Window Function "Run Conditions"

2022-03-22 Thread David Rowley
On Wed, 16 Mar 2022 at 10:24, Greg Stark wrote: > > This looks like an awesome addition. Thanks > I have one technical questions... > > Is it possible to actually transform the row_number case into a LIMIT > clause or make the planner support for this case equivalent to it (in > which case we

Re: Window Function "Run Conditions"

2022-03-22 Thread David Rowley
ly ignore the original qual because the window function + * will use the run condition to stop at the right time. + */ +static bool +check_and_push_window_quals(Query *subquery, RangeTblEntry *rte, Index rti, + Node *clause) +{ + OpExpr *

Re: Window Function "Run Conditions"

2022-03-21 Thread Andres Freund
Hi, On 2021-08-19 18:35:27 +1200, David Rowley wrote: > Anyway, I'll take a few more days to think about it before posting a fix. The patch in the CF entry doesn't apply: http://cfbot.cputube.org/patch_37_3234.log The quoted message was ~6 months ago. I think this CF entry should be marked as

Re: Window Function "Run Conditions"

2022-03-16 Thread Corey Huinker
On Tue, Mar 15, 2022 at 5:24 PM Greg Stark wrote: > This looks like an awesome addition. > > I have one technical questions... > > Is it possible to actually transform the row_number case into a LIMIT > clause or make the planner support for this case equivalent to it (in > which case we can

Re: Window Function "Run Conditions"

2022-03-15 Thread Greg Stark
This looks like an awesome addition. I have one technical questions... Is it possible to actually transform the row_number case into a LIMIT clause or make the planner support for this case equivalent to it (in which case we can replace the LIMIT clause planning to transform into a window

Re: Window Function "Run Conditions"

2021-08-25 Thread Andy Fan
On Thu, Aug 19, 2021 at 2:35 PM David Rowley wrote: > > On Thu, 19 Aug 2021 at 00:20, Andy Fan wrote: > > In the current master, the result is: > > > > empno | salary | c | dr > > ---++---+ > > 8 | 6000 | 4 | 1 > > > In the patched version, the result is: > > > > empno

Re: Window Function "Run Conditions"

2021-08-19 Thread David Rowley
On Thu, 19 Aug 2021 at 00:20, Andy Fan wrote: > In the current master, the result is: > > empno | salary | c | dr > ---++---+ > 8 | 6000 | 4 | 1 > In the patched version, the result is: > > empno | salary | c | dr > ---++---+ > 8 | 6000 | 1 | 1

Re: Window Function "Run Conditions"

2021-08-18 Thread Andy Fan
Hi David: Thanks for the patch. On Wed, Aug 18, 2021 at 6:40 PM David Rowley wrote: > > On Tue, 17 Aug 2021 at 03:51, Zhihong Yu wrote: > > + if ((res->monotonic & MONOTONICFUNC_INCREASING) == > > MONOTONICFUNC_INCREASING) > > > > The above can be simplified as 'if

Re: Window Function "Run Conditions"

2021-08-18 Thread David Rowley
On Tue, 17 Aug 2021 at 03:51, Zhihong Yu wrote: > + if ((res->monotonic & MONOTONICFUNC_INCREASING) == > MONOTONICFUNC_INCREASING) > > The above can be simplified as 'if (res->monotonic & > MONOTONICFUNC_INCREASING) ' True. I've attached an updated patch. David

Re: Window Function "Run Conditions"

2021-08-16 Thread Zhihong Yu
On Mon, Aug 16, 2021 at 3:28 AM David Rowley wrote: > On Thu, 1 Jul 2021 at 21:11, David Rowley wrote: > > 1) Unsure of the API to the prosupport function. I wonder if the > > prosupport function should just be able to say if the function is > > either monotonically increasing or decreasing or

Re: Window Function "Run Conditions"

2021-08-16 Thread David Rowley
On Thu, 1 Jul 2021 at 21:11, David Rowley wrote: > 1) Unsure of the API to the prosupport function. I wonder if the > prosupport function should just be able to say if the function is > either monotonically increasing or decreasing or neither then have > core code build a qual. That would make

Re: Window Function "Run Conditions"

2021-07-01 Thread Pavel Stehule
čt 1. 7. 2021 v 11:11 odesílatel David Rowley napsal: > It seems like a few too many years of an SQL standard without any > standardised way to LIMIT the number of records in a result set caused > various applications to adopt some strange ways to get this behaviour. > Over here in the

Window Function "Run Conditions"

2021-07-01 Thread David Rowley
It seems like a few too many years of an SQL standard without any standardised way to LIMIT the number of records in a result set caused various applications to adopt some strange ways to get this behaviour. Over here in the PostgreSQL world, we just type LIMIT n; at the end of our queries. I