Re: [HACKERS] Combining Aggregates

2015-12-02 Thread David Rowley
On 27 July 2015 at 04:58, Heikki Linnakangas wrote: > > This patch seems sane to me, as far as it goes. However, there's no > planner or executor code to use the aggregate combining for anything. I'm > not a big fan of dead code, I'd really like to see something to use this. > I've attached an u

Re: [HACKERS] Combining Aggregates

2015-07-26 Thread David Rowley
On 27 July 2015 at 12:14, Kouhei Kaigai wrote: > > The main use case people have been talking about is parallel query, but > > is there some other case this would be useful right now, without the > > parallel query feature? You and Simon talked about this case: > > > > > 2. Queries such as: > > >

Re: [HACKERS] Combining Aggregates

2015-07-26 Thread David Rowley
On 27 July 2015 at 04:58, Heikki Linnakangas wrote: > On 04/01/2015 06:28 PM, Robert Haas wrote: > >> On Mon, Mar 30, 2015 at 1:28 AM, Michael Paquier >> wrote: >> >>> I've been thinking of bumping this patch to the June commitfest as the patch only exists to provide the basic infrastructur

Re: [HACKERS] Combining Aggregates

2015-07-26 Thread Kouhei Kaigai
> On 04/01/2015 06:28 PM, Robert Haas wrote: > > On Mon, Mar 30, 2015 at 1:28 AM, Michael Paquier > > wrote: > >>> I've been thinking of bumping this patch to the June commitfest as the > >>> patch only exists to provide the basic infrastructure for things like > >>> parallel aggregation, aggregat

Re: [HACKERS] Combining Aggregates

2015-07-26 Thread Heikki Linnakangas
On 04/01/2015 06:28 PM, Robert Haas wrote: On Mon, Mar 30, 2015 at 1:28 AM, Michael Paquier wrote: I've been thinking of bumping this patch to the June commitfest as the patch only exists to provide the basic infrastructure for things like parallel aggregation, aggregate before join, and perhap

Re: [HACKERS] Combining Aggregates

2015-04-01 Thread Robert Haas
On Mon, Mar 30, 2015 at 1:28 AM, Michael Paquier wrote: >> I've been thinking of bumping this patch to the June commitfest as the >> patch only exists to provide the basic infrastructure for things like >> parallel aggregation, aggregate before join, and perhaps auto updating >> materialised views

Re: [HACKERS] Combining Aggregates

2015-03-30 Thread Simon Riggs
On 30 March 2015 at 01:08, David Rowley wrote: > On 18 December 2014 at 02:48, Simon Riggs wrote: >> >> David, if you can update your patch with some docs to explain the >> behaviour, it looks complete enough to think about committing it in >> early January, to allow other patches that depend upo

Re: [HACKERS] Combining Aggregates

2015-03-29 Thread Michael Paquier
On Mon, Mar 30, 2015 at 2:08 PM, David Rowley wrote: > > On 18 December 2014 at 02:48, Simon Riggs wrote: > >> David, if you can update your patch with some docs to explain the >> behaviour, it looks complete enough to think about committing it in >> early January, to allow other patches that de

Re: [HACKERS] Combining Aggregates

2015-03-29 Thread David Rowley
On 18 December 2014 at 02:48, Simon Riggs wrote: > David, if you can update your patch with some docs to explain the > behaviour, it looks complete enough to think about committing it in > early January, to allow other patches that depend upon it to stand a > chance of getting into 9.5. (It is no

Re: [HACKERS] Combining Aggregates

2015-03-05 Thread Ashutosh Bapat
On Fri, Mar 6, 2015 at 12:41 PM, David Rowley wrote: > On 6 March 2015 at 19:01, Ashutosh Bapat > wrote: > >> Postgres-XC solved this question by creating a plan with two Agg/Group >> nodes, one for combining transitioned result and one for creating the >> distributed transition results (one per

Re: [HACKERS] Combining Aggregates

2015-03-05 Thread David Rowley
On 6 March 2015 at 19:01, Ashutosh Bapat wrote: > Postgres-XC solved this question by creating a plan with two Agg/Group > nodes, one for combining transitioned result and one for creating the > distributed transition results (one per distributed run per group). > > So, Agg/Group for combining

Re: [HACKERS] Combining Aggregates

2015-03-05 Thread Ashutosh Bapat
On Thu, Mar 5, 2015 at 9:30 AM, Kouhei Kaigai wrote: > > On Wed, Mar 4, 2015 at 4:41 AM, David Rowley > wrote: > > >> This thread mentions "parallel queries" as a use case, but that means > > >> passing data between processes, and that requires being able to > > >> serialize and deserialize the

Re: [HACKERS] Combining Aggregates

2015-03-05 Thread Robert Haas
On Wed, Mar 4, 2015 at 11:00 PM, Kouhei Kaigai wrote: > Anyway, I could find out, at least, these complicated issues around > two-phase aggregate integration with planner. Someone can suggest > minimum invasive way for these integration? I don't think I have the brain space to think about that un

Re: [HACKERS] Combining Aggregates

2015-03-04 Thread Kouhei Kaigai
> On Wed, Mar 4, 2015 at 4:41 AM, David Rowley wrote: > >> This thread mentions "parallel queries" as a use case, but that means > >> passing data between processes, and that requires being able to > >> serialize and deserialize the aggregate state somehow. For actual data > >> types that's not ov

Re: [HACKERS] Combining Aggregates

2015-03-04 Thread Robert Haas
On Wed, Mar 4, 2015 at 4:41 AM, David Rowley wrote: >> This thread mentions "parallel queries" as a use case, but that means >> passing data between processes, and that requires being able to >> serialize and deserialize the aggregate state somehow. For actual data >> types that's not overly diffi

Re: [HACKERS] Combining Aggregates

2015-03-04 Thread David Rowley
On 21 February 2015 at 07:16, Tomas Vondra wrote: > > 2) serialize/deserialize functions > -- > > This thread mentions "parallel queries" as a use case, but that means > passing data between processes, and that requires being able to > serialize and deserialize the

Re: [HACKERS] Combining Aggregates

2015-03-04 Thread David Rowley
On 18 February 2015 at 21:13, Kouhei Kaigai wrote: > > This patch itself looks good as an infrastructure towards > the big picture, however, we still don't reach the consensus > how combined functions are used instead of usual translation > functions. Thank you for taking the time to look at the

Re: [HACKERS] Combining Aggregates

2015-03-04 Thread David Rowley
On 25 February 2015 at 08:15, Peter Eisentraut wrote: > On 2/20/15 3:32 PM, Tomas Vondra wrote: > > > Also, there are aggregate functions like array_agg() or string_agg() > > that make this impossible, just like for many custom aggregates (like > > hyperloglog for example). Again, I might not und

Re: [HACKERS] Combining Aggregates

2015-03-03 Thread Tom Lane
Robert Haas writes: > On Tue, Feb 24, 2015 at 2:20 PM, Peter Eisentraut wrote: >> I think the combine function is not actually a property of the >> aggregate, but a property of the transition function. If two aggregates >> have the same transition function, they will also have the same combine >

Re: [HACKERS] Combining Aggregates

2015-03-03 Thread Robert Haas
On Tue, Feb 24, 2015 at 2:20 PM, Peter Eisentraut wrote: > On 2/20/15 3:09 PM, Tomas Vondra wrote: >> The 'combine' function gets two such 'state' values, while transition >> gets 'state' + next value. > > I think the combine function is not actually a property of the > aggregate, but a property o

Re: [HACKERS] Combining Aggregates

2015-02-24 Thread Peter Eisentraut
On 2/20/15 3:09 PM, Tomas Vondra wrote: > The 'combine' function gets two such 'state' values, while transition > gets 'state' + next value. I think the combine function is not actually a property of the aggregate, but a property of the transition function. If two aggregates have the same transit

Re: [HACKERS] Combining Aggregates

2015-02-24 Thread Peter Eisentraut
On 2/20/15 3:32 PM, Tomas Vondra wrote: >> That's just because the count is hidden there in an opaque custom >> transition function. If, say, we had instead an array of transition >> functions {inc, plus, plussq} and we knew that plus and plussq are >> associative operators, all we'd need to spe

Re: [HACKERS] Combining Aggregates

2015-02-20 Thread Tomas Vondra
On 20.2.2015 21:23, Peter Eisentraut wrote: > On 2/20/15 3:09 PM, Tomas Vondra wrote: >> On 20.2.2015 21:01, Peter Eisentraut wrote: >>> Is there a case where the combining function is different from the >>> transition function, other than for count? >> >> It's different in all the cases when the a

Re: [HACKERS] Combining Aggregates

2015-02-20 Thread Peter Eisentraut
On 2/20/15 3:09 PM, Tomas Vondra wrote: > On 20.2.2015 21:01, Peter Eisentraut wrote: >> Is there a case where the combining function is different from the >> transition function, other than for count? > > It's different in all the cases when the aggregate state is not > identical to a single valu

Re: [HACKERS] Combining Aggregates

2015-02-20 Thread Tomas Vondra
On 20.2.2015 21:01, Peter Eisentraut wrote: > Is there a case where the combining function is different from the > transition function, other than for count? It's different in all the cases when the aggregate state is not identical to a single value - for example the usual avg(), sum() and stddev(

Re: [HACKERS] Combining Aggregates

2015-02-20 Thread Peter Eisentraut
Is there a case where the combining function is different from the transition function, other than for count? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Combining Aggregates

2015-02-20 Thread Tomas Vondra
Hi, On 18.2.2015 09:13, Kouhei Kaigai wrote: >> >> In addition to MIN(), MAX(), BIT_AND(), BIT_OR, SUM() for floating >> point types, cash and interval. I've now added combine functions >> for count(*) and count(col). It seems that int8pl() is suitable for >> this. >> >> >> Do you think it's worth

Re: [HACKERS] Combining Aggregates

2015-02-18 Thread Kouhei Kaigai
Kohei > -Original Message- > From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of David Rowley > Sent: Friday, December 19, 2014 8:39 PM > To: Simon Riggs > Cc: Kaigai Kouhei(海外 浩平); PostgreSQL-development; Amit Kapila > Subjec

Re: [HACKERS] Combining Aggregates

2014-12-19 Thread David Rowley
On 18 December 2014 at 02:48, Simon Riggs wrote: > > > David, if you can update your patch with some docs to explain the > behaviour, it looks complete enough to think about committing it in > early January, to allow other patches that depend upon it to stand a > chance of getting into 9.5. (It is

Re: [HACKERS] Combining Aggregates

2014-12-17 Thread Kouhei Kaigai
Hi Atri, > So are you proposing not calling transfuncs at all and just use combined > functions? > No. It is discretion of software component that distribute an aggregate into multiple partial-aggregates. > That sounds counterintuitive to me. I am not able to see why you would want > to avoid tr

Re: [HACKERS] Combining Aggregates

2014-12-17 Thread Simon Riggs
On 17 December 2014 at 19:06, David Rowley wrote: > Standalone calls to the combine/merge functions I don't think would be > testing anything new. Guess its a simple enough API, doesn't really need a specific test. > That's the reason I thought it wasn't really acceptable until we have a use >

Re: [HACKERS] Combining Aggregates

2014-12-17 Thread David Rowley
On 18 December 2014 at 02:48, Simon Riggs wrote: > > On 17 December 2014 at 12:35, Kouhei Kaigai wrote: > > > Its concept is good to me. I think, the new combined function should be > > responsible to take a state data type as argument and update state object > > of the aggregate function. In oth

Re: [HACKERS] Combining Aggregates

2014-12-17 Thread David Rowley
On 18 December 2014 at 01:31, Simon Riggs wrote: > > On 17 December 2014 at 10:20, David Rowley wrote: > > On 17 December 2014 at 22:53, Simon Riggs wrote: > >> > >> KaiGai, David Rowley and myself have all made mention of various ways > >> we could optimize aggregates. > >> > >> Following WIP p

Re: [HACKERS] Combining Aggregates

2014-12-17 Thread Atri Sharma
On Wed, Dec 17, 2014 at 7:18 PM, Simon Riggs wrote: > > On 17 December 2014 at 12:35, Kouhei Kaigai wrote: > > > Its concept is good to me. I think, the new combined function should be > > responsible to take a state data type as argument and update state object > > of the aggregate function. In

Re: [HACKERS] Combining Aggregates

2014-12-17 Thread Simon Riggs
On 17 December 2014 at 12:35, Kouhei Kaigai wrote: > Its concept is good to me. I think, the new combined function should be > responsible to take a state data type as argument and update state object > of the aggregate function. In other words, combined function performs like > transition functi

Re: [HACKERS] Combining Aggregates

2014-12-17 Thread Atri Sharma
On Wed, Dec 17, 2014 at 6:05 PM, Kouhei Kaigai wrote: > > Simon, > > Its concept is good to me. I think, the new combined function should be > responsible to take a state data type as argument and update state object > of the aggregate function. In other words, combined function performs like > tr

Re: [HACKERS] Combining Aggregates

2014-12-17 Thread Kouhei Kaigai
Simon, Its concept is good to me. I think, the new combined function should be responsible to take a state data type as argument and update state object of the aggregate function. In other words, combined function performs like transition function but can update state object according to the summa

Re: [HACKERS] Combining Aggregates

2014-12-17 Thread Simon Riggs
On 17 December 2014 at 10:20, David Rowley wrote: > On 17 December 2014 at 22:53, Simon Riggs wrote: >> >> KaiGai, David Rowley and myself have all made mention of various ways >> we could optimize aggregates. >> >> Following WIP patch adds an extra function called a "combining >> function", that

Re: [HACKERS] Combining Aggregates

2014-12-17 Thread David Rowley
On 17 December 2014 at 22:53, Simon Riggs wrote: > > KaiGai, David Rowley and myself have all made mention of various ways > we could optimize aggregates. > > Following WIP patch adds an extra function called a "combining > function", that is intended to allow the user to specify a > semantically

Re: [HACKERS] Combining Aggregates

2014-12-17 Thread Simon Riggs
On 17 December 2014 at 10:02, Atri Sharma wrote: > > > On Wed, Dec 17, 2014 at 3:23 PM, Simon Riggs wrote: >> >> KaiGai, David Rowley and myself have all made mention of various ways >> we could optimize aggregates. >> >> Following WIP patch adds an extra function called a "combining >> function"

Re: [HACKERS] Combining Aggregates

2014-12-17 Thread Petr Jelinek
On 17/12/14 11:02, Atri Sharma wrote: On Wed, Dec 17, 2014 at 3:23 PM, Simon Riggs mailto:si...@2ndquadrant.com>> wrote: KaiGai, David Rowley and myself have all made mention of various ways we could optimize aggregates. Following WIP patch adds an extra function called a "combini

Re: [HACKERS] Combining Aggregates

2014-12-17 Thread Atri Sharma
On Wed, Dec 17, 2014 at 3:23 PM, Simon Riggs wrote: > > KaiGai, David Rowley and myself have all made mention of various ways > we could optimize aggregates. > > Following WIP patch adds an extra function called a "combining > function", that is intended to allow the user to specify a > semantical

<    1   2