Re: [HACKERS] Combining Aggregates

On 27 July 2015 at 04:58, Heikki Linnakangas wrote: > > This patch seems sane to me, as far as it goes. However, there's no > planner or executor code to use the aggregate combining for anything. I'm > not a big fan of dead code, I'd really like to see something to use this. >

Re: [HACKERS] Combining Aggregates

On 04/01/2015 06:28 PM, Robert Haas wrote: On Mon, Mar 30, 2015 at 1:28 AM, Michael Paquier michael.paqu...@gmail.com wrote: I've been thinking of bumping this patch to the June commitfest as the patch only exists to provide the basic infrastructure for things like parallel aggregation,

Re: [HACKERS] Combining Aggregates

On 27 July 2015 at 04:58, Heikki Linnakangas hlinn...@iki.fi wrote: On 04/01/2015 06:28 PM, Robert Haas wrote: On Mon, Mar 30, 2015 at 1:28 AM, Michael Paquier michael.paqu...@gmail.com wrote: I've been thinking of bumping this patch to the June commitfest as the patch only exists to

Re: [HACKERS] Combining Aggregates

On 04/01/2015 06:28 PM, Robert Haas wrote: On Mon, Mar 30, 2015 at 1:28 AM, Michael Paquier michael.paqu...@gmail.com wrote: I've been thinking of bumping this patch to the June commitfest as the patch only exists to provide the basic infrastructure for things like parallel aggregation,

Re: [HACKERS] Combining Aggregates

On 27 July 2015 at 12:14, Kouhei Kaigai kai...@ak.jp.nec.com wrote: The main use case people have been talking about is parallel query, but is there some other case this would be useful right now, without the parallel query feature? You and Simon talked about this case: 2. Queries such

Re: [HACKERS] Combining Aggregates

On Mon, Mar 30, 2015 at 1:28 AM, Michael Paquier michael.paqu...@gmail.com wrote: I've been thinking of bumping this patch to the June commitfest as the patch only exists to provide the basic infrastructure for things like parallel aggregation, aggregate before join, and perhaps auto updating

Re: [HACKERS] Combining Aggregates

On 30 March 2015 at 01:08, David Rowley dgrowle...@gmail.com wrote: On 18 December 2014 at 02:48, Simon Riggs si...@2ndquadrant.com wrote: David, if you can update your patch with some docs to explain the behaviour, it looks complete enough to think about committing it in early January, to

Re: [HACKERS] Combining Aggregates

On 18 December 2014 at 02:48, Simon Riggs si...@2ndquadrant.com wrote: David, if you can update your patch with some docs to explain the behaviour, it looks complete enough to think about committing it in early January, to allow other patches that depend upon it to stand a chance of getting

Re: [HACKERS] Combining Aggregates

On Mon, Mar 30, 2015 at 2:08 PM, David Rowley dgrowle...@gmail.com wrote: On 18 December 2014 at 02:48, Simon Riggs si...@2ndquadrant.com wrote: David, if you can update your patch with some docs to explain the behaviour, it looks complete enough to think about committing it in early

Re: [HACKERS] Combining Aggregates

On Wed, Mar 4, 2015 at 11:00 PM, Kouhei Kaigai kai...@ak.jp.nec.com wrote: Anyway, I could find out, at least, these complicated issues around two-phase aggregate integration with planner. Someone can suggest minimum invasive way for these integration? I don't think I have the brain space to

Re: [HACKERS] Combining Aggregates

On Thu, Mar 5, 2015 at 9:30 AM, Kouhei Kaigai kai...@ak.jp.nec.com wrote: On Wed, Mar 4, 2015 at 4:41 AM, David Rowley dgrowle...@gmail.com wrote: This thread mentions parallel queries as a use case, but that means passing data between processes, and that requires being able to

Re: [HACKERS] Combining Aggregates

On 6 March 2015 at 19:01, Ashutosh Bapat ashutosh.ba...@enterprisedb.com wrote: Postgres-XC solved this question by creating a plan with two Agg/Group nodes, one for combining transitioned result and one for creating the distributed transition results (one per distributed run per group).

Re: [HACKERS] Combining Aggregates

On Fri, Mar 6, 2015 at 12:41 PM, David Rowley dgrow...@gmail.com wrote: On 6 March 2015 at 19:01, Ashutosh Bapat ashutosh.ba...@enterprisedb.com wrote: Postgres-XC solved this question by creating a plan with two Agg/Group nodes, one for combining transitioned result and one for creating the

Re: [HACKERS] Combining Aggregates

On Wed, Mar 4, 2015 at 4:41 AM, David Rowley dgrowle...@gmail.com wrote: This thread mentions parallel queries as a use case, but that means passing data between processes, and that requires being able to serialize and deserialize the aggregate state somehow. For actual data types that's not

Re: [HACKERS] Combining Aggregates

On Wed, Mar 4, 2015 at 4:41 AM, David Rowley dgrowle...@gmail.com wrote: This thread mentions parallel queries as a use case, but that means passing data between processes, and that requires being able to serialize and deserialize the aggregate state somehow. For actual data types that's

Re: [HACKERS] Combining Aggregates

On 25 February 2015 at 08:15, Peter Eisentraut pete...@gmx.net wrote: On 2/20/15 3:32 PM, Tomas Vondra wrote: Also, there are aggregate functions like array_agg() or string_agg() that make this impossible, just like for many custom aggregates (like hyperloglog for example). Again, I might

Re: [HACKERS] Combining Aggregates

On 21 February 2015 at 07:16, Tomas Vondra tomas.von...@2ndquadrant.com wrote: 2) serialize/deserialize functions -- This thread mentions parallel queries as a use case, but that means passing data between processes, and that requires being able to serialize

Re: [HACKERS] Combining Aggregates

On 18 February 2015 at 21:13, Kouhei Kaigai kai...@ak.jp.nec.com wrote: This patch itself looks good as an infrastructure towards the big picture, however, we still don't reach the consensus how combined functions are used instead of usual translation functions. Thank you for taking the

Re: [HACKERS] Combining Aggregates

Robert Haas robertmh...@gmail.com writes: On Tue, Feb 24, 2015 at 2:20 PM, Peter Eisentraut pete...@gmx.net wrote: I think the combine function is not actually a property of the aggregate, but a property of the transition function. If two aggregates have the same transition function, they

Re: [HACKERS] Combining Aggregates

On Tue, Feb 24, 2015 at 2:20 PM, Peter Eisentraut pete...@gmx.net wrote: On 2/20/15 3:09 PM, Tomas Vondra wrote: The 'combine' function gets two such 'state' values, while transition gets 'state' + next value. I think the combine function is not actually a property of the aggregate, but a

Re: [HACKERS] Combining Aggregates

On 2/20/15 3:32 PM, Tomas Vondra wrote: That's just because the count is hidden there in an opaque custom transition function. If, say, we had instead an array of transition functions {inc, plus, plussq} and we knew that plus and plussq are associative operators, all we'd need to special

Re: [HACKERS] Combining Aggregates

On 2/20/15 3:09 PM, Tomas Vondra wrote: The 'combine' function gets two such 'state' values, while transition gets 'state' + next value. I think the combine function is not actually a property of the aggregate, but a property of the transition function. If two aggregates have the same

Re: [HACKERS] Combining Aggregates

On 20.2.2015 21:23, Peter Eisentraut wrote: On 2/20/15 3:09 PM, Tomas Vondra wrote: On 20.2.2015 21:01, Peter Eisentraut wrote: Is there a case where the combining function is different from the transition function, other than for count? It's different in all the cases when the aggregate

Re: [HACKERS] Combining Aggregates

On 2/20/15 3:09 PM, Tomas Vondra wrote: On 20.2.2015 21:01, Peter Eisentraut wrote: Is there a case where the combining function is different from the transition function, other than for count? It's different in all the cases when the aggregate state is not identical to a single value - for

Re: [HACKERS] Combining Aggregates

Hi, On 18.2.2015 09:13, Kouhei Kaigai wrote: In addition to MIN(), MAX(), BIT_AND(), BIT_OR, SUM() for floating point types, cash and interval. I've now added combine functions for count(*) and count(col). It seems that int8pl() is suitable for this. Do you think it's worth adding any new

Re: [HACKERS] Combining Aggregates

Is there a case where the combining function is different from the transition function, other than for count? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Combining Aggregates

On 20.2.2015 21:01, Peter Eisentraut wrote: Is there a case where the combining function is different from the transition function, other than for count? It's different in all the cases when the aggregate state is not identical to a single value - for example the usual avg(), sum() and stddev()

Re: [HACKERS] Combining Aggregates

] Combining Aggregates On 18 December 2014 at 02:48, Simon Riggs si...@2ndquadrant.com wrote: David, if you can update your patch with some docs to explain the behaviour, it looks complete enough to think about committing it in early January, to allow other patches

Re: [HACKERS] Combining Aggregates

On 18 December 2014 at 02:48, Simon Riggs si...@2ndquadrant.com wrote: David, if you can update your patch with some docs to explain the behaviour, it looks complete enough to think about committing it in early January, to allow other patches that depend upon it to stand a chance of getting

[HACKERS] Combining Aggregates

KaiGai, David Rowley and myself have all made mention of various ways we could optimize aggregates. Following WIP patch adds an extra function called a combining function, that is intended to allow the user to specify a semantically correct way of breaking down an aggregate into multiple steps.

Re: [HACKERS] Combining Aggregates

On Wed, Dec 17, 2014 at 3:23 PM, Simon Riggs si...@2ndquadrant.com wrote: KaiGai, David Rowley and myself have all made mention of various ways we could optimize aggregates. Following WIP patch adds an extra function called a combining function, that is intended to allow the user to specify

Re: [HACKERS] Combining Aggregates

On 17/12/14 11:02, Atri Sharma wrote: On Wed, Dec 17, 2014 at 3:23 PM, Simon Riggs si...@2ndquadrant.com mailto:si...@2ndquadrant.com wrote: KaiGai, David Rowley and myself have all made mention of various ways we could optimize aggregates. Following WIP patch adds an extra

Re: [HACKERS] Combining Aggregates

On 17 December 2014 at 10:02, Atri Sharma atri.j...@gmail.com wrote: On Wed, Dec 17, 2014 at 3:23 PM, Simon Riggs si...@2ndquadrant.com wrote: KaiGai, David Rowley and myself have all made mention of various ways we could optimize aggregates. Following WIP patch adds an extra function

Re: [HACKERS] Combining Aggregates

On 17 December 2014 at 22:53, Simon Riggs si...@2ndquadrant.com wrote: KaiGai, David Rowley and myself have all made mention of various ways we could optimize aggregates. Following WIP patch adds an extra function called a combining function, that is intended to allow the user to specify a

Re: [HACKERS] Combining Aggregates

On 17 December 2014 at 10:20, David Rowley dgrowle...@gmail.com wrote: On 17 December 2014 at 22:53, Simon Riggs si...@2ndquadrant.com wrote: KaiGai, David Rowley and myself have all made mention of various ways we could optimize aggregates. Following WIP patch adds an extra function called

Re: [HACKERS] Combining Aggregates

Simon, Its concept is good to me. I think, the new combined function should be responsible to take a state data type as argument and update state object of the aggregate function. In other words, combined function performs like transition function but can update state object according to the

Re: [HACKERS] Combining Aggregates

On Wed, Dec 17, 2014 at 6:05 PM, Kouhei Kaigai kai...@ak.jp.nec.com wrote: Simon, Its concept is good to me. I think, the new combined function should be responsible to take a state data type as argument and update state object of the aggregate function. In other words, combined function

Re: [HACKERS] Combining Aggregates

On 17 December 2014 at 12:35, Kouhei Kaigai kai...@ak.jp.nec.com wrote: Its concept is good to me. I think, the new combined function should be responsible to take a state data type as argument and update state object of the aggregate function. In other words, combined function performs like

Re: [HACKERS] Combining Aggregates

On Wed, Dec 17, 2014 at 7:18 PM, Simon Riggs si...@2ndquadrant.com wrote: On 17 December 2014 at 12:35, Kouhei Kaigai kai...@ak.jp.nec.com wrote: Its concept is good to me. I think, the new combined function should be responsible to take a state data type as argument and update state object

Re: [HACKERS] Combining Aggregates

On 17 December 2014 at 19:06, David Rowley dgrowle...@gmail.com wrote: Standalone calls to the combine/merge functions I don't think would be testing anything new. Guess its a simple enough API, doesn't really need a specific test. That's the reason I thought it wasn't really acceptable

Re: [HACKERS] Combining Aggregates

Hi Atri, So are you proposing not calling transfuncs at all and just use combined functions? No. It is discretion of software component that distribute an aggregate into multiple partial-aggregates. That sounds counterintuitive to me. I am not able to see why you would want to avoid

<    1   2