Re: [HACKERS] Improving executor performance

2016-11-08 Thread Andres Freund
Hi, On 2016-11-04 15:25:58 -0700, Doug Doole wrote: > On 07/13/2016 06:18 PM, Andres Freund wrote: > > Attached (in patch 0003) is a proof-of-concept implementing an > > expression evalution framework that doesn't use recursion. Instead > > ExecInitExpr2 computes a number of 'steps' necessary to

Re: [HACKERS] Improving executor performance

2016-11-07 Thread Doug Doole
Attached (in patch 0003) is a proof-of-concept implementing an expression evalution framework that doesn't use recursion. Instead ExecInitExpr2 computes a number of 'steps' necessary to compute an expression. These steps are stored in a linear array, and executed one after another (save boolean

Re: [HACKERS] Improving executor performance

2016-11-04 Thread Doug Doole
On 07/13/2016 06:18 PM, Andres Freund wrote: Attached (in patch 0003) is a proof-of-concept implementing an expression evalution framework that doesn't use recursion. Instead ExecInitExpr2 computes a number of 'steps' necessary to compute an expression. These steps are stored in a linear array,

Re: [HACKERS] Improving executor performance

2016-07-18 Thread Peter Geoghegan
On Wed, Jul 13, 2016 at 6:18 PM, Andres Freund wrote: > While, as in 6) above, removing linked lists from the relevant > structures helps, it's not that much. Staring at this for a long while > made me realize that, somewhat surprisingly to me, is that one of the > major

Re: [HACKERS] Improving executor performance - tidbitmap

2016-07-17 Thread Andres Freund
On 2016-07-17 08:32:17 -0400, Robert Haas wrote: > On Wed, Jul 13, 2016 at 11:06 PM, Andres Freund wrote: > > The major issue with the simplehash implementation in the path is > > probably the deletion; which should rather move cells around, rather > > than use toombstones.

Re: [HACKERS] Improving executor performance - tidbitmap

2016-07-17 Thread Robert Haas
On Wed, Jul 13, 2016 at 11:06 PM, Andres Freund wrote: > On 2016-06-24 16:29:53 -0700, Andres Freund wrote: >> 4) Various missing micro optimizations have to be performed, for more >>architectural issues to become visible. E.g. [2] causes such bad >>slowdowns in

Re: [HACKERS] Improving executor performance - tidbitmap

2016-07-14 Thread Peter Geoghegan
On Thu, Jul 14, 2016 at 8:45 PM, Andres Freund wrote: > Brin indexes IIRC always end up using tidbitmap.c, so the benefits > should be there as well ;) Right. Might the improvement be even more pronounced, though? I'm not sure how a BRIN index with a suitable

Re: [HACKERS] Improving executor performance - tidbitmap

2016-07-14 Thread Andres Freund
On 2016-07-14 20:41:21 -0700, Peter Geoghegan wrote: > On Wed, Jul 13, 2016 at 8:06 PM, Andres Freund wrote: > > I've quickly hacked up an alternative linear addressing hashtable > > implementation. And the improvements are quite remarkable. > > > > Example Query: > > EXPLAIN

Re: [HACKERS] Improving executor performance - tidbitmap

2016-07-14 Thread Peter Geoghegan
On Wed, Jul 13, 2016 at 8:06 PM, Andres Freund wrote: > I've quickly hacked up an alternative linear addressing hashtable > implementation. And the improvements are quite remarkable. > > Example Query: > EXPLAIN ANALYZE SELECT SUM(l_extendedprice) FROM lineitem WHERE

Re: [HACKERS] Improving executor performance

2016-07-14 Thread Andreas Seltenreich
Andres Freund writes: > On 2016-07-14 23:03:10 +0200, Andreas Seltenreich wrote: >> That's the plan, yes. I'm sorry there's no publishable code yet on the >> the postgres side of things. Using libFirm[1], the plan is to. > > Why libfirm? - It has a more modern IR than LLVM (they're catching up

Re: [HACKERS] Improving executor performance

2016-07-14 Thread Andres Freund
On 2016-07-14 23:03:10 +0200, Andreas Seltenreich wrote: > That's the plan, yes. I'm sorry there's no publishable code yet on the > the postgres side of things. Using libFirm[1], the plan is to. Why libfirm? It seems to only have x86 and sparc backends, and no windows support? > 1.

Re: [HACKERS] Improving executor performance

2016-07-14 Thread Andreas Seltenreich
Andres Freund writes: > The problem is that the previous form has a lot of ad-hoc analysis > strewn in. The interesting part is getting rid of all that. That's what > the new ExecInitExpr2() does. The target form can be both evaluated more > efficiently in the dispatch manner in the patch, and

Re: [HACKERS] Improving executor performance

2016-07-14 Thread Andres Freund
On 2016-07-14 20:04:03 +0200, Andreas Seltenreich wrote: > Andres Freund writes: > > > Having expression evaluation and slot deforming as a series of simple > > sequential steps, instead of complex recursive calls, would also make it > > fairly straightforward to optionally just-in-time compile

Re: [HACKERS] Improving executor performance

2016-07-14 Thread Andreas Seltenreich
Andres Freund writes: > Having expression evaluation and slot deforming as a series of simple > sequential steps, instead of complex recursive calls, would also make it > fairly straightforward to optionally just-in-time compile those. I don't think that JIT becomes easier by this change.

Re: [HACKERS] Improving executor performance - tidbitmap

2016-07-13 Thread Andres Freund
Hi, On 2016-06-24 16:29:53 -0700, Andres Freund wrote: > 4) Various missing micro optimizations have to be performed, for more >architectural issues to become visible. E.g. [2] causes such bad >slowdowns in hash-agg workloads, that other bottlenecks are hidden. One such issue is the

Re: [HACKERS] Improving executor performance

2016-07-13 Thread Andres Freund
Hi, On 2016-06-24 16:29:53 -0700, Andres Freund wrote: > 2) Baring 1) tuple deforming is the biggest bottleneck in nearly all >queries I looked at. There's various places we trigger deforming, >most ending in either slot_deform_tuple(), heap_getattr(), >heap_deform_tuple(). > >

Re: [HACKERS] Improving executor performance

2016-06-29 Thread Craig Ringer
On 30 June 2016 at 02:32, Andres Freund wrote: > > Hi, > > On 2016-06-28 10:01:28 +, Rajeev rastogi wrote: > > >3) Our 1-by-1 tuple flow in the executor has two major issues: > > > > Agreed, In order to tackle this IMHO, we should > > 1. Makes the processing data-centric

Re: [HACKERS] Improving executor performance

2016-06-29 Thread Andres Freund
Hi, On 2016-06-28 10:01:28 +, Rajeev rastogi wrote: > >3) Our 1-by-1 tuple flow in the executor has two major issues: > > Agreed, In order to tackle this IMHO, we should > 1. Makes the processing data-centric instead of operator centric. > 2. Instead of pulling each tuple from immediate

Re: [HACKERS] Improving executor performance

2016-06-28 Thread Rajeev rastogi
On 25 June 2016 05:00, Andres Freund Wrote: >To: pgsql-hackers@postgresql.org >Subject: [HACKERS] Improving executor performance > >My observations about the performance bottlenecks I found, and partially >addressed, are in rough order of importance (there's interdependence >between most of them):

Re: [HACKERS] Improving executor performance

2016-06-27 Thread Bruce Momjian
On Fri, Jun 24, 2016 at 05:25:27PM -0700, Peter Geoghegan wrote: > On Fri, Jun 24, 2016 at 4:29 PM, Andres Freund wrote: > > As a motivation, here's a somewhat juicy example of the benefits that > > can be gained (disabled parallelism, results vary too much): > > SELECT

Re: [HACKERS] Improving executor performance

2016-06-24 Thread Peter Geoghegan
On Fri, Jun 24, 2016 at 4:29 PM, Andres Freund wrote: > As a motivation, here's a somewhat juicy example of the benefits that > can be gained (disabled parallelism, results vary too much): > SELECT l_linestatus, SUM(l_quantity) FROM lineitem GROUP BY 1 ORDER BY 2 DESC > LIMIT

Re: [HACKERS] Improving executor performance

2016-06-24 Thread Andres Freund
Hi, On 2016-06-24 16:29:53 -0700, Andres Freund wrote: > Comments so far? Different suggestions on how to get improvements > around this merged? Oh, and if somebody is interested on collaborating on some of these... This is far too big for me to tackle alone. Andres -- Sent via