Re: [HACKERS] CLUSTER, reform_and_rewrite_tuple(), and parallelism

2016-10-27 Thread Claudio Freire
On Wed, Oct 26, 2016 at 9:33 PM, Peter Geoghegan wrote: > Besides, parallel CREATE INDEX alone will probably > be quite effective at speeding up CLUSTER in practice, simply because > that's often where most wall clock time is spent during a CLUSTER > operation. Creating all

Re: [HACKERS] CLUSTER, reform_and_rewrite_tuple(), and parallelism

2016-10-26 Thread Peter Geoghegan
On Wed, Aug 17, 2016 at 4:12 PM, Peter Geoghegan wrote: > During preliminary analysis of what it would take to produce a > parallel CLUSTER patch that is analogous of what I came up with for > CREATE INDEX, which in general seems quite possible, I identified >

Re: [HACKERS] CLUSTER, reform_and_rewrite_tuple(), and parallelism

2016-08-19 Thread Amit Kapila
On Thu, Aug 18, 2016 at 4:42 AM, Peter Geoghegan wrote: > > Does anyone have any ideas on how to: > > 1). Directly address the reform_and_rewrite_tuple() bottleneck. > > and/or: > > 2). Push down some or all of the reform_and_rewrite_tuple() work till > before tuples are passed

Re: [HACKERS] CLUSTER, reform_and_rewrite_tuple(), and parallelism

2016-08-17 Thread Andres Freund
On 2016-08-17 17:35:32 -0700, Peter Geoghegan wrote: > On Wed, Aug 17, 2016 at 5:20 PM, Andres Freund wrote: > > libc isn't compiled with -fno-omit-frame-pointer (and even if, it uses > > assembly without setup of the frame pointer), so frame pointer based > > call graphs are

Re: [HACKERS] CLUSTER, reform_and_rewrite_tuple(), and parallelism

2016-08-17 Thread Peter Geoghegan
On Wed, Aug 17, 2016 at 5:20 PM, Andres Freund wrote: > libc isn't compiled with -fno-omit-frame-pointer (and even if, it uses > assembly without setup of the frame pointer), so frame pointer based > call graphs are wrong through libc. The attributions are based on > random

Re: [HACKERS] CLUSTER, reform_and_rewrite_tuple(), and parallelism

2016-08-17 Thread Andres Freund
On 2016-08-17 21:17:56 -0300, Alvaro Herrera wrote: > Peter Geoghegan wrote: > > > This doesn't seem that interesting, but not sure what you're looking for. > > > > I also attach cycles flamegraph. > > I may be blind, but what are those write() calls attributed to > heap_form_tuple? libc isn't

Re: [HACKERS] CLUSTER, reform_and_rewrite_tuple(), and parallelism

2016-08-17 Thread Alvaro Herrera
Peter Geoghegan wrote: > This doesn't seem that interesting, but not sure what you're looking for. > > I also attach cycles flamegraph. I may be blind, but what are those write() calls attributed to heap_form_tuple? -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL

Re: [HACKERS] CLUSTER, reform_and_rewrite_tuple(), and parallelism

2016-08-17 Thread Andres Freund
On 2016-08-17 16:58:49 -0700, Peter Geoghegan wrote: > On Wed, Aug 17, 2016 at 4:28 PM, Andres Freund wrote: > > Could you also provide a strace -ttt -T -c and a cpu cycles flamegraph? > > Here is the output from that strace invocation, plus a -p (to attach > to the relevant

Re: [HACKERS] CLUSTER, reform_and_rewrite_tuple(), and parallelism

2016-08-17 Thread Peter Geoghegan
On Wed, Aug 17, 2016 at 4:28 PM, Andres Freund wrote: > Could you also provide a strace -ttt -T -c and a cpu cycles flamegraph? Here is the output from that strace invocation, plus a -p (to attach to the relevant backend): strace: -t has no effect with -c strace: -T has no

Re: [HACKERS] CLUSTER, reform_and_rewrite_tuple(), and parallelism

2016-08-17 Thread Andres Freund
On 2016-08-17 16:23:29 -0700, Peter Geoghegan wrote: > On Wed, Aug 17, 2016 at 4:16 PM, Andres Freund wrote: > >> Does anyone have any ideas on how to: > >> > >> 1). Directly address the reform_and_rewrite_tuple() bottleneck. > > > > What part of is actually the expensive bit?

Re: [HACKERS] CLUSTER, reform_and_rewrite_tuple(), and parallelism

2016-08-17 Thread Peter Geoghegan
On Wed, Aug 17, 2016 at 4:16 PM, Andres Freund wrote: >> Does anyone have any ideas on how to: >> >> 1). Directly address the reform_and_rewrite_tuple() bottleneck. > > What part of is actually the expensive bit? It does a whole lot of > things. Forming/Deforming tuples, the

Re: [HACKERS] CLUSTER, reform_and_rewrite_tuple(), and parallelism

2016-08-17 Thread Andres Freund
> Does anyone have any ideas on how to: > > 1). Directly address the reform_and_rewrite_tuple() bottleneck. What part of is actually the expensive bit? It does a whole lot of things. Forming/Deforming tuples, the hash lookups in rewrite_heap_tuple(), ...? Andres -- Sent via pgsql-hackers