Re: [HACKERS] 8.4 open item: copy performance regression?

2009-06-20 Thread Greg Smith
On Sat, 20 Jun 2009, Simon Riggs wrote: At the time, I also proposed a filled buffer list change to bufmgr to allow bgwriter to preferentially target COPY's filled blocks, which would also help with this effect. One of the things I keep meaning to investigate is whether there's any benefit

Re: [HACKERS] 8.4 open item: copy performance regression?

2009-06-20 Thread Simon Riggs
On Sat, 2009-06-20 at 02:53 -0400, Greg Smith wrote: On Sat, 20 Jun 2009, Simon Riggs wrote: At the time, I also proposed a filled buffer list change to bufmgr to allow bgwriter to preferentially target COPY's filled blocks, which would also help with this effect. One of the things I

Re: [HACKERS] 8.4 open item: copy performance regression?

2009-06-20 Thread Simon Riggs
On Fri, 2009-06-19 at 22:03 -0400, Greg Smith wrote: This makes me wonder if in addition to the ring buffering issue, there isn't just plain more writing per average completed transaction in 8.4 with this type of COPY. I would suggest that we check how much WAL has been written. There may be

Re: [HACKERS] Synch Rep: communication between backends and walsender

2009-06-20 Thread Markus Wanner
Hi, Fujii Masao wrote: One of the major complaints about the current synch rep patch is that signals are used for communication between backends and walsender. On some platforms, a signal doesn't interrupt sleep (i.e. poll or select system call), which would increase the performance overhead

Re: [HACKERS] 8.4 open item: copy performance regression?

2009-06-20 Thread Greg Stark
On Sat, Jun 20, 2009 at 9:22 AM, Simon Riggssi...@2ndquadrant.com wrote: That would seem to me to be a more robust general approach for solving this class of problem than the whole ring buffer idea, which is a great start but bound to run into situations where the size of the buffer just isn't

Re: [HACKERS] 8.4 open item: copy performance regression?

2009-06-20 Thread Stefan Kaltenbrunner
Greg Smith wrote: On Fri, 19 Jun 2009, Stefan Kaltenbrunner wrote: In my case both the CPU (an Intel E5530 Nehalem) and the IO subsystem (8GB Fiberchannel connected NetApp with 4GB cache) are pretty fast. The server Alan identified as Solaris 10 8/07 s10x_u4wos_12b X86 has a Xeon E5320

Re: [HACKERS] 8.4 open item: copy performance regression?

2009-06-20 Thread Greg Stark
On Sat, Jun 20, 2009 at 12:10 PM, Greg Starkgsst...@mit.edu wrote: I don't understand what you mean by size of the buffer either. Ok, having gone back and read the whole thread I understand the context for that statement. Nevermind. -- greg http://mit.edu/~gsstark/resume.pdf -- Sent via

Re: [HACKERS] 8.4 open item: copy performance regression?

2009-06-20 Thread Tom Lane
Stefan Kaltenbrunner ste...@kaltenbrunner.cc writes: and I believe I have seen the more IO with 8.4 thing here too but I have not actually paid enough attention yet to be sure. FSM/VM overhead maybe? I think COPY IN is setting the SKIP_FSM bit, but I wonder if there's some vestigial

Re: [HACKERS] 8.4 open item: copy performance regression?

2009-06-20 Thread Simon Riggs
On Sat, 2009-06-20 at 13:15 +0200, Stefan Kaltenbrunner wrote: 8.3.7: 0m39.266s 0m43.269s (alan: 36.2 - 39.2) 8192: 0m40.715s 0m42.480s 16384: 0m41.318s 0m42.118s 65536: 0m41.675s 0m42.955s hmm interesting - I just did a bunch of runs using the lineitem table from

Re: [HACKERS] 8.4 open item: copy performance regression?

2009-06-20 Thread Greg Smith
On Sat, 20 Jun 2009, Simon Riggs wrote: The reason for not doing that would be that we don't know that the blocks are free to use; we know very little about them. The longer we leave them the more likely they are to be reused, so putting buffers onto the freelist when they aren't actually free

[HACKERS] array_agg versus repeated execution of aggregate final functions

2009-06-20 Thread Tom Lane
I've identified the cause of Merlin Moncure's crash report here: http://archives.postgresql.org/pgsql-hackers/2009-06/msg01171.php (Thanks to Merlin for making some proprietary stuff available to me for testing it.) The problem query generates a plan in which a HashAggregate node is on the inside

Re: [HACKERS] array_agg versus repeated execution of aggregate final functions

2009-06-20 Thread Merlin Moncure
On Sat, Jun 20, 2009 at 1:31 PM, Tom Lanet...@sss.pgh.pa.us wrote: I've identified the cause of Merlin Moncure's crash report here: http://archives.postgresql.org/pgsql-hackers/2009-06/msg01171.php (Thanks to Merlin for making some proprietary stuff available to me for testing it.) The

Re: [HACKERS] array_agg versus repeated execution of aggregate final functions

2009-06-20 Thread Tom Lane
Merlin Moncure mmonc...@gmail.com writes: On Sat, Jun 20, 2009 at 1:31 PM, Tom Lanet...@sss.pgh.pa.us wrote: There are basically two ways that we could respond to this: (1) Decide that the bug is array_agg_finalfn's, and make the necessary fixes so that it doesn't modify or free its argument

Re: [HACKERS] 8.4 open item: copy performance regression?

2009-06-20 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On Sat, 2009-06-20 at 13:15 +0200, Stefan Kaltenbrunner wrote: 8192 6m43.203s/6m48.293s 16384 6m24.980s/6m24.116s 32768 6m20.753s/6m22.083s 65536 6m22.913s/6m22.449s 1048576 6m23.765s/6m24.645s The rest of the patch should have had a greater effect