Re: [PERFORM] : Performance Improvement Strategy

2011-10-10 Thread Leonardo Francalanci
       * Allow CLUSTER to sort the table rather than scanning the index  when it seems likely to be cheaper (Leonardo Francalanci) Looks like I owe Leonardo Francalanci a pizza. Well, the patch started from a work by Gregory Stark, and Tom fixed a nasty bug; but I'll take a slice ;)

[PERFORM] Adding more memory = hugh cpu load

2011-10-10 Thread alexandre - aldeia digital
Hi, Yesterday, a customer increased the server memory from 16GB to 48GB. Today, the load of the server hit 40 ~ 50 points. With 16 GB, the load not surpasses 5 ~ 8 points. The only parameter that I changed is effective_cache_size (from 14 GB to 40GB) and shared_buffers (from 1 GB to 5 GB).

Re: [PERFORM] Adding more memory = hugh cpu load

2011-10-10 Thread Kevin Grittner
alexandre - aldeia digital adald...@gmail.com wrote: Yesterday, a customer increased the server memory from 16GB to 48GB. That's usually for the better, but be aware that on some hardware adding RAM beyond a certain point causes slower RAM access. Without knowing more details, it's

Re: [PERFORM] Adding more memory = hugh cpu load

2011-10-10 Thread Shaun Thomas
On 10/10/2011 08:26 AM, alexandre - aldeia digital wrote: Yesterday, a customer increased the server memory from 16GB to 48GB. Today, the load of the server hit 40 ~ 50 points. With 16 GB, the load not surpasses 5 ~ 8 points. That's not entirely surprising. The problem with having lots of

Re: [PERFORM] Adding more memory = hugh cpu load

2011-10-10 Thread Leonardo Francalanci
That's not entirely surprising. The problem with having lots of memory is... that you have lots of memory. The operating system likes to cache, and this includes writes. Normally this isn't a problem, but with 48GB of RAM, the defaults (for CentOS 5.5 in particular) are to use up to 40%

Re: [PERFORM] Adding more memory = hugh cpu load

2011-10-10 Thread Greg Smith
On 10/10/2011 10:04 AM, Shaun Thomas wrote: The problem with having lots of memory is... that you have lots of memory. The operating system likes to cache, and this includes writes. Normally this isn't a problem, but with 48GB of RAM, the defaults (for CentOS 5.5 in particular) are to use up

Re: [PERFORM] Adding more memory = hugh cpu load

2011-10-10 Thread Leonardo Francalanci
Then the  database makes the fsync call, and suddenly the OS wants to flush 2-6GB of data straight to disk. Without that background trickle, you now have a flood that only the highest-end disk controller or a backing-store full of SSDs or PCIe NVRAM could ever hope to absorb. Isn't 

Re: [PERFORM] Adding more memory = hugh cpu load

2011-10-10 Thread alexandre - aldeia digital
Em 10-10-2011 11:04, Shaun Thomas wrote: That's not entirely surprising. The problem with having lots of memory is... that you have lots of memory. The operating system likes to cache, and this includes writes. Normally this isn't a problem, but with 48GB of RAM, the defaults (for CentOS 5.5 in

Re: [PERFORM] Adding more memory = hugh cpu load

2011-10-10 Thread Kevin Grittner
alexandre - aldeia digital adald...@gmail.com wrote: Notice that we have no idle % in cpu column. So they're making full use of all the CPUs they paid for. That in itself isn't a problem. Unfortunately you haven't given us nearly enough information to know whether there is indeed a problem,

Re: [PERFORM] Adding more memory = hugh cpu load

2011-10-10 Thread alexandre - aldeia digital
Em 10-10-2011 14:46, Kevin Grittner escreveu: alexandre - aldeia digitaladald...@gmail.com wrote: Notice that we have no idle % in cpu column. So they're making full use of all the CPUs they paid for. That in itself isn't a problem. Unfortunately you haven't given us nearly enough

Re: [PERFORM] Adding more memory = hugh cpu load

2011-10-10 Thread Kevin Grittner
alexandre - aldeia digital adald...@gmail.com wrote: From the point of view of the client, the question is simple: until the last friday (with 16 GB of RAM), the load average of server rarely surpasses 4. Nothing change in normal database use. Really? The application still performs as well

Re: [PERFORM] Adding more memory = hugh cpu load

2011-10-10 Thread Shaun Thomas
On 10/10/2011 12:31 PM, alexandre - aldeia digital wrote: 2011-10-10 14:18:48 BRT LOG: checkpoint complete: wrote 6885 buffers (1.1%); 0 transaction log file(s) added, 0 removed, 1 recycled; write=29.862 s, sync=28.466 s, total=58.651 s 28.466s sync time?! That's horrifying. At this point, I

Re: [PERFORM] Adding more memory = hugh cpu load

2011-10-10 Thread alexandre - aldeia digital
Em 10-10-2011 16:39, Kevin Grittner escreveu: alexandre - aldeia digitaladald...@gmail.com wrote: From the point of view of the client, the question is simple: until the last friday (with 16 GB of RAM), the load average of server rarely surpasses 4. Nothing change in normal database use.

Re: [PERFORM] Adding more memory = hugh cpu load

2011-10-10 Thread Kevin Grittner
alexandre - aldeia digital adald...@gmail.com wrote: I came to the list to see if anyone else has experienced the same problem A high load average or low idle CPU isn't a problem, it's a potentially useful bit of information in diagnosing a problem. I was hoping to hear what the actual

Re: [PERFORM] Adding more memory = hugh cpu load

2011-10-10 Thread Samuel Gendler
On Mon, Oct 10, 2011 at 1:52 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: alexandre - aldeia digital adald...@gmail.com wrote: I came to the list to see if anyone else has experienced the same problem A high load average or low idle CPU isn't a problem, it's a potentially

Re: [PERFORM] Adding more memory = hugh cpu load

2011-10-10 Thread Claudio Freire
On Tue, Oct 11, 2011 at 12:02 AM, Samuel Gendler sgend...@ideasculptor.com wrote: The original question doesn't actually say that performance has gone down, only that cpu utilization has gone up. Presumably, with lots more RAM, it is blocking on I/O a lot less, so it isn't necessarily