[PERFORM] Re: autovacuum vacuum creates bad statistics for planner when it log index scans: 0

2014-06-05 Thread tim_wilson
I have now created a repeatable test for this ...bug, well that may be debatable, but getting the query plan this wrong after vacum and analyze have run certainly looks like a bug to me. I have created a test case that matches my problem domain but can probably be simplified. postgres_bug.sql

Re: [PERFORM] Re: autovacuum vacuum creates bad statistics for planner when it log index scans: 0

2014-06-05 Thread Tom Lane
tim_wilson writes: > Thanks for you response Tom: > but what does index_scans:0 mean? vs index scans: 1? I believe the former means that VACUUM found no removable tuples, so it had no need to make any passes over the table's indexes. (Ordinarily you wouldn't see the number of scans as more than

[PERFORM] Re: autovacuum vacuum creates bad statistics for planner when it log index scans: 0

2014-06-05 Thread tim_wilson
Thanks for you response Tom: but what does index_scans:0 mean? vs index scans: 1? I have had a look at the c code but cannot see when it that would be the case. -- View this message in context: http://postgresql.1045698.n5.nabble.com/autovacuum-vacuum-creates-bad-statistics-for-planner-when-it

Re: [PERFORM] CPU load spikes when CentOS tries to reclaim 'cached' memory

2014-06-05 Thread Vincent Lasmarias
Thanks for the informative responses and suggestions. My responses below: * Sorry for the double post. I posted the original message using my gmail account and got a "is not a member of any of the restrict_post groups" response and when I didn't see it for a day, I ended up wondering if it was due

[PERFORM] postgres files in use not staying in linux file cache

2014-06-05 Thread Brio
Hi, I'm trying to investigate a performance problem. We have a large database (over 1TB) running on a server with 160GB of RAM and 32 cores (Xeon E5-2650). The database files are on a NetApp mount. The software is Postgres 9.3.1 on Ubuntu 12.04, Linux 3.2.0-38-generic. Generally, when a query is

Re: [PERFORM] Seqscan on big table, when an Index-Usage should be possible

2014-06-05 Thread Igor Neyman
Stefan, See below > -Original Message- > From: pgsql-performance-ow...@postgresql.org [mailto:pgsql- > performance-ow...@postgresql.org] On Behalf Of Weinzierl Stefan > Sent: Thursday, June 05, 2014 3:36 PM > To: pgsql-performance@postgresql.org > Subject: [PERFORM] Seqscan on big table,

Re: [PERFORM] High CPU load when 'free -m' shows low 'free' memory even though large 'cached' memory still available

2014-06-05 Thread Merlin Moncure
On Thu, Jun 5, 2014 at 8:47 AM, Tom Lane wrote: > Vince Lasmarias writes: >> For the past few days, we've been seeing unexpected high CPU spikes in our >> system. > > Recent reports have suggested that disabling transparent huge page > management in your kernel can help with this. If the excess

Re: [PERFORM] CPU load spikes when CentOS tries to reclaim 'cached' memory

2014-06-05 Thread Merlin Moncure
On Thu, Jun 5, 2014 at 2:47 PM, Deron wrote: > We saw very similar issues with a CentOS server with 40 cores (32 > virtualized) when moving from a physical server to a virtual server (I think > it had 128GB RAM). Never had the problem on a physical server. We checked > the same things as noted

Re: [PERFORM] CPU load spikes when CentOS tries to reclaim 'cached' memory

2014-06-05 Thread Deron
We saw very similar issues with a CentOS server with 40 cores (32 virtualized) when moving from a physical server to a virtual server (I think it had 128GB RAM). Never had the problem on a physical server. We checked the same things as noted here, but never found a bug. We really thought it ha

[PERFORM] Seqscan on big table, when an Index-Usage should be possible

2014-06-05 Thread Weinzierl Stefan
Hello, I'm currently testing some queries on data which I had imported from an other database-system into Postgres 9.4. After the import I did create the indexes, run an analyze and vacuum. I also played a little bit with seq_page_cost and random_page_cost. But currently I have no clue, whic

Re: [PERFORM] CPU load spikes when CentOS tries to reclaim 'cached' memory

2014-06-05 Thread Merlin Moncure
On Thu, Jun 5, 2014 at 10:58 AM, Jeff Janes wrote: > This sounds like a kernel problem, probably either the zone reclaim issue, > or the transparent huge pages issue. I at first thought maybe same, but I don't think THP was introduced until 2.6.38...OP is running 2.6.32-431.11.2.el6.x86_6. Maybe

Re: [PERFORM] CPU load spikes when CentOS tries to reclaim 'cached' memory

2014-06-05 Thread Jeff Janes
On Wed, Jun 4, 2014 at 5:27 PM, vlasmarias wrote: > For the past few days, we've been seeing unexpected extremely high CPU > spikes > in our system. We observed the following: the 'free' memory would go down > to > lower than 300 MB; at that point, 'cached' slowly starts to go down, and > then CP

Re: [PERFORM] High CPU load when 'free -m' shows low 'free' memory even though large 'cached' memory still available

2014-06-05 Thread Tom Lane
Vince Lasmarias writes: > For the past few days, we've been seeing unexpected high CPU spikes in our > system. Recent reports have suggested that disabling transparent huge page management in your kernel can help with this. If the excess CPU load is mostly "system" time not "user" time then this

Re: [PERFORM] Possible performance regression in PostgreSQL 9.2/9.3?

2014-06-05 Thread Linos
On 05/06/14 15:29, Igor Neyman wrote: >> -Original Message- >> From: pgsql-performance-ow...@postgresql.org [mailto:pgsql- >> performance-ow...@postgresql.org] On Behalf Of Linos >> Sent: Wednesday, June 04, 2014 6:10 PM >> To: Merlin Moncure >> Cc: pgsql-performance@postgresql.org >> Subje

Re: [PERFORM] Possible performance regression in PostgreSQL 9.2/9.3?

2014-06-05 Thread Igor Neyman
> -Original Message- > From: pgsql-performance-ow...@postgresql.org [mailto:pgsql- > performance-ow...@postgresql.org] On Behalf Of Linos > Sent: Wednesday, June 04, 2014 6:10 PM > To: Merlin Moncure > Cc: pgsql-performance@postgresql.org > Subject: Re: [PERFORM] Possible performance regres