Re: Autoanalyze CPU usage

2017-12-20 Thread Habib Nahas
Thanks for confirming that it is the end timestamp, the doc wasn't quite clear if it was the start or end. There is a gap in our monitoring that makes diagnosis of such events very difficult after the fact. Something like a 10-sec periodic dump of pg_stat_activity along with a similar dump of pg_t

Re: Autoanalyze CPU usage

2017-12-20 Thread Nikolay Samokhvalov
On Tue, Dec 19, 2017 at 7:47 PM, Habib Nahas wrote: > Hi, > > We operate an RDS postgres 9.5 instance and have periodic CPU spikes to > 100%. These spikes appear to be due to autoanalyze kicking on our larger > tables. > How did you draw such conclusion? How did you find that autoanalyze is the

Re: Autoanalyze CPU usage

2017-12-19 Thread Laurenz Albe
Habib Nahas wrote: > The CPU spike occurred between 13:05 - 13:15. last_autoanalyze for the table > shows a time of 12:49; last_autovacuum does not show any activity around > this time for any table. Checkpoint logs are also normal around this time. > I'd like to understand if there are any other s

Re: Autoanalyze CPU usage

2017-12-19 Thread michael...@sqlexec.com
Perhaps consider running manual vacuum analyze at low load times daily if you have that opportunity. This may stop autovacuums from hitting thresholds during high load times or do the normal/aggressive autovacuum tuning to make it more aggressive during low load times and less aggressive during

Re: Autoanalyze CPU usage

2017-12-19 Thread Justin Pryzby
On Tue, Dec 19, 2017 at 02:37:18PM -0800, Habib Nahas wrote: > As it happens our larger tables operate as a business log and are also > insert only. > > - There is no partitioning at this time since we expect to have an > automated process to delete rows older than a certain date. This is a prima

Re: Autoanalyze CPU usage

2017-12-19 Thread Habib Nahas
The autoanalyze factor is set to 0.05 for the db, and we have not changed the default statistics target. The CPU spike occurred between 13:05 - 13:15. last_autoanalyze for the table shows a time of 12:49; last_autovacuum does not show any activity around this time for any table. Checkpoint logs ar

Re: Autoanalyze CPU usage

2017-12-19 Thread Habib Nahas
As it happens our larger tables operate as a business log and are also insert only. - There is no partitioning at this time since we expect to have an automated process to delete rows older than a certain date. - Analyzing doing off-hours sounds like a good idea; if there is no other way to determ

Re: Autoanalyze CPU usage

2017-12-19 Thread Tomas Vondra
On 12/19/2017 05:47 PM, Habib Nahas wrote: > Hi, > > We operate an RDS postgres 9.5 instance and have periodic CPU spikes to > 100%. These spikes appear to be due to autoanalyze kicking on our larger > tables. > > Our largest table has 75 million rows and the autoanalyze scale factor > is set t

Re: Autoanalyze CPU usage

2017-12-19 Thread Justin Pryzby
On Tue, Dec 19, 2017 at 08:47:52AM -0800, Habib Nahas wrote: > We operate an RDS postgres 9.5 instance and have periodic CPU spikes to > 100%. These spikes appear to be due to autoanalyze kicking on our larger > tables. Not sure if it'll help you, but for our large, insert-only tables partitioned