Re: [GENERAL] Strange checkpoint behavior - checkpoints take alongtime

2017-10-10 Thread Vladimir Nicolici
In fact it was a single delete statement. From: Vladimir Nicolici Sent: Tuesday, October 10, 2017 17:30 To: Achilleas Mantzios; pgsql-general@postgresql.org Subject: RE: [GENERAL] Strange checkpoint behavior - checkpoints take alongtime No, it didn’t. The delete was done in a single transaction.

Re: [GENERAL] Strange checkpoint behavior - checkpoints take a longtime

2017-10-10 Thread Vladimir Nicolici
those tables ? Default autovacuum_freeze_max_age is exactly set at 200,000,000 . Did you check your vacuum stats afterwards (pg_stat_*_tables) ? Can you show the code which performed the deletes? On 10/10/2017 16:56, Vladimir Nicolici wrote: I experimented some more with the settings this

Re: [GENERAL] Strange checkpoint behavior - checkpoints take a long time

2017-10-10 Thread Vladimir Nicolici
I experimented some more with the settings this weekend, while doing some large write operations (deleting 200 million records from a table), and I realized that the database is capable of generating much more WAL than I estimated. And it seems that spikes in write activity, when longer than a f

Re: [GENERAL] Strange checkpoint behavior - checkpoints take a long time

2017-10-06 Thread Vladimir Nicolici
Further updates: Yesterday checkpoints were finishing more or less on time with the configuration for 25 minutes out of 30 minutes, taking 26 minutes at most. So for today I reduced the time reserved for checkpoint writes to 20 minutes out of 30 minutes, by setting checkpoint_completion_target

Re: [GENERAL] Strange checkpoint behavior - checkpoints take a longtime

2017-10-05 Thread Vladimir Nicolici
: Friday, October 6, 2017 04:51 To: Vladimir Nicolici Cc: pgsql-general@postgresql.org Subject: Re: [GENERAL] Strange checkpoint behavior - checkpoints take a longtime Hi, On 2017-10-05 22:58:31 +0300, Vladimir Nicolici wrote: > I changed some configuration parameters during the night to the values I

Re: [GENERAL] Strange checkpoint behavior - checkpoints take a longtime

2017-10-05 Thread Vladimir Nicolici
mbination, I will probably set it to something like 0.90 target, so that it distributes the writes over 27 minutes. Thanks, Vlad From: Igor Polishchuk Sent: Friday, October 6, 2017 02:56 To: Vladimir Nicolici Cc: pgsql-general@postgresql.org Subject: Re: [GENERAL] Strange checkpoint behavior - chec

Re: [GENERAL] Strange checkpoint behavior - checkpoints take a long time

2017-10-05 Thread Vladimir Nicolici
Some further updates about the issue. I did a bit of benchmarking on the disk system with iozone, and the during the test the SSDs seemed to be able to easily sustain 200 MB/second of writes each, they fluctuated between 200 MB/s and 400 MB/s when doing 96 GB of random writes in a file. That wo

[GENERAL] Strange checkpoint behavior - checkpoints take a long time

2017-10-04 Thread Vladimir Nicolici
I have a large database, 1.3 TB, with quite a bit of write activity. The machine has, 2 cpus x 6 cores x 2 threads (2 x E5-2630 v2 @ 2.60GHz), 4 x EVO Pro 2TB SSDs in a RAID 1+0 software raid configuration, on a SATA 3 controller. The machine has a lot of memory, 384 GB, so it doesn’t do a lot o