Re: [PERFORM] Can't get Dell PE T420 (Perc H710) perform better than a MacMini with PostgreSQL

2015-04-01 Thread Ilya Kosmodemiansky
-related and autovacuum-related settings. And as a first step, please compare postgresql.conf on Mac and on the server: sometimes (with some mac installers) default postgresql.conf can be not the same as on server. Best regards, Ilya -- Ilya Kosmodemiansky, PostgreSQL-Consulting.com tel

Re: [PERFORM] Very slow checkpoints

2015-03-18 Thread Ilya Kosmodemiansky
, %iowait and %util, such parameters are very helpful. And I am always suspicious about zfs under heavy writes. It is reliable and quite comfortable in terms of configuration, but for speed ext4 or xfs with disabled barrier looks more reasonable -- Ilya Kosmodemiansky, PostgreSQL-Consulting.com

Re: [PERFORM] Very slow checkpoints

2015-03-18 Thread Ilya Kosmodemiansky
(vacuuming has its price). autovacuum_vacuum_scale_factor = 0.01 and autovacuum_analyze_scale_factor = 0.05 will be OK And if you see all your autovacuum workers active all the time (more than 80% of the time for example) it is a reason to increase autovacuum_max_workers -- Ilya Kosmodemiansky

Re: [PERFORM] Very slow checkpoints

2015-03-18 Thread Ilya Kosmodemiansky
keeps the database stable And an important addition: how your autovacuum is configured? -- Ilya Kosmodemiansky, PostgreSQL-Consulting.com tel. +14084142500 cell. +4915144336040 i...@postgresql-consulting.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org

Re: [PERFORM] Very slow checkpoints

2015-03-18 Thread Ilya Kosmodemiansky
Values for this settings are really dependent of RAID (and BBU size). And about further problem description: have you any graphical representation of your % disc utilization? Best regards, Ilya -- Ilya Kosmodemiansky, PostgreSQL-Consulting.com tel. +14084142500 cell. +4915144336040 i

Re: [PERFORM] MusicBrainz postgres performance issues

2015-03-15 Thread Ilya Kosmodemiansky
On Mar 15, 2015, at 13:45, Josh Krupka jkru...@gmail.com wrote: Hmm that's definitely odd that it's swapping since it has plenty of free memory at the moment. Is it still under heavy load right now? Has the output of free consistently looked like that during your trouble times? And it

Re: [PERFORM] MusicBrainz postgres performance issues

2015-03-15 Thread Ilya Kosmodemiansky
Hi! What shows your pg_stat_bgwriter for one day? On Mar 15, 2015, at 11:54, Robert Kaye r...@musicbrainz.org wrote: Hi! We at MusicBrainz have been having trouble with our Postgres install for the past few days. I’ve collected all the relevant information here:

Re: [PERFORM] MusicBrainz postgres performance issues

2015-03-15 Thread Ilya Kosmodemiansky
, 24x7 Support, Training Services -- Ilya Kosmodemiansky, PostgreSQL-Consulting.com tel. +14084142500 cell. +4915144336040 i...@postgresql-consulting.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http

Re: [PERFORM] MusicBrainz postgres performance issues

2015-03-15 Thread Ilya Kosmodemiansky
work mem AND max_connections = 500 is a recipe for disaster. The problem could be in session mode of pgbouncer. If you can work with transaction mode - do it. Best regards, Ilya Kosmodemiansky, PostgreSQL-Consulting.com tel. +14084142500 cell. +4915144336040 i...@postgresql-consulting.com

Re: [PERFORM] Server vacuuming the same table again and again

2014-04-25 Thread Ilya Kosmodemiansky
? Best regards, Ilya Best regards, Dmitriy Shalashov -- Ilya Kosmodemiansky, PostgreSQL-Consulting.com tel. +14084142500 cell. +4915144336040 i...@postgresql-consulting.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http

Re: [PERFORM] Server vacuuming the same table again and again

2014-04-25 Thread Ilya Kosmodemiansky
| 10 autovacuum_vacuum_cost_limit| -1 autovacuum_vacuum_scale_factor | 0.01 autovacuum_vacuum_threshold | 10 log_autovacuum_min_duration | -1 Best regards, Ilya Best regards, Dmitriy Shalashov 2014-04-25 12:12 GMT+04:00 Ilya Kosmodemiansky

Re: [PERFORM] Server vacuuming the same table again and again

2014-04-25 Thread Ilya Kosmodemiansky
for detecting slow queries? Best regards, Dmitriy Shalashov 2014-04-25 13:22 GMT+04:00 Ilya Kosmodemiansky ilya.kosmodemian...@postgresql-consulting.com: On Fri, Apr 25, 2014 at 10:29 AM, Дмитрий Шалашов skau...@gmail.com wrote: Previously during load disk was 100% busy; now we have

Re: [PERFORM] PGSQL, checkpoints, and file system syncs

2014-04-04 Thread Ilya Kosmodemiansky
starting point. -- Ilya Kosmodemiansky, PostgreSQL-Consulting.com tel. +14084142500 cell. +4915144336040 i...@postgresql-consulting.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql

Re: [PERFORM] Why shared_buffers max is 8GB?

2014-03-26 Thread Ilya Kosmodemiansky
% of RAM. Best regards, Ilya -- Ilya Kosmodemiansky, PostgreSQL-Consulting.com tel. +14084142500 cell. +4915144336040 i...@postgresql-consulting.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [PERFORM] Why shared_buffers max is 8GB?

2014-03-26 Thread Ilya Kosmodemiansky
checkpoint_segments | 300 checkpoint_timeout | 3600 and it really makes sense -- Ilya Kosmodemiansky, PostgreSQL-Consulting.com tel. +14084142500 cell. +4915144336040 i...@postgresql-consulting.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make

Re: [PERFORM] slow join not using index properly

2014-03-25 Thread Ilya Kosmodemiansky
:02 PM, Ilya Kosmodemiansky ilya.kosmodemian...@postgresql-consulting.com wrote: Hi Stefan! Probably you need to rewrite your query like this (check it first): with RECURSIVE qq(cont_key, anc_key) as ( select min(a1.context_key), ancestor_key from virtual_ancestors a1 union select

Re: [PERFORM] pg_dump vs pg_basebackup

2014-03-25 Thread Ilya Kosmodemiansky
. Best regards, Ilya On Tue, Mar 25, 2014 at 5:45 AM, gianfranco caca limpc...@yahoo.com wrote: Hai, Can anyone tell me the difference and performance between pgdump and pg_basebackup if I want to backup a large database. Thanks -- Ilya Kosmodemiansky, PostgreSQL-Consulting.com tel

Re: [PERFORM] pg_dump vs pg_basebackup

2014-03-25 Thread Ilya Kosmodemiansky
on transaction time? Thanks On Tuesday, 25 March 2014, 15:13, Ilya Kosmodemiansky ilya.kosmodemian...@postgresql-consulting.com wrote: Hi gianfranco, How exactly large is your database and how heavy is a workload on it? Usually if you have more than ~200Gb, better to use pg_basebackup because pg_dump

Re: [PERFORM] pg_dump vs pg_basebackup

2014-03-25 Thread Ilya Kosmodemiansky
-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance -- Ilya Kosmodemiansky, PostgreSQL-Consulting.com tel. +14084142500 cell. +4915144336040 i...@postgresql-consulting.com -- Sent via pgsql-performance mailing list (pgsql

Re: [PERFORM] pg_dump vs pg_basebackup

2014-03-25 Thread Ilya Kosmodemiansky
, Professional Services and Development High Availability, Oracle Conversion, Postgres-XC, @cmdpromptinc Political Correctness is for cowards. -- Ilya Kosmodemiansky, PostgreSQL-Consulting.com tel. +14084142500 cell. +4915144336040 i...@postgresql-consulting.com -- Sent via pgsql-performance mailing

Re: [PERFORM] pg_dump vs pg_basebackup

2014-03-25 Thread Ilya Kosmodemiansky
with pg_start/stop_backup(). There are definitely reasons you'd prefer rsync over pg_basebackup, but I don't believe simplicity is one of them. //Magnus On Tue, Mar 25, 2014 at 4:18 PM, Ilya Kosmodemiansky ilya.kosmodemian...@postgresql-consulting.com wrote: Joshua, that is really good

Re: [PERFORM] pg_dump vs pg_basebackup

2014-03-25 Thread Ilya Kosmodemiansky
to choose the *other* method (pg_basebackup), and the rsync method is for more advanced usecases. But it's definitely good to know both! //Magnus On Tue, Mar 25, 2014 at 4:37 PM, Ilya Kosmodemiansky ilya.kosmodemian...@postgresql-consulting.com wrote: Magnus, That is correct, but I'am

Re: [PERFORM] slow join not using index properly

2014-03-21 Thread Ilya Kosmodemiansky
to use the optimal plan? Thanks in advance! /Stefan -- - Stefan Amshey -- Ilya Kosmodemiansky, PostgreSQL-Consulting.com tel. +14084142500 cell. +4915144336040 i...@postgresql-consulting.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make

Re: [PERFORM] Problem with ExclusiveLock on inserts

2014-02-19 Thread Ilya Kosmodemiansky
, Бородин Владимир r...@simply.name wrote: 13.02.2014, в 13:29, Ilya Kosmodemiansky hydrobi...@gmail.com написал(а): Vladimir, And, any effect on your problem? It worked without problems longer than previous configuration but repeated again several minutes ago :( On Thu, Feb 13, 2014

Re: [PERFORM] Problem with ExclusiveLock on inserts

2014-02-13 Thread Ilya Kosmodemiansky
parameters to turn it back. But yesterday I ran vacuum analyze for all relations manually but that did not help. How do your autovacuum parameters look like now? 13.02.2014, в 0:14, Ilya Kosmodemiansky hydrobi...@gmail.com написал(а): On Wed, Feb 12, 2014 at 8:57 PM, Бородин Владимир r...@simply.name

Re: [PERFORM] Problem with ExclusiveLock on inserts

2014-02-13 Thread Ilya Kosmodemiansky
, Ilya Kosmodemiansky ilya.kosmodemian...@postgresql-consulting.com wrote: Vladimir, pgbouncer works with pl/proxy in transaction pooling mode. A wide spread phrase that statement mode is for plproxy does not mean any limitations for transaction pooling mode until you have atocommit on client

Re: [PERFORM] Problem with ExclusiveLock on inserts

2014-02-12 Thread Ilya Kosmodemiansky
Hi Vladimir, Just in case: how is your ext4 mount? Best regards, Ilya On Feb 12, 2014, at 17:59, Бородин Владимир r...@simply.name wrote: Hi all. Today I have started getting errors like below in logs (seems that I have not changed anything for last week). When it happens the db gets

Re: [PERFORM] Problem with ExclusiveLock on inserts

2014-02-12 Thread Ilya Kosmodemiansky
data /etc/fstab UUID=f815fd3f-e4e4-43a6-a6a1-bce1203db3e0 /var/lib/pgsql/9.3/data ext4 noatime,nodiratime 0 1 root@rpopdb01e ~ # According to iostat the disks are not the bottleneck. 12.02.2014, в 21:30, Ilya Kosmodemiansky hydrobi...@gmail.com написал(а): Hi Vladimir, Just in case

Re: [PERFORM] Problem with ExclusiveLock on inserts

2014-02-12 Thread Ilya Kosmodemiansky
type ext4 (rw,noatime,nodiratime,nobarrier) root@rpopdb01e ~ # 12.02.2014, в 21:56, Ilya Kosmodemiansky hydrobi...@gmail.com написал(а): My question was actually about barrier option, by default it is enabled on RHEL6/ext4 and could cause serious bottleneck on io before disks are actually

Re: [PERFORM] Problem with ExclusiveLock on inserts

2014-02-12 Thread Ilya Kosmodemiansky
checkpoint. By default it usually happens too often because checkpoint_timeout is 5min by default. Without bbu, on software raid that leads to io spike and you commit waits for wal. 12.02.2014, в 23:37, Ilya Kosmodemiansky hydrobi...@gmail.com написал(а): another thing which is arguable