Re: [PERFORM] Can't get Dell PE T420 (Perc H710) perform better than a MacMini with PostgreSQL

2015-04-01 Thread Ilya Kosmodemiansky
ts-related and autovacuum-related settings. And as a first step, please compare postgresql.conf on Mac and on the server: sometimes (with some mac installers) default postgresql.conf can be not the same as on server. Best regards, Ilya -- Ilya Kosmodemiansky, PostgreSQL-Consulting.com tel. +14

Re: [PERFORM] Very slow checkpoints

2015-03-18 Thread Ilya Kosmodemiansky
too frequently (vacuuming has its price). autovacuum_vacuum_scale_factor = 0.01 and autovacuum_analyze_scale_factor = 0.05 will be OK And if you see all your autovacuum workers active all the time (more than 80% of the time for example) it is a reason to increase autovacuum_max_worke

Re: [PERFORM] Very slow checkpoints

2015-03-18 Thread Ilya Kosmodemiansky
the database stable And an important addition: how your autovacuum is configured? -- Ilya Kosmodemiansky, PostgreSQL-Consulting.com tel. +14084142500 cell. +4915144336040 i...@postgresql-consulting.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make cha

Re: [PERFORM] Very slow checkpoints

2015-03-18 Thread Ilya Kosmodemiansky
t and %util, such parameters are very helpful. And I am always suspicious about zfs under heavy writes. It is reliable and quite comfortable in terms of configuration, but for speed ext4 or xfs with disabled barrier looks more reasonable >> >> -- >> Ilya Kosmodemiansky, >

Re: [PERFORM] Very slow checkpoints

2015-03-18 Thread Ilya Kosmodemiansky
; vm.dirty_writeback_centisecs = 500 Values for this settings are really dependent of RAID (and BBU size). And about further problem description: have you any graphical representation of your % disc utilization? Best regards, Ilya -- Ilya Kosmodemiansky, PostgreSQL-Consulting.com tel. +

Re: [PERFORM] MusicBrainz postgres performance issues

2015-03-15 Thread Ilya Kosmodemiansky
om/ > PostgreSQL Development, 24x7 Support, Training & Services -- Ilya Kosmodemiansky, PostgreSQL-Consulting.com tel. +14084142500 cell. +4915144336040 i...@postgresql-consulting.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to yo

Re: [PERFORM] MusicBrainz postgres performance issues

2015-03-15 Thread Ilya Kosmodemiansky
ree with Scott - 64MB work mem AND max_connections = 500 is a recipe for disaster. The problem could be in session mode of pgbouncer. If you can work with transaction mode - do it. Best regards, Ilya Kosmodemiansky, PostgreSQL-Consulting.com tel. +14084142500 cell. +4915144336040 i...@postgresql-c

Re: [PERFORM] MusicBrainz postgres performance issues

2015-03-15 Thread Ilya Kosmodemiansky
> On Mar 15, 2015, at 13:45, Josh Krupka wrote: > Hmm that's definitely odd that it's swapping since it has plenty of free > memory at the moment. Is it still under heavy load right now? Has the > output of free consistently looked like that during your trouble times? And it seems better t

Re: [PERFORM] MusicBrainz postgres performance issues

2015-03-15 Thread Ilya Kosmodemiansky
Hi! What shows your pg_stat_bgwriter for one day? > On Mar 15, 2015, at 11:54, Robert Kaye wrote: > > Hi! > > We at MusicBrainz have been having trouble with our Postgres install for the > past few days. I’ve collected all the relevant information here: > > http://blog.musicbrainz.org/20

Re: [PERFORM] Server vacuuming the same table again and again

2014-04-25 Thread Ilya Kosmodemiansky
y other method for detecting slow queries? > > > Best regards, > Dmitriy Shalashov > > > 2014-04-25 13:22 GMT+04:00 Ilya Kosmodemiansky > : > >> On Fri, Apr 25, 2014 at 10:29 AM, Дмитрий Шалашов >> wrote: >> > Previously during load disk was 100% b

Re: [PERFORM] Server vacuuming the same table again and again

2014-04-25 Thread Ilya Kosmodemiansky
need to do as a first thing - configure your >>> autovacuum aggressively enough and then mayde ionice autovacuum >>> instead of mission critical ckeckpointer or bgwriter. >>> >>> Which exact values have you in the following settings: >>> >>>

Re: [PERFORM] Server vacuuming the same table again and again

2014-04-25 Thread Ilya Kosmodemiansky
| 10 autovacuum_vacuum_cost_limit| -1 autovacuum_vacuum_scale_factor | 0.01 autovacuum_vacuum_threshold | 10 log_autovacuum_min_duration | -1 Best regards, Ilya > Best regards, > Dmitriy Shalashov > > > 2014-04-25 12:12 GMT+04:00 Ilya Kosmodemians

Re: [PERFORM] Server vacuuming the same table again and again

2014-04-25 Thread Ilya Kosmodemiansky
egards, Ilya > > Best regards, > Dmitriy Shalashov -- Ilya Kosmodemiansky, PostgreSQL-Consulting.com tel. +14084142500 cell. +4915144336040 i...@postgresql-consulting.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your sub

Re: [PERFORM] PGSQL, checkpoints, and file system syncs

2014-04-03 Thread Ilya Kosmodemiansky
arting point. -- Ilya Kosmodemiansky, PostgreSQL-Consulting.com tel. +14084142500 cell. +4915144336040 i...@postgresql-consulting.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] Why shared_buffers max is 8GB?

2014-03-26 Thread Ilya Kosmodemiansky
t_segments | 300 checkpoint_timeout | 3600 and it really makes sense -- Ilya Kosmodemiansky, PostgreSQL-Consulting.com tel. +14084142500 cell. +4915144336040 i...@postgresql-consulting.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org

Re: [PERFORM] Why shared_buffers max is 8GB?

2014-03-26 Thread Ilya Kosmodemiansky
AM. Best regards, Ilya -- Ilya Kosmodemiansky, PostgreSQL-Consulting.com tel. +14084142500 cell. +4915144336040 i...@postgresql-consulting.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailp

Re: [PERFORM] pg_dump vs pg_basebackup

2014-03-25 Thread Ilya Kosmodemiansky
to choose the *other* method > (pg_basebackup), and the rsync method is for more advanced usecases. But > it's definitely good to know both! > > //Magnus > > > > On Tue, Mar 25, 2014 at 4:37 PM, Ilya Kosmodemiansky > wrote: >> >> Magnus, >> >&g

Re: [PERFORM] pg_dump vs pg_basebackup

2014-03-25 Thread Ilya Kosmodemiansky
eal with > pg_start/stop_backup(). > > There are definitely reasons you'd prefer rsync over pg_basebackup, but I > don't believe simplicity is one of them. > > //Magnus > > > On Tue, Mar 25, 2014 at 4:18 PM, Ilya Kosmodemiansky > wrote: >> >

Re: [PERFORM] pg_dump vs pg_basebackup

2014-03-25 Thread Ilya Kosmodemiansky
Training, Professional Services and Development > High Availability, Oracle Conversion, Postgres-XC, @cmdpromptinc > Political Correctness is for cowards. -- Ilya Kosmodemiansky, PostgreSQL-Consulting.com tel. +14084142500 cell. +4915144336040 i...@postgresql-consulting.com -- Sent via

Re: [PERFORM] pg_dump vs pg_basebackup

2014-03-25 Thread Ilya Kosmodemiansky
cle Conversion, Postgres-XC, @cmdpromptinc > Political Correctness is for cowards. > > > > -- > Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-performance -- Ilya

Re: [PERFORM] pg_dump vs pg_basebackup

2014-03-25 Thread Ilya Kosmodemiansky
ransaction time? > > Thanks > > > On Tuesday, 25 March 2014, 15:13, Ilya Kosmodemiansky > wrote: > Hi gianfranco, > > > How exactly large is your database and how heavy is a workload on it? > Usually if you have more than ~200Gb, better to use pg_basebackup > because

Re: [PERFORM] pg_dump vs pg_basebackup

2014-03-25 Thread Ilya Kosmodemiansky
. Best regards, Ilya On Tue, Mar 25, 2014 at 5:45 AM, gianfranco caca wrote: > Hai, > > Can anyone tell me the difference and performance between pgdump and > pg_basebackup if I want to backup a large database. > > Thanks -- Ilya Kosmodemiansky, PostgreSQL-Consulting.com tel.

Re: [PERFORM] slow join not using index properly

2014-03-25 Thread Ilya Kosmodemiansky
o > add a GROUP BY clause, and doing that changed the final number of rows > selected: >> >> ERROR: column "a1.ancestor_key" must appear in the GROUP BY clause or be >> used in an aggregate function >> LINE 4: min( a1.context_key ), ancestor_key

Re: [PERFORM] slow join not using index properly

2014-03-20 Thread Ilya Kosmodemiansky
n_data_context_key_index on > collection_data (cost=0.00..79.24 rows=56 width=8) (actual > time=0.004..0.005 rows=1 loops=1954) > Index Cond: (collection_context_key = a2.context_key) > Buffers: shared hit=6132 > -> Index Only Scan usi

Re: [PERFORM] Problem with ExclusiveLock on inserts

2014-02-19 Thread Ilya Kosmodemiansky
, Бородин Владимир wrote: > > 13.02.2014, в 13:29, Ilya Kosmodemiansky > написал(а): > > Vladimir, > > And, any effect on your problem? > > > It worked without problems longer than previous configuration but repeated > again several minutes ago :( > > >

Re: [PERFORM] Problem with ExclusiveLock on inserts

2014-02-13 Thread Ilya Kosmodemiansky
, Ilya Kosmodemiansky < ilya.kosmodemian...@postgresql-consulting.com> wrote: > Vladimir, > > pgbouncer works with pl/proxy in transaction pooling mode. A wide spread > phrase that statement mode is for plproxy does not mean any limitations for > transaction pooling mode until

Re: [PERFORM] Problem with ExclusiveLock on inserts

2014-02-13 Thread Ilya Kosmodemiansky
ameters to turn it back. But yesterday I ran "vacuum > analyze" for all relations manually but that did not help. How do your autovacuum parameters look like now? > 13.02.2014, в 0:14, Ilya Kosmodemiansky написал(а): > > On Wed, Feb 12, 2014 at 8:57 PM, Бородин Владимир wro

Re: [PERFORM] Problem with ExclusiveLock on inserts

2014-02-12 Thread Ilya Kosmodemiansky
huge shared buffers any time your database performs checkpoint. By default it usually happens too often because checkpoint_timeout is 5min by default. Without bbu, on software raid that leads to io spike and you commit waits for wal. > > 12.02.2014, в 23:37, Ilya Kosmodemiansky написал(а): >

Re: [PERFORM] Problem with ExclusiveLock on inserts

2014-02-12 Thread Ilya Kosmodemiansky
/lib/pgsql/9.3/data type ext4 > (rw,noatime,nodiratime,nobarrier) > root@rpopdb01e ~ # > > 12.02.2014, в 21:56, Ilya Kosmodemiansky > написал(а): > > My question was actually about barrier option, by default it is enabled on > RHEL6/ext4 and could cause serious bottleneck

Re: [PERFORM] Problem with ExclusiveLock on inserts

2014-02-12 Thread Ilya Kosmodemiansky
data /etc/fstab > UUID=f815fd3f-e4e4-43a6-a6a1-bce1203db3e0 /var/lib/pgsql/9.3/data ext4 > noatime,nodiratime 0 1 > root@rpopdb01e ~ # > > According to iostat the disks are not the bottleneck. > >> 12.02.2014, в 21:30, Ilya Kosmodemiansky написал(а): >> >> Hi Vl

Re: [PERFORM] Problem with ExclusiveLock on inserts

2014-02-12 Thread Ilya Kosmodemiansky
Hi Vladimir, Just in case: how is your ext4 mount? Best regards, Ilya > On Feb 12, 2014, at 17:59, Бородин Владимир wrote: > > Hi all. > > Today I have started getting errors like below in logs (seems that I have not > changed anything for last week). When it happens the db gets lots of >