Re: [PERFORM] Very poor read performance, query independent

2017-07-25 Thread Charles Nadeau
I now know more about Postgresql tuning. Charles On Mon, Jul 10, 2017 at 4:03 PM, Charles Nadeau wrote: > I’m running PostgreSQL 9.6.3 on Ubuntu 16.10 (kernel 4.4.0-85-generic). > Hardware is: > > *2x Intel Xeon E5550 > > *72GB RAM > > *Hardware RAID10 (4 x 146GB SAS 10k) P

Re: [PERFORM] Very poor read performance, query independent

2017-07-20 Thread Charles Nadeau
; On 15/07/17 11:09, Mark Kirkwood wrote: > >> Ah yes - that seems more sensible (but still slower than I would expect >> for 5 disks RAID 0). >> > > > > -- > Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) > To make changes t

Re: [PERFORM] Very poor read performance, query independent

2017-07-19 Thread Charles Nadeau
k. Thanks! Charles On Tue, Jul 18, 2017 at 8:01 PM, Justin Pryzby wrote: > On Tue, Jul 18, 2017 at 02:13:58PM -0300, Claudio Freire wrote: > > On Tue, Jul 18, 2017 at 1:01 PM, Claudio Freire > wrote: > > > On Tue, Jul 18, 2017 at 6:20 AM, Charles Nadeau > > > wro

Re: [PERFORM] Very poor read performance, query independent

2017-07-18 Thread Charles Nadeau
7 at 10:56 PM, Claudio Freire wrote: > On Fri, Jul 14, 2017 at 12:34 PM, Charles Nadeau > wrote: > > Workers Planned: 12 > > Workers Launched: 12 > > Buffers: sh

Re: [PERFORM] Very poor read performance, query independent

2017-07-17 Thread Charles Nadeau
srcport) "flows_srcport_dstport_idx" btree (srcport, dstport) Thanks! Charles On Fri, Jul 14, 2017 at 10:18 PM, Igor Neyman wrote: > > > > > *From:* pgsql-performance-ow...@postgresql.org [mailto:pgsql-performance- > ow...@postgresql.org] *On Behalf Of *Igor Neyman

Re: [PERFORM] Very poor read performance, query independent

2017-07-16 Thread Charles Nadeau
Scott, The temp tablespace is on a disk of his own. Thanks! Charles On Sat, Jul 15, 2017 at 7:58 PM, Scott Marlowe wrote: > On Sat, Jul 15, 2017 at 11:53 AM, Charles Nadeau > wrote: > > Mark, > > > > I increased the read ahead to 16384 and it doesn't improve per

Re: [PERFORM] Very poor read performance, query independent

2017-07-16 Thread Charles Nadeau
st. Thanks! Charles On Fri, Jul 14, 2017 at 9:13 PM, Igor Neyman wrote: > *From:* Charles Nadeau [mailto:charles.nad...@gmail.com] > *Sent:* Friday, July 14, 2017 11:35 AM > *To:* Igor Neyman > *Cc:* Jeff Janes ; pgsql-performance@postgresql.org > *Subject:* Re: [PE

Re: [PERFORM] Very poor read performance, query independent

2017-07-15 Thread Charles Nadeau
o worth investigating is RAID stripe size - for DW work it makes sense > for it to be reasonably big (256K to 1M), which again will help speed is > sequential scans. > > Cheers > > Mark > > P.s I used to work for Greenplum, so this type of problem came up a lot > :-) . The be

Re: [PERFORM] Very poor read performance, query independent

2017-07-15 Thread Charles Nadeau
s > > Mark > > On 15/07/17 11:09, Mark Kirkwood wrote: > >> Ah yes - that seems more sensible (but still slower than I would expect >> for 5 disks RAID 0). >> > > > > -- > Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) &g

Re: [PERFORM] Very poor read performance, query independent

2017-07-14 Thread Charles Nadeau
finition: SELECT DISTINCT flowscompact.dstaddr FROM flowscompact LEFT JOIN mynetworks ON mynetworks.ipaddr >> flowscompact.dstaddr::ip4r WHERE mynetworks.ipaddr IS NULL; Thanks! Charles On Wed, Jul 12, 2017 at 6:39 PM, Igor Neyman wrote: > > > > > *From:* pgsql-perf

Re: [PERFORM] Very poor read performance, query independent

2017-07-14 Thread Charles Nadeau
gqu-sz await r_await w_await svctm %util > sda 0.00 0.00 926.000.00 114.89 0.00 254.10 >1.902.032.030.00 1.08 100.00 > > > So might be useful for us to see something like that from your system - > note you need to check you really have

Re: [PERFORM] Very poor read performance, query independent

2017-07-12 Thread Charles Nadeau
up > hugepages ( https://www.postgresql.org/docs/9.6/static/kernel- > resources.html#LINUX-HUGE-PAGES ), and bump effective_io_concurrency up > quite a bit as well (256 ?). > > > On Mon, Jul 10, 2017 at 10:03 AM, Charles Nadeau > wrote: > >> I’m running PostgreSQL 9.6.3

Re: [PERFORM] Very poor read performance, query independent

2017-07-12 Thread Charles Nadeau
=4 loops=10) Buffers: shared hit=10 Planning time: 6.689 ms Execution time: 2764860.853 ms (58 rows) Regarding "Also using dstat I can see that iowait time is at about 25%", I don't think the server was doing anything else. If it is important, I can repeat the benchmarks. Thanks!

Re: [PERFORM] Very poor read performance, query independent

2017-07-12 Thread Charles Nadeau
] deadline cfq Thanks! Charles On Wed, Jul 12, 2017 at 1:42 AM, Joshua D. Drake wrote: > On 07/11/2017 04:15 PM, Merlin Moncure wrote: > >> On Mon, Jul 10, 2017 at 9:03 AM, Charles Nadeau >> wrote: >> >>> I’m running PostgreSQL 9.6.3 on Ubuntu 16.10 (kernel 4

Re: [PERFORM] Very poor read performance, query independent

2017-07-12 Thread Charles Nadeau
? Thanks! Charles On Tue, Jul 11, 2017 at 5:16 PM, Igor Neyman wrote: > > > *From:* pgsql-performance-ow...@postgresql.org [mailto:pgsql-performance- > ow...@postgresql.org] *On Behalf Of *Igor Neyman > *Sent:* Tuesday, July 11, 2017 10:34 AM > *To:* Charles Nadeau > *C

Re: [PERFORM] Very poor read performance, query independent

2017-07-11 Thread Charles Nadeau
Igor, The sum of effective_cache_size and shared_buffer will be higher than the physical memory I have. Is it OK? Thanks! Charles On Tue, Jul 11, 2017 at 4:34 PM, Igor Neyman wrote: > > > *From:* Charles Nadeau [mailto:charles.nad...@gmail.com] > *Sent:* Tuesday, July 11, 2017 6

Re: [PERFORM] Very poor read performance, query independent

2017-07-11 Thread Charles Nadeau
lto:pgsql-performance- > ow...@postgresql.org] *On Behalf Of *Charles Nadeau > *Sent:* Monday, July 10, 2017 11:48 AM > *To:* Andreas Kretschmer > *Cc:* pgsql-performance@postgresql.org > *Subject:* Re: [PERFORM] Very poor read performance, query independent > > > > Andreas, &

Re: [PERFORM] Very poor read performance, query independent

2017-07-11 Thread Charles Nadeau
aults 0swaps 241297594904 Could you suggest another way to benchmark random reads? Thanks for your help! Charles On Mon, Jul 10, 2017 at 9:24 PM, Jeff Janes wrote: > On Mon, Jul 10, 2017 at 7:03 AM, Charles Nadeau > wrote: > >> >> The problem I have is very poor re

Re: [PERFORM] Very poor read performance, query independent

2017-07-11 Thread Charles Nadeau
m:* pgsql-performance-ow...@postgresql.org [mailto:pgsql-performance- > ow...@postgresql.org] *On Behalf Of *Charles Nadeau > *Sent:* Monday, July 10, 2017 11:48 AM > *To:* Andreas Kretschmer > *Cc:* pgsql-performance@postgresql.org > *Subject:* Re: [PERFORM] Very poor read perfor

Re: [PERFORM] Very poor read performance, query independent

2017-07-11 Thread Charles Nadeau
, Jul 10, 2017 at 10:03 AM, Charles Nadeau > wrote: > >> I’m running PostgreSQL 9.6.3 on Ubuntu 16.10 (kernel 4.4.0-85-generic). >> Hardware is: >> >> *2x Intel Xeon E5550 >> >> *72GB RAM >> >> *Hardware RAID10 (4 x 146GB SAS 10k) P410i controller wit

Re: [PERFORM] Very poor read performance, query independent

2017-07-10 Thread Charles Nadeau
17 um 16:03 schrieb Charles Nadeau: > >> random_page_cost | 22 >> > > > why such a high value for random_page_cost? > > Regards, Andreas > > -- > 2ndQuadrant - The PostgreSQL Support Company. > www.2ndQuadrant.com > > > > -- > Sent via pgsql-p

[PERFORM] Very poor read performance, query independent

2017-07-10 Thread Charles Nadeau
’t seem to have any issues, I am really puzzled. Thanks! Charles -- Charles Nadeau Ph.D.