Re: [GENERAL] effective_io_concurrency increasing

2017-06-19 Thread Peter Geoghegan
On Mon, Jun 19, 2017 at 4:51 PM, Peter Geoghegan wrote: > This would make only the first lookup for each distinct value on the > outer side actually do an index scan on the inner side. I can imagine > the optimization saving certain queries from consuming a lot of memory > bandwidth, as well as sa

Re: [GENERAL] effective_io_concurrency increasing

2017-06-19 Thread Peter Geoghegan
On Mon, Jun 19, 2017 at 4:35 PM, Andres Freund wrote: >> I think that this is the way index scan prefetch is normally >> implemented. Index scans will on average have a much more random >> access pattern than what is typical for bitmap heap scans, making this >> optimization more compelling, so ho

Re: [GENERAL] effective_io_concurrency increasing

2017-06-19 Thread Andres Freund
On 2017-06-19 15:21:20 -0700, Peter Geoghegan wrote: > On Mon, Jun 19, 2017 at 8:36 AM, Jeff Janes wrote: > > Unfortunately, it is only implemented in very narrow circumstances. You > > have to be doing bitmap index scans of many widely scattered rows to make it > > useful. I don't think that th

Re: [GENERAL] effective_io_concurrency increasing

2017-06-19 Thread Peter Geoghegan
On Mon, Jun 19, 2017 at 3:25 PM, Alvaro Herrera wrote: > Surely you could prefetch all the heap pages pointed to by index items > in the current leaf index page ... I'm sure that you could do that too. I'm not sure how valuable each prefetching optimization is. I can imagine prefetching heap pag

Re: [GENERAL] effective_io_concurrency increasing

2017-06-19 Thread Alvaro Herrera
Peter Geoghegan wrote: > On Mon, Jun 19, 2017 at 8:36 AM, Jeff Janes wrote: > > Unfortunately, it is only implemented in very narrow circumstances. You > > have to be doing bitmap index scans of many widely scattered rows to make it > > useful. I don't think that this is all that common of a sit

Re: [GENERAL] effective_io_concurrency increasing

2017-06-19 Thread Peter Geoghegan
On Mon, Jun 19, 2017 at 8:36 AM, Jeff Janes wrote: > Unfortunately, it is only implemented in very narrow circumstances. You > have to be doing bitmap index scans of many widely scattered rows to make it > useful. I don't think that this is all that common of a situation. The > problem is that

Re: [GENERAL] effective_io_concurrency increasing

2017-06-19 Thread Bruce Momjian
On Mon, Jun 19, 2017 at 10:49:59AM -0500, Merlin Moncure wrote: > On Mon, Jun 19, 2017 at 10:36 AM, Jeff Janes wrote: > > If you have a RAID, set it to the number of spindles in your RAID and forget > > it. It is usually one of the less interesting knobs to play with. (Unless > > your usage patte

Re: [GENERAL] effective_io_concurrency increasing

2017-06-19 Thread Jeff Janes
On Mon, Jun 19, 2017 at 8:49 AM, Merlin Moncure wrote: > On Mon, Jun 19, 2017 at 10:36 AM, Jeff Janes wrote: > > If you have a RAID, set it to the number of spindles in your RAID and > forget > > it. It is usually one of the less interesting knobs to play with. > (Unless > > your usage pattern o

Re: [GENERAL] effective_io_concurrency increasing

2017-06-19 Thread Merlin Moncure
On Mon, Jun 19, 2017 at 10:36 AM, Jeff Janes wrote: > If you have a RAID, set it to the number of spindles in your RAID and forget > it. It is usually one of the less interesting knobs to play with. (Unless > your usage pattern of the database is unusual and exact fits the above > pattern.) Isn'

Re: [GENERAL] effective_io_concurrency increasing

2017-06-19 Thread Jeff Janes
On Sun, Jun 18, 2017 at 7:09 PM, David G. Johnston < david.g.johns...@gmail.com> wrote: > On Sun, Jun 18, 2017 at 6:02 PM, Patrick B > wrote: > >> Hi guys. >> >> I just wanna understand the effective_io_concurrency value better. >> >> My current Master database server has 16 vCPUS and I use >> ​​

Re: [GENERAL] effective_io_concurrency increasing

2017-06-19 Thread Andreas Kretschmer
Am 19.06.2017 um 03:02 schrieb Patrick B: Hi guys. I just wanna understand the effective_io_concurrency value better. My current Master database server has 16 vCPUS and I use effective_io_concurrency = 0. What can be the benefits of increasing that number? Also, do you guys have any recom

Re: [GENERAL] effective_io_concurrency increasing

2017-06-18 Thread David G. Johnston
On Sun, Jun 18, 2017 at 6:02 PM, Patrick B wrote: > Hi guys. > > I just wanna understand the effective_io_concurrency value better. > > My current Master database server has 16 vCPUS and I use > ​​ > effective_io_concurrency = 0. > ​It seems as though the number of virtual CPUs little to no bea

Re: [GENERAL] effective_io_concurrency increasing

2017-06-18 Thread Melvin Davidson
*As per the docs:* *1. This is dependent on whether or not you are using a RAID disk,2. "Some experimentation may be needed to find the best value"* *IOW, there is no general recommendation.* On Sun, Jun 18, 2017 at 9:24 PM, Patrick B wrote: > > > 2017-06-19 13:19 GMT+12:00 Melvin Davidson :

Re: [GENERAL] effective_io_concurrency increasing

2017-06-18 Thread Patrick B
2017-06-19 13:19 GMT+12:00 Melvin Davidson : > > > On Sun, Jun 18, 2017 at 9:02 PM, Patrick B > wrote: > >> Hi guys. >> >> I just wanna understand the effective_io_concurrency value better. >> >> My current Master database server has 16 vCPUS and I >> use effective_io_concurrency = 0. >> >> What

Re: [GENERAL] effective_io_concurrency increasing

2017-06-18 Thread Melvin Davidson
On Sun, Jun 18, 2017 at 9:02 PM, Patrick B wrote: > Hi guys. > > I just wanna understand the effective_io_concurrency value better. > > My current Master database server has 16 vCPUS and I > use effective_io_concurrency = 0. > > What can be the benefits of increasing that number? Also, do you guy

[GENERAL] effective_io_concurrency increasing

2017-06-18 Thread Patrick B
Hi guys. I just wanna understand the effective_io_concurrency value better. My current Master database server has 16 vCPUS and I use effective_io_concurrency = 0. What can be the benefits of increasing that number? Also, do you guys have any recommendations? I'm using PG 9.2 and the official do

Re: [GENERAL] effective_io_concurrency with an SSD SAN?

2013-08-26 Thread Samrat Revagade
>Given a system with 32 cores, an SSD SAN with 48x drives, and 2x 8Gbps >paths from the server to the SAN, what would be a good starting point >to set effective_io_concurrency? I currently have it set to 32, but I >kind of feel like the right setting would be "2" since we have two >paths. We don'

[GENERAL] effective_io_concurrency with an SSD SAN?

2013-08-20 Thread Evan D. Hoffman
Given a system with 32 cores, an SSD SAN with 48x drives, and 2x 8Gbps paths from the server to the SAN, what would be a good starting point to set effective_io_concurrency? I currently have it set to 32, but I kind of feel like the right setting would be "2" since we have two paths. We don't oft

Re: [GENERAL] effective_io_concurrency on Windows

2013-03-26 Thread Merlin Moncure
On Tue, Mar 26, 2013 at 3:35 AM, Bartosz Dmytrak wrote: > Hi all > is it possible to introduce similar solution for Windows systems in future? > I am aware it is not available because of lack of posix_fadvise function, > but I believe there is a way to introduce this feature for Win systems. Prob

[GENERAL] effective_io_concurrency on Windows

2013-03-26 Thread Bartosz Dmytrak
Hi all is it possible to introduce similar solution for Windows systems in future? I am aware it is not available because of lack of posix_fadvise function, but I believe there is a way to introduce this feature for Win systems. Regards, Bartek

Re: [GENERAL] effective_io_concurrency

2011-02-06 Thread Yves Weißig
Thanks for the detailed answer and explanation. I am curious if there is another way to force this behaviour. As a part of my masters thesis I am trying to develop an index which heavily relies on requesting blocks in parallel. Without knowing the code, do I have to dive into it and change it manua

Re: [GENERAL] effective_io_concurrency

2011-02-03 Thread Greg Smith
Yves Weißig wrote: I was wondering if there is more information about this switch in the configuration. Does it really work? Where in the source code can I follow how it works? "sgmgr.c" seems to be an entry point, but where exactly is it used? Currently the code only kicks in when you're do

[GENERAL] effective_io_concurrency

2011-02-02 Thread Yves Weißig
Hi pgsql-general group, I was wondering if there is more information about this switch in the configuration. Does it really work? Where in the source code can I follow how it works? "sgmgr.c" seems to be an entry point, but where exactly is it used? Greets, Yves -- Sent via pgsql-general mailin

Re: [GENERAL] effective_io_concurrency details

2010-05-26 Thread Kenichiro Tanaka
Hello. >Is effective_io_concurrency working on WinXP sp2 ? No. "effective_io_concurrency" works when an OS can use posix_fadvise(). But I think WinXP doesn't have posix_fadvise(). >and what is the difference between effective_io_concurrency = 0 >and effective_io_concurrency = 1 If effective_io

Re: [GENERAL] effective_io_concurrency details

2010-05-26 Thread Cédric Villemain
2010/5/26 pasman pasmański : > Hello. > > I have 2 questions: > Is effective_io_concurrency working on WinXP sp2 ? no > and what is the difference between effective_io_concurrency = 0 > and effective_io_concurrency = 1 0 disable prefetching. 1 allow a short prefetch window > > > Postgres 8.4.4

[GENERAL] effective_io_concurrency details

2010-05-26 Thread pasman pasmański
Hello. I have 2 questions: Is effective_io_concurrency working on WinXP sp2 ? and what is the difference between effective_io_concurrency = 0 and effective_io_concurrency = 1 Postgres 8.4.4 pasman -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make change