Re: [GENERAL] [PERFORM] [PERFORMANCE] expanding to SAN: which portion best to move

2011-06-10 Thread Robert Klemme
On Thu, Jun 9, 2011 at 7:44 PM, Greg Smith g...@2ndquadrant.com wrote: ** On 06/09/2011 07:43 AM, Willy-Bas Loos wrote: Well, after reading your article i have been reading some materail about it on the internet, stating that separating indexes from data for performance benefits is a myth.

Re: [PERFORM] [PERFORMANCE] expanding to SAN: which portion best to move

2011-06-09 Thread Willy-Bas Loos
On Mon, May 16, 2011 at 10:19 AM, Robert Klemme shortcut...@googlemail.comwrote: On Fri, May 13, 2011 at 9:04 PM, Robert Haas robertmh...@gmail.com wrote: Separating index and tables might not be a totally good idea generally. Richard Foote has an excellent article about Oracle but I assume

Re: [GENERAL] [PERFORM] [PERFORMANCE] expanding to SAN: which portion best to move

2011-06-09 Thread Greg Smith
On 06/09/2011 07:43 AM, Willy-Bas Loos wrote: Well, after reading your article i have been reading some materail about it on the internet, stating that separating indexes from data for performance benefits is a myth. I found your comment So then a single query will only ever access one of

Re: [PERFORM] [PERFORMANCE] expanding to SAN: which portion best to move

2011-05-25 Thread Vitalii Tymchyshyn
24.05.11 21:48, Greg Smith написав(ла): Bitmap heap scan: Here, the exact list of blocks to fetch is known in advance, they're random, and it's quite possible for the kernel to schedule them more efficiently than serial access of them can do. This was added as the effective_io_concurrency

Re: [PERFORM] [PERFORMANCE] expanding to SAN: which portion best to move

2011-05-24 Thread Greg Smith
On 05/17/2011 05:47 AM, Craig Ringer wrote: This makes me wonder if Pg attempts to pre-fetch blocks of interest for areas where I/O needs can be known in advance, while there's still other works or other I/O to do. For example, pre-fetching for the next iteration of a nested loop while still

Re: [PERFORM] [PERFORMANCE] expanding to SAN: which portion best to move

2011-05-17 Thread Robert Klemme
On Mon, May 16, 2011 at 4:31 PM, Robert Haas robertmh...@gmail.com wrote: On Mon, May 16, 2011 at 4:19 AM, Robert Klemme shortcut...@googlemail.com wrote: - If the planner chooses a Bitmap Index Scan, it effectively scans the index to figure out which table blocks to read, and then reads those

Re: [PERFORM] [PERFORMANCE] expanding to SAN: which portion best to move

2011-05-17 Thread Craig Ringer
On 05/17/2011 03:00 PM, Robert Klemme wrote: The main point is that you do not benefit from the larger IO bandwidth if access patterns do not permit parallel access to both disks (e.g. because you first need to read index blocks in order to know the table blocks to read). This makes me wonder

Re: [PERFORM] [PERFORMANCE] expanding to SAN: which portion best to move

2011-05-17 Thread Cédric Villemain
2011/5/17 Craig Ringer cr...@postnewspapers.com.au: On 05/17/2011 03:00 PM, Robert Klemme wrote: The main point is that you do not benefit from the larger IO bandwidth if access patterns do not permit parallel access to both disks (e.g. because you first need to read index blocks in order to

Re: [PERFORM] [PERFORMANCE] expanding to SAN: which portion best to move

2011-05-17 Thread Robert Klemme
On Tue, May 17, 2011 at 11:47 AM, Craig Ringer cr...@postnewspapers.com.au wrote: On 05/17/2011 03:00 PM, Robert Klemme wrote: The main point is that you do not benefit from the larger IO bandwidth if access patterns do not permit parallel access to both disks (e.g. because you first need to

Re: [PERFORM] [PERFORMANCE] expanding to SAN: which portion best to move

2011-05-16 Thread Robert Klemme
On Fri, May 13, 2011 at 9:04 PM, Robert Haas robertmh...@gmail.com wrote: On Wed, May 4, 2011 at 6:31 AM, Willy-Bas Loos willy...@gmail.com wrote: I'm asking them for (real) benchmarks, thanks for the advice. (fio is not available for us now to do it myself, grmbl) It just occurred to me that

Re: [PERFORM] [PERFORMANCE] expanding to SAN: which portion best to move

2011-05-16 Thread Robert Haas
On Mon, May 16, 2011 at 4:19 AM, Robert Klemme shortcut...@googlemail.com wrote: - If the planner chooses a Bitmap Index Scan, it effectively scans the index to figure out which table blocks to read, and then reads those table blocks in block number order, so that the I/O is sequential, with

Re: [PERFORM] [PERFORMANCE] expanding to SAN: which portion best to move

2011-05-13 Thread Robert Haas
On Wed, May 4, 2011 at 6:31 AM, Willy-Bas Loos willy...@gmail.com wrote: I'm asking them for (real) benchmarks, thanks for the advice. (fio is not available for us now to do it myself, grmbl) It just occurred to me that it is not necessarily the case that reading the indexes causes a lot of

[PERFORM] [PERFORMANCE] expanding to SAN: which portion best to move

2011-05-03 Thread Willy-Bas Loos
Hi, Our database has gotten rather large and we are running out of disk space. our disks are 15K rpm SAS disks in RAID 10. We are going to rent some space on a FibreChannel SAN. That gives us the opportunity to separate the data and the indexes. Now i thought it would be best to move the indexes

Re: [PERFORM] [PERFORMANCE] expanding to SAN: which portion best to move

2011-05-03 Thread Jesper Krogh
On 2011-05-03 17:52, Willy-Bas Loos wrote: Our database has gotten rather large and we are running out of disk space. our disks are 15K rpm SAS disks in RAID 10. We are going to rent some space on a FibreChannel SAN. That gives us the opportunity to separate the data and the indexes. Now i

Re: [PERFORM] [PERFORMANCE] expanding to SAN: which portion best to move

2011-05-03 Thread Willy-Bas Loos
are you saying that, generally speaking, moving the data would be better unless the SAN performs worse than the disks? besides your point that it depends on what our end looks like i mean. (and what do you mean by the DAS way, sry no native speaker) cheers, wbl On Wed, May 4, 2011 at 6:43 AM,

Re: [PERFORM] [PERFORMANCE] expanding to SAN: which portion best to move

2011-05-03 Thread Jesper Krogh
On 2011-05-04 07:25, Willy-Bas Loos wrote: are you saying that, generally speaking, moving the data would be better unless the SAN performs worse than the disks? It was more, given all the incertainties, that seems like the least risky. The SAN might actually be less well performing than what