Re: [PERFORM] Using IOZone to simulate DB access patterns

2009-04-27 Thread Laurent Laborde
you can also play with this-tiny-shiny tool : http://pgfoundry.org/projects/pgiosim/ It just works and heavily stress the disk with random read/write. -- F4FQM Kerunix Flan Laurent Laborde -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your sub

Re: [PERFORM] Using IOZone to simulate DB access patterns

2009-04-26 Thread Mark Wong
On Sat, Apr 11, 2009 at 7:00 PM, Scott Carey wrote: > > > On 4/11/09 11:44 AM, "Mark Wong" wrote: > >> On Fri, Apr 10, 2009 at 11:01 AM, Greg Smith wrote: >>> On Fri, 10 Apr 2009, Scott Carey wrote: >>> FIO with profiles such as the below samples are easy to set up >>> >>> There are some mo

Re: [PERFORM] Using IOZone to simulate DB access patterns

2009-04-26 Thread Mark Wong
On Sat, Apr 11, 2009 at 11:44 AM, Mark Wong wrote: > On Fri, Apr 10, 2009 at 11:01 AM, Greg Smith wrote: >> On Fri, 10 Apr 2009, Scott Carey wrote: >> >>> FIO with profiles such as the below samples are easy to set up >> >> There are some more sample FIO profiles with results from various >> file

Re: [PERFORM] Using IOZone to simulate DB access patterns

2009-04-11 Thread Scott Carey
On 4/11/09 11:44 AM, "Mark Wong" wrote: > On Fri, Apr 10, 2009 at 11:01 AM, Greg Smith wrote: >> On Fri, 10 Apr 2009, Scott Carey wrote: >> >>> FIO with profiles such as the below samples are easy to set up >> >> There are some more sample FIO profiles with results from various >> filesystem

Re: [PERFORM] Using IOZone to simulate DB access patterns

2009-04-11 Thread Mark Wong
On Fri, Apr 10, 2009 at 11:01 AM, Greg Smith wrote: > On Fri, 10 Apr 2009, Scott Carey wrote: > >> FIO with profiles such as the below samples are easy to set up > > There are some more sample FIO profiles with results from various > filesystems at > http://wiki.postgresql.org/wiki/HP_ProLiant_DL3

Re: [PERFORM] Using IOZone to simulate DB access patterns

2009-04-10 Thread M. Edward (Ed) Borasky
I've done quite a bit with IOzone, but if you're on Linux, you have lots of options. In particular, you can actually capture I/O patterns from a running application with blktrace, and then replay them with btrecord / btreplay. The documentation for this stuff is a bit hard to find. Some of the dis

Re: [PERFORM] Using IOZone to simulate DB access patterns

2009-04-10 Thread Greg Smith
On Fri, 10 Apr 2009, Scott Carey wrote: I wish to thank Greg here as many of my profile variations came from the above as a starting point. That page was mainly Mark Wong's work, I just remembered where it was. -- * Greg Smith gsm...@gregsmith.com http://www.gregsmith.com Baltimore, MD -- Se

Re: [PERFORM] Using IOZone to simulate DB access patterns

2009-04-10 Thread Scott Carey
On 4/10/09 11:01 AM, "Greg Smith" wrote: > On Fri, 10 Apr 2009, Scott Carey wrote: > >> FIO with profiles such as the below samples are easy to set up > > There are some more sample FIO profiles with results from various > filesystems at > http://wiki.postgresql.org/wiki/HP_ProLiant_DL380_G5_T

Re: [PERFORM] Using IOZone to simulate DB access patterns

2009-04-10 Thread Greg Smith
On Fri, 10 Apr 2009, Scott Carey wrote: FIO with profiles such as the below samples are easy to set up There are some more sample FIO profiles with results from various filesystems at http://wiki.postgresql.org/wiki/HP_ProLiant_DL380_G5_Tuning_Guide -- * Greg Smith gsm...@gregsmith.com htt

Re: [PERFORM] Using IOZone to simulate DB access patterns

2009-04-10 Thread Scott Carey
On 4/10/09 10:31 AM, "Josh Berkus" wrote: > Scott, > >> FIO with profiles such as the below samples are easy to set up, and they can >> be mix/matched to test what happens with mixed read/write seq/rand -- with >> surprising and useful tuning results. Forcing a cache flush or sync before >> or

Re: [PERFORM] Using IOZone to simulate DB access patterns

2009-04-10 Thread Josh Berkus
Scott, FIO with profiles such as the below samples are easy to set up, and they can be mix/matched to test what happens with mixed read/write seq/rand -- with surprising and useful tuning results. Forcing a cache flush or sync before or after a run is trivial. Changing to asynchronous I/O, dir

Re: [PERFORM] Using IOZone to simulate DB access patterns

2009-04-10 Thread Josh Berkus
JD, In order to test real interactivity (AFAIK) with iozone you have to launch multiple iozone instances. You also need to do them from separate directories, otherwise it all starts writing the same file. The work I did here: Actually, current IOZone allows you to specify multiple files. For

Re: [PERFORM] Using IOZone to simulate DB access patterns

2009-04-10 Thread Scott Carey
I've switched to using FIO. Bonnie in my experience produces poor results and is better suited to testing desktop/workstation type load. Most of its tests don't apply to how postgres writes/reads anyway. IOZone is a bit more troublesome to get it to work on the file(s) you want under concurrency

Re: [PERFORM] Using IOZone to simulate DB access patterns

2009-04-10 Thread Josh Berkus
On 4/9/09 11:26 PM, Mark Kirkwood wrote: Josh Berkus wrote: All, Wow, am I really the only person here who's used IOZone? No - I used to use it exclusively, but everyone else tended to demand I redo stuff with bonnie before taking any finding seriously... so I've kinda 'submitted to the Borg

Re: [PERFORM] Using IOZone to simulate DB access patterns

2009-04-10 Thread Joshua D. Drake
On Fri, 2009-04-03 at 17:09 -0700, Josh Berkus wrote: > On 4/3/09 4:12 PM, Josh Berkus wrote: > > All, > > > > I've been using Bonnie++ for ages to do filesystem testing of new DB > > servers. But Josh Drake recently turned me on to IOZone. > > Related to this: is IOZone really multi-threaded? I'

Re: [PERFORM] Using IOZone to simulate DB access patterns

2009-04-09 Thread Mark Kirkwood
Josh Berkus wrote: All, Wow, am I really the only person here who's used IOZone? No - I used to use it exclusively, but everyone else tended to demand I redo stuff with bonnie before taking any finding seriously... so I've kinda 'submitted to the Borg' as it were -- Sent via pgsql-per

Re: [PERFORM] Using IOZone to simulate DB access patterns

2009-04-09 Thread Josh Berkus
All, Wow, am I really the only person here who's used IOZone? -- Josh Berkus PostgreSQL Experts Inc. www.pgexperts.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] Using IOZone to simulate DB access patterns

2009-04-04 Thread henk de wit
> $ apt-cache search iozone > iozone3 - Filesystem and Disk Benchmarking Tool You are right. I was confused with IOMeter, which can't be run on Linux (the Dynamo part can, but that's not really useful without the 'command & control' part). __

Re: [PERFORM] Using IOZone to simulate DB access patterns

2009-04-04 Thread Jesper Krogh
henk de wit wrote: I've been using Bonnie++ for ages to do filesystem testing of new DB servers. But Josh Drake recently turned me on to IOZone. Perhaps a little off-topic here, but I'm assuming you are using Linux to test your DB server (since you mention Bonnie++). But it seems to me that

Re: [PERFORM] Using IOZone to simulate DB access patterns

2009-04-04 Thread henk de wit
> I've been using Bonnie++ for ages to do filesystem testing of new DB servers. > But Josh Drake recently turned me on to IOZone. Perhaps a little off-topic here, but I'm assuming you are using Linux to test your DB server (since you mention Bonnie++). But it seems to me that IOZone only has a

Re: [PERFORM] Using IOZone to simulate DB access patterns

2009-04-03 Thread Josh Berkus
On 4/3/09 4:12 PM, Josh Berkus wrote: All, I've been using Bonnie++ for ages to do filesystem testing of new DB servers. But Josh Drake recently turned me on to IOZone. Related to this: is IOZone really multi-threaded? I'm doing a test run right now, and only one CPU is actually active. Whi

[PERFORM] Using IOZone to simulate DB access patterns

2009-04-03 Thread Josh Berkus
All, I've been using Bonnie++ for ages to do filesystem testing of new DB servers. But Josh Drake recently turned me on to IOZone. Thing is, IOZone offers a huge complex series of parameters, so I'd really like to have some idea of how to configure it so its results are applicable to databa