Re: [PERFORM] Using IOZone to simulate DB access patterns

2009-04-10 Thread M. Edward (Ed) Borasky
of the distros don't install it by default. But have a look at http://ow.ly/2zyW for some Getting Started info. -- M. Edward (Ed) Borasky http://www.linkedin.com/in/edborasky I've never met a happy clam. In fact, most of them were pretty steamed.

[PERFORM] iowait bug?

2009-03-20 Thread M. Edward (Ed) Borasky
I just discovered this on a LinkedIn user group: http://bugzilla.kernel.org/show_bug.cgi?id=12309 Is anyone here seeing evidence of this in PostgreSQL?? -- M. Edward (Ed) Borasky http://www.linkedin.com/in/edborasky I've never met a happy clam. In fact, most of them were pretty steamed

Re: [PERFORM] [PERFORMANCE] Buying hardware

2009-01-27 Thread M. Edward (Ed) Borasky
become a. How little memory can you buy without putting your service level agreements at risk? b. How do you allocate the PostgreSQL-specific memory buffers at the expense of the Linux page cache for optimum performance? -- M. Edward (Ed) Borasky I've never met a happy clam. In fact, most

Re: [PERFORM] [PERFORMANCE] Buying hardware

2009-01-27 Thread M. Edward (Ed) Borasky
M. Edward (Ed) Borasky wrote: Given large amounts of RAM and only PostgreSQL running in the server, the interesting trade-offs become a. How little memory can you buy without putting your service level agreements at risk? b. How do you allocate the PostgreSQL-specific memory buffers

Re: [PERFORM] [PERFORMANCE] Buying hardware

2009-01-27 Thread M. Edward (Ed) Borasky
consuming processor or disk time to do so. But you would need to poison the caches between runs, and restart PostgreSQL if you're modifying its memory allocations. -- M. Edward (Ed) Borasky I've never met a happy clam. In fact, most of them were pretty steamed. -- Sent via pgsql-performance mailing

Re: [PERFORM] postgresql 8.3 tps rate

2009-01-26 Thread M. Edward (Ed) Borasky
Matthew Wakeling wrote: On Sun, 25 Jan 2009, M. Edward (Ed) Borasky wrote: Actually, this isn't so much a 'pgbench' exercise as it is a source of 'real-world application' data for my Linux I/O performance visualization tools. I've done 'iozone' tests, though not recently. But what I'm

Re: [PERFORM] postgresql 8.3 tps rate

2009-01-26 Thread M. Edward (Ed) Borasky
Craig Ringer wrote: M. Edward (Ed) Borasky wrote: At the CMG meeting I asked the disk drive engineers, well, if the drives are doing the scheduling, why does Linux go to all the trouble? One big reason is that Linux knows more about the relative importance of I/O operations than

Re: [PERFORM] postgresql 8.3 tps rate

2009-01-26 Thread M. Edward (Ed) Borasky
I'll see what results I get with an untuned config. -- * Greg Smith gsm...@gregsmith.com http://www.gregsmith.com Baltimore, MD Thanks!! I'm just getting into the PostgreSQL tuning part of things. -- M. Edward (Ed) Borasky I've never met a happy clam. In fact, most of them were pretty

Re: [PERFORM] postgresql 8.3 tps rate

2009-01-26 Thread M. Edward (Ed) Borasky
Ron Mayer wrote: M. Edward (Ed) Borasky wrote: At the CMG meeting I asked the disk drive engineers, well, if the drives are doing the scheduling, why does Linux go to all the trouble? Their answer was something like, smart disk drives are a relatively recent invention. But One more reason

Re: [PERFORM] postgresql 8.3 tps rate

2009-01-25 Thread M. Edward (Ed) Borasky
could lower your throughput. -- M. Edward (Ed) Borasky I've never met a happy clam. In fact, most of them were pretty steamed. -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] postgresql 8.3 tps rate

2009-01-25 Thread M. Edward (Ed) Borasky
periods, the seeks must be killing me. This is actually not a benchmark, but a way of generating sample blktrace data, so when it's finally working, I'll have some block-layer traces to tell me exactly what's going on. :) -- M. Edward (Ed) Borasky I've never met a happy clam. In fact, most of them

Re: [PERFORM] postgresql 8.3 tps rate

2009-01-25 Thread M. Edward (Ed) Borasky
out of the drive with the xfs filesystem and the cfq scheduler. And the choice of scheduler might not matter. And the choice of filesystem might not matter. I may be getting all the drive can do. -- M. Edward (Ed) Borasky I've never met a happy clam. In fact, most of them were pretty steamed

Re: [PERFORM] SSD performance

2009-01-23 Thread M. Edward (Ed) Borasky
-bound are good enough. And given that, a moderate amount of software tweaking and balancing will get you close to a local optimum. -- M. Edward (Ed) Borasky I've never met a happy clam. In fact, most of them were pretty steamed. -- Sent via pgsql-performance mailing list (pgsql-performance

Re: [PERFORM] SSD performance

2009-01-23 Thread M. Edward (Ed) Borasky
hardware at performance issues. There is no correct in left. There is no correct in right. Correctness is the result of friction caused by the mingling of the two. The only good I/O is a dead I/O -- Mark Friedman -- M. Edward (Ed) Borasky I've never met a happy clam. In fact, most

Re: [PERFORM] linux, memory (mis)accounting/reporting, and the planner/optimizer

2009-01-22 Thread M. Edward (Ed) Borasky
Greg Smith wrote: On Wed, 21 Jan 2009, M. Edward (Ed) Borasky wrote: Re the OOM killer -- maybe a patch to the kernel could make things better?? People have tried to raise awareness of it; sample: http://lkml.org/lkml/2007/2/9/275 without much success. The Linux kernel hackers

Re: [PERFORM] linux, memory (mis)accounting/reporting, and the planner/optimizer

2009-01-21 Thread M. Edward (Ed) Borasky
accounting in Linux got better in the 2.6.25 kernel, although I'm not sure the user space tools are fully deployed even today to track it. And of course, lots of servers still use kernels older than 2.6.25. Re the OOM killer -- maybe a patch to the kernel could make things better?? -- M. Edward (Ed

Re: [PERFORM] block device benchmarking

2009-01-11 Thread M. Edward (Ed) Borasky
Markus Wanner wrote: M. Edward (Ed) Borasky wrote: Check out the work of Jens Axboe and Alan Brunelle, specifically the packages blktrace and fio. BTW ... I am working on my blktrace howto even as I type this. I don't have an ETA -- that's going to depend on how long it takes me to get

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-11 Thread M. Edward (Ed) Borasky
*will* have some major OS risk is with testing-level software or bleeding edge Linux distros like Fedora. Quite frankly, I don't know why people run Fedora servers -- if it's Red Hat compatibility you want, there's CentOS. -- M. Edward (Ed) Borasky, FBG, AB, PTA, PGS, MS, MNLP, NST, ACMC(P), WOM I've

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-11 Thread M. Edward (Ed) Borasky
fixes too. Otherwise, what's the point of spending money for an OS? :) -- M. Edward (Ed) Borasky, FBG, AB, PTA, PGS, MS, MNLP, NST, ACMC(P), WOM I've never met a happy clam. In fact, most of them were pretty steamed. -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-11 Thread M. Edward (Ed) Borasky
or completely worthless for database use is rather depressing. Well, of course, they won't make *all* your problems go away. But still, I'd much rather have an IBM or HP or Dell server running Windows 2003 or RHEL or SLES than some no-name hardware running Fedora or Ubuntu. -- M. Edward (Ed) Borasky, FBG

Re: [PERFORM] block device benchmarking

2009-01-10 Thread M. Edward (Ed) Borasky
Markus Wanner wrote: Hi, I'm fiddling with a hand-made block device based benchmarking thingie, which I want to run random reads and writes of relatively small blocks (somewhat similar to databases). I'm much less interested in measuring throughput, but rather in latency. Besides varying

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-07 Thread M. Edward (Ed) Borasky
Simon Waters wrote: On Wednesday 07 January 2009 04:17:10 M. Edward (Ed) Borasky wrote: 1. The package it lives in is called sysstat. Most Linux distros do *not* install sysstat by default. Somebody should beat up on them about that. :) Hehe, although sysstat and friends did have issues

Re: [PERFORM] Are random writes optimized sequentially by Linux kernel?

2009-01-07 Thread M. Edward (Ed) Borasky
da...@lang.hm wrote: On Thu, 8 Jan 2009, Dmitry Koterov wrote: OK, thank you. Now - PostgreSQL-related question. If the system reorders writes to minimize seeking, I suppose that in heavy write-loaded PostgreSQL instalation dstat (or iostat) realtime write statistics should be close to

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-07 Thread M. Edward (Ed) Borasky
Scott Marlowe wrote: On Wed, Jan 7, 2009 at 3:34 PM, Greg Smith gsm...@gregsmith.com wrote: On Wed, 7 Jan 2009, Scott Marlowe wrote: I cannot understand how Dell stays in business. There's a continuous stream of people who expect RAID5 to perform well, too, yet this surprises you? I guess

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-06 Thread M. Edward (Ed) Borasky
David Rees wrote: On Tue, Jan 6, 2009 at 11:02 AM, Stefano Nichele stefano.nich...@gmail.com wrote: BTW, why did you said I/O bound ? Which are the parameters that highlight that ? Sorry for my ignorance In addition to the percentage of time spent in wait as Scott said, you can also

Re: [PERFORM] file system and raid performance

2008-12-09 Thread M. Edward (Ed) Borasky
Scott Marlowe wrote: On Sun, Dec 7, 2008 at 10:59 PM, M. Edward (Ed) Borasky [EMAIL PROTECTED] wrote: Ah, but shouldn't a PostgreSQL (or any other database, for that matter) have its own set of filesystems tuned to the application's I/O patterns? Sure, there are some people who need to have

Re: [PERFORM] file system and raid performance

2008-12-07 Thread M. Edward (Ed) Borasky
. Edward (Ed) Borasky, FBG, AB, PTA, PGS, MS, MNLP, NST, ACMC(P), WOM A mathematician is a device for turning coffee into theorems. -- Alfréd Rényi via Paul Erdős -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http