Re: [PERFORM] Read/Write block sizes

2005-08-24 Thread Lance Obermeyer
Since Bruce referred to the "corporate software world" I'll chime in... It has been a while since adding knobs and dials has been considered a good idea. Customers are almost always bad at tuning their systems, which decreases customer satisfaction. While many people assume the corporate types

[PERFORM] RAID arrays (and vendors)

2005-08-24 Thread Jim C. Nasby
I'm looking for an external RAID array (with external controller); either ~8 15kRPM SCSI drives or something with more SATA drives. This will be used in a test environment and could get moved between machines, so I'd like something with it's own RAID controller. Support for a broad range of OSes is

Re: [PERFORM] Performance indexing of a simple query

2005-08-24 Thread Tom Lane
Mark Fox <[EMAIL PROTECTED]> writes: > The sort of queries I want to execute (among others) are like: > SELECT * FROM jobs > WHERE completion_time > SOMEDATE AND start_time < SOMEDATE; > In plain english: All the jobs that were running at SOMEDATE. AFAIK there is no good way to do this with btree

Re: [PERFORM] Caching by Postgres

2005-08-24 Thread mark
On Wed, Aug 24, 2005 at 05:09:04PM -0400, Alan Stange wrote: > The older 32bit RISC processors do have 64 bit registers, ALUs and > datapaths, and they are marketed toward high end scientific computing, > and you're claiming that such a processor is slower than one which has > the addition of 64

Re: [PERFORM] Read/Write block sizes

2005-08-24 Thread Jim C. Nasby
On Wed, Aug 24, 2005 at 12:12:22PM -0400, Chris Browne wrote: > Everyone involved in development seems to me to have a reasonably keen > understanding as to what the potential benefits of threading are; the > value is that there fall out plenty of opportunities to parallelize > the evaluation of po

Re: [PERFORM] Caching by Postgres

2005-08-24 Thread Michael Stone
On Wed, Aug 24, 2005 at 03:34:41PM -0400, [EMAIL PROTECTED] wrote: It isn't an urban myth that 64-bit math on a 64-bit processor is faster, at least if done using registers. It definately is faster. It may be an urban myth, though, that most applications perform a sufficient amount of 64-bit arit

Re: [PERFORM] Caching by Postgres

2005-08-24 Thread Alan Stange
[EMAIL PROTECTED] wrote: On Wed, Aug 24, 2005 at 02:47:09PM -0400, Alan Stange wrote: At least on Sparc processors, v8 and newer, any double precision math (including longs) is performed with a single instruction, just like for a 32 bit datum. Loads and stores of 8 byte datums are also hand

Re: [PERFORM] Performance indexing of a simple query

2005-08-24 Thread Jim C. Nasby
Try CREATE INDEX start_complete ON jobs( start_time, completion_time ); Try also completion_time, start_time. One might work better than the other. Or, depending on your data, you might want to keep both. In 8.1 you'll be able to do bitmap-based index combination, which might allow making use of

Re: [PERFORM] Caching by Postgres

2005-08-24 Thread PFC
At least on Sparc processors, v8 and newer, any double precision math (including longs) is performed with a single instruction, just like for a 32 bit datum. Loads and stores of 8 byte datums are also handled via a single instruction. The urban myth that 64bit math is different/better

Re: [PERFORM] Read/Write block sizes

2005-08-24 Thread Jignesh Shah
Agreed!!! But the knowledge to Auto-tune your application comes from years of understanding of how users are using the so-called "knobs".. But if the "knobs" are not there in the first place.. how do you know what people are using? The "so-called" big boys are also using their knowledge base of

Re: [PERFORM] Some ideas for comment

2005-08-24 Thread Chris Hoover
On 8/24/05, Merlin Moncure <[EMAIL PROTECTED]> wrote: > > Ok, there is always a lot of talk about tuning PostgreSQL on linux and > > how PostgreSQL uses the linux kernel cache to cache the tables and > > indexes. > [...] > > > > 1. Implement a partition type layout using views and rules - This > >

[PERFORM] Performance indexing of a simple query

2005-08-24 Thread Mark Fox
I have a table called 'jobs' with several million rows, and the only columns that are important to this discussion are 'start_time' and 'completion_time'. The sort of queries I want to execute (among others) are like: SELECT * FROM jobs WHERE completion_time > SOMEDATE AND start_time < SOMEDATE;

Re: [PERFORM] Caching by Postgres

2005-08-24 Thread mark
On Wed, Aug 24, 2005 at 02:47:09PM -0400, Alan Stange wrote: > At least on Sparc processors, v8 and newer, any double precision math > (including longs) is performed with a single instruction, just like for > a 32 bit datum. Loads and stores of 8 byte datums are also handled via > a single inst

Re: [PERFORM] Some ideas for comment

2005-08-24 Thread Tom Lane
"Merlin Moncure" <[EMAIL PROTECTED]> writes: >> Right now, we are still on 7.3.4. However, these ideas would be >> implemented as part of an upgrade to 8.x (plus, we'll initialize the >> new clusters with a C locale). > yes, do this! Moving from 7.3 to 8.0 is alone likely to give you a noticeabl

Re: [PERFORM] Some ideas for comment

2005-08-24 Thread Alvaro Herrera
On Wed, Aug 24, 2005 at 12:56:54PM -0400, Chris Hoover wrote: > I don't have real numbers to give you, but we know that our systems > are hurting i/o wise and we are growing by about 2GB+ per week (net). > We actually grow by about 5GB/week/server. However, when I run my > weekly maintenance of

Re: [PERFORM] Caching by Postgres

2005-08-24 Thread Alan Stange
[EMAIL PROTECTED] wrote: So then we move on to what 64-bit is really useful for. Obviously, there is the arithmetic. If you were previously doing 64-bit arithmetic through software, you will notice an immediate speed improvement when doing it through hardware instead. If you have a program that i

Re: [PERFORM] Some ideas for comment

2005-08-24 Thread Merlin Moncure
> Ok, there is always a lot of talk about tuning PostgreSQL on linux and > how PostgreSQL uses the linux kernel cache to cache the tables and > indexes. [...] > > 1. Implement a partition type layout using views and rules - This > will allow me to have one table in each view with the "active" dat

Re: [PERFORM] Caching by Postgres

2005-08-24 Thread William Yu
Donald Courtney wrote: I built postgreSQL 8.1 64K bit on solaris 10 a few months ago and side by side with the 32 bit postgreSQL build saw no improvement. In fact the 64 bit result was slightly lower. I'm not surprised 32-bit binaries running on a 64-bit OS would be faster than 64-bit/64-bit.

Re: [PERFORM] Caching by Postgres

2005-08-24 Thread mark
On Wed, Aug 24, 2005 at 09:21:12AM -0400, Donald Courtney wrote: > I built postgreSQL 8.1 64K bit on solaris 10 a few months ago > and side by side with the 32 bit postgreSQL build saw no improvement. > In fact the 64 bit result was slightly lower. I've had this sort of argument with a friend of

Re: [PERFORM] performance drop on RAID5

2005-08-24 Thread Arjen van der Meijden
On 24-8-2005 16:43, Alexandre Barros wrote: Hello, i have a pg-8.0.3 running on Linux kernel 2.6.8, CPU Sempron 2600+, 1Gb RAM on IDE HD ( which could be called a "heavy desktop" ), measuring this performance with pgbench ( found on /contrib ) it gave me an average ( after several runs ) o

[PERFORM] Some ideas for comment

2005-08-24 Thread Chris Hoover
Ok, there is always a lot of talk about tuning PostgreSQL on linux and how PostgreSQL uses the linux kernel cache to cache the tables and indexes. My question is, is there anyway to see what files linux is caching at this moment? My reasoning behind this question is: I have several database syst

Re: [PERFORM] performance drop on RAID5

2005-08-24 Thread Merlin Moncure
> Hello, > i have a pg-8.0.3 running on Linux kernel 2.6.8, CPU Sempron 2600+, > 1Gb RAM on IDE HD ( which could be called a "heavy desktop" ), measuring > this performance with pgbench ( found on /contrib ) it gave me an > average ( after several runs ) of 170 transactions per second; 170 tps

Re: [PERFORM] performance drop on RAID5

2005-08-24 Thread Joshua D. Drake
Alexandre Barros wrote: Hello, i have a pg-8.0.3 running on Linux kernel 2.6.8, CPU Sempron 2600+, 1Gb RAM on IDE HD ( which could be called a "heavy desktop" ), measuring this performance with pgbench ( found on /contrib ) it gave me an average ( after several runs ) of 170 transactions

Re: [PERFORM] Read/Write block sizes

2005-08-24 Thread Josh Berkus
Tom, Gavin, > > To get decent I/O you need 1MB fundamental units all the way down the > > stack. > > It would also be a good idea to have an application that isn't likely > to change a single bit in a 1MB range and then expect you to record > that change. This pretty much lets Postgres out of th

Re: [PERFORM] performance drop on RAID5

2005-08-24 Thread Rosser Schwarz
On 8/24/05, Alexandre Barros <[EMAIL PROTECTED]> wrote: > i wouldn't be so stunned if the newer machine was ( say ) twice faster > than the older server, but over three times faster is disturbing. RAID5 on so few spindles is a known losing case for PostgreSQL. You'd be far, far better off doing

Re: [PERFORM] Performance for relative large DB

2005-08-24 Thread Chris Browne
"tobbe" <[EMAIL PROTECTED]> writes: > Hi Chris. > > Thanks for the answer. > Sorry that i was a bit unclear. > > 1) We update around 20.000 posts per night. No surprise there; I would have been surprised to see 100/nite or 6M/nite... > 2) What i meant was that we suspect that the DBMS called Perv

Re: [PERFORM] Read/Write block sizes

2005-08-24 Thread Chris Browne
[EMAIL PROTECTED] (Steve Poe) writes: > Chris, > > Unless I am wrong, you're making the assumpting the amount of time spent > and ROI is known. Maybe those who've been down this path know how to get > that additional 2-4% in 30 minutes or less? > > While each person and business' performance gains

Re: [PERFORM] Caching by Postgres

2005-08-24 Thread PFC
Really? Cool, I'd like to see that. Could you follow up with Hans? Or give me his e-mail? You can subscribe to the Reiser mailinglist on namesys.com or : [EMAIL PROTECTED] ---(end of broadcast)--- TIP 1: if posting/reading

Re: [PERFORM] performance drop on RAID5

2005-08-24 Thread Frank Wiles
On Wed, 24 Aug 2005 11:43:05 -0300 Alexandre Barros <[EMAIL PROTECTED]> wrote: > I've googled a little, and maybe the cluster size might be one > problem, but despite that, the performance dropping when running on > "server-class" hardware with RAID-5 SCSI-2 drives was way above my > most delir

[PERFORM] performance drop on RAID5

2005-08-24 Thread Alexandre Barros
Hello, i have a pg-8.0.3 running on Linux kernel 2.6.8, CPU Sempron 2600+, 1Gb RAM on IDE HD ( which could be called a "heavy desktop" ), measuring this performance with pgbench ( found on /contrib ) it gave me an average ( after several runs ) of 170 transactions per second; for the sake

Re: [PERFORM] Caching by Postgres

2005-08-24 Thread Stephen Frost
* Donald Courtney ([EMAIL PROTECTED]) wrote: > To be clear - > I built postgreSQL 8.1 64K bit on solaris 10 a few months ago > and side by side with the 32 bit postgreSQL build saw no improvement. > In fact the 64 bit result was slightly lower. That makes some sense actually. It really depends o

Re: [PERFORM] Read/Write block sizes

2005-08-24 Thread Bruce Momjian
This thread covers several performance ideas. First is the idea that more parameters should be configurable. While this seems like a noble goal, we try to make parameters auto-tuning, or if users have to configure it, the parameter should be useful for a significant number of users. In the com

Re: [PERFORM] Caching by Postgres

2005-08-24 Thread Donald Courtney
Great discussion and illuminating for those of us who are still learning the subtleties of postGres. William To be clear - I built postgreSQL 8.1 64K bit on solaris 10 a few months ago and side by side with the 32 bit postgreSQL build saw no improvement. In fact the 64 bit result was slightly

Re: [PERFORM] Read/Write block sizes

2005-08-24 Thread PFC
of effort reinventing the wheel ... but our time will be repaid much more if we work at levels that the OS cannot have knowledge of, such as join planning and data statistics. Considering a global budget of man-hours which is the best ? 1- Spend it on reimplementing half of VFS in pos