Re: [PERFORM] SCSI vs SATA

2007-04-06 Thread Tom Lane
[EMAIL PROTECTED] writes: On Thu, 5 Apr 2007, Ron wrote: Yep. Folks should google bath tub curve of statistical failure or similar. Basically, always burn in your drives for at least 1/2 a day before using them in a production or mission critical role. for this and your first point,

Re: [PERFORM] SCSI vs SATA

2007-04-06 Thread Greg Smith
On Fri, 6 Apr 2007, Tom Lane wrote: It seems hard to believe that the vendors themselves wouldn't burn in the drives for half a day, if that's all it takes to eliminate a large fraction of infant mortality. I've read that much of the damage that causes hard drive infant mortality is related

Re: [PERFORM] Premature view materialization in 8.2?

2007-04-06 Thread Tom Lane
Jonathan Ellis [EMAIL PROTECTED] writes: On 4/5/07, Tom Lane [EMAIL PROTECTED] wrote: Is this a regression, or a feature of 8.2? Hard to say without EXPLAIN ANALYZE output to compare. To my eye they are identical other than the speed but perhaps I am missing something. Yeah, it sure is

Re: [PERFORM] SCSI vs SATA

2007-04-06 Thread Tom Lane
Greg Smith [EMAIL PROTECTED] writes: On Fri, 6 Apr 2007, Tom Lane wrote: It seems hard to believe that the vendors themselves wouldn't burn in the drives for half a day, if that's all it takes to eliminate a large fraction of infant mortality. I've read that much of the damage that causes

Re: [PERFORM] SCSI vs SATA

2007-04-06 Thread Michael Stone
On Thu, Apr 05, 2007 at 11:19:04PM -0400, Ron wrote: Both statements are the literal truth. Repeating something over and over again doesn't make it truth. The OP asked for statistical evidence (presumably real-world field evidence) to support that assertion. Thus far, all the publicly

Re: [PERFORM] SCSI vs SATA

2007-04-06 Thread Michael Stone
On Fri, Apr 06, 2007 at 02:00:15AM -0400, Tom Lane wrote: It seems hard to believe that the vendors themselves wouldn't burn in the drives for half a day, if that's all it takes to eliminate a large fraction of infant mortality. The savings in return processing and customer goodwill would

Re: [PERFORM] SCSI vs SATA

2007-04-06 Thread Ron
I read them as soon as they were available. Then I shrugged and noted YMMV to myself. 1= Those studies are valid for =those= users under =those= users' circumstances in =those= users' environments. How well do those circumstances and environments mimic anyone else's? I don't know since the

Re: [PERFORM] SCSI vs SATA

2007-04-06 Thread Geoffrey
Tom Lane wrote: Greg Smith [EMAIL PROTECTED] writes: On Fri, 6 Apr 2007, Tom Lane wrote: It seems hard to believe that the vendors themselves wouldn't burn in the drives for half a day, if that's all it takes to eliminate a large fraction of infant mortality. I've read that much of the

Re: [PERFORM] SCSI vs SATA

2007-04-06 Thread Ron
At 07:38 AM 4/6/2007, Michael Stone wrote: On Thu, Apr 05, 2007 at 11:19:04PM -0400, Ron wrote: Both statements are the literal truth. Repeating something over and over again doesn't make it truth. The OP asked for statistical evidence (presumably real-world field evidence) to support that

Re: [PERFORM] SCSI vs SATA

2007-04-06 Thread Geoffrey
Michael Stone wrote: On Fri, Apr 06, 2007 at 02:00:15AM -0400, Tom Lane wrote: It seems hard to believe that the vendors themselves wouldn't burn in the drives for half a day, if that's all it takes to eliminate a large fraction of infant mortality. The savings in return processing and

Re: [PERFORM] SCSI vs SATA

2007-04-06 Thread Michael Stone
On Fri, Apr 06, 2007 at 08:49:08AM -0400, Ron wrote: Not quite. Each of our professional experiences is +also+ statistical evidence. Even if it is a personally skewed sample. I'm not sure that word means what you think it means. I think the one you're looking for is anecdotal. My

Re: [PERFORM] Premature view materialization in 8.2?

2007-04-06 Thread Jonathan Ellis
On 4/6/07, Tom Lane [EMAIL PROTECTED] wrote: Jonathan Ellis [EMAIL PROTECTED] writes: On 4/5/07, Tom Lane [EMAIL PROTECTED] wrote: Is this a regression, or a feature of 8.2? Hard to say without EXPLAIN ANALYZE output to compare. To my eye they are identical other than the speed but perhaps

Re: [PERFORM] High Load on Postgres 7.4.16 Server

2007-04-06 Thread John Allgood
Hey Guys thanks for the input. I do have some more questions. I am looking a doing some additional tuning on the system. The first think I am looking at is the OS tuning. What kernel parameters would I look at setting on a RHEL 4 box? I have set the SHMMAX and SHMALL to 1GB. What other tuning

Re: [PERFORM] SCSI vs SATA

2007-04-06 Thread Scott Marlowe
On Thu, 2007-04-05 at 23:37, Greg Smith wrote: On Thu, 5 Apr 2007, Scott Marlowe wrote: On Thu, 2007-04-05 at 14:30, James Mansion wrote: Can you cite any statistical evidence for this? Logic? OK, everyone who hasn't already needs to read the Google and CMU papers. I'll even provide

[PERFORM] more on high load on postgres 7.4.16

2007-04-06 Thread Geoffrey
We are trying to attack this problem from multiple avenues, thus I'm starting a separate thread. This is with regard to the problem posted via thread: http://archives.postgresql.org/pgsql-performance/2007-04/msg00120.php One thing we are seeing with this move to the new hardware (and rhas 4)

Re: [PERFORM] more on high load on postgres 7.4.16

2007-04-06 Thread Stefan Kaltenbrunner
Geoffrey wrote: We are trying to attack this problem from multiple avenues, thus I'm starting a separate thread. This is with regard to the problem posted via thread: http://archives.postgresql.org/pgsql-performance/2007-04/msg00120.php One thing we are seeing with this move to the new

Re: [PERFORM] SCSI vs SATA

2007-04-06 Thread Ron
At 09:23 AM 4/6/2007, Michael Stone wrote: On Fri, Apr 06, 2007 at 08:49:08AM -0400, Ron wrote: Not quite. Each of our professional experiences is +also+ statistical evidence. Even if it is a personally skewed sample. I'm not sure that word means what you think it means. I think the one

Re: [PERFORM] more on high load on postgres 7.4.16

2007-04-06 Thread Geoffrey
Stefan Kaltenbrunner wrote: Geoffrey wrote: We are trying to attack this problem from multiple avenues, thus I'm starting a separate thread. This is with regard to the problem posted via thread: http://archives.postgresql.org/pgsql-performance/2007-04/msg00120.php One thing we are seeing

Re: [PERFORM] Premature view materialization in 8.2?

2007-04-06 Thread Jonathan Ellis
On 4/6/07, Tom Lane [EMAIL PROTECTED] wrote: Jonathan Ellis [EMAIL PROTECTED] writes: On 4/6/07, Tom Lane [EMAIL PROTECTED] wrote: Yeah, it sure is the same plan, and 8.2 seems to be a tad faster right up to the hash join on user_id. Is user_id a textual datatype? user_id is an int; they

Re: [PERFORM] Premature view materialization in 8.2?

2007-04-06 Thread Tom Lane
Jonathan Ellis [EMAIL PROTECTED] writes: I can do that... you don't think the fact I mentioned, that redefining the view to leave out the expensive function fixes the problem, is relevant? Hm, I'd not have thought that an expensive function would get evaluated partway up the join tree, but

Re: [PERFORM] Premature view materialization in 8.2?

2007-04-06 Thread Jonathan Ellis
On 4/6/07, Tom Lane [EMAIL PROTECTED] wrote: Jonathan Ellis [EMAIL PROTECTED] writes: I can do that... you don't think the fact I mentioned, that redefining the view to leave out the expensive function fixes the problem, is relevant? Hm, I'd not have thought that an expensive function would

Re: [PERFORM] SCSI vs SATA

2007-04-06 Thread Michael Stone
On Fri, Apr 06, 2007 at 12:41:25PM -0400, Ron wrote: 3.based on personal observation, case study reports, or random investigations rather than systematic scientific evaluation: anecdotal evidence. Here you even quote the appropriate definition before ignoring it. In short, professional

Re: [PERFORM] Premature view materialization in 8.2?

2007-04-06 Thread Tom Lane
Jonathan Ellis [EMAIL PROTECTED] writes: It was in my original post unless it got clipped: Sorry, I had forgotten. The problem seems to be that clan_members_v contains a call to an expensive function: I'll bet that the function is marked VOLATILE. 8.2 is more conservative about optimizing

Re: [PERFORM] SCSI vs SATA

2007-04-06 Thread Ron
At 02:19 PM 4/6/2007, Michael Stone wrote: On Fri, Apr 06, 2007 at 12:41:25PM -0400, Ron wrote: 3.based on personal observation, case study reports, or random investigations rather than systematic scientific evaluation: anecdotal evidence. Here you even quote the appropriate definition

Re: [PERFORM] Premature view materialization in 8.2?

2007-04-06 Thread Jonathan Ellis
On 4/6/07, Tom Lane [EMAIL PROTECTED] wrote: The problem seems to be that clan_members_v contains a call to an expensive function: I'll bet that the function is marked VOLATILE. 8.2 is more conservative about optimizing away volatile functions than previous releases. If it has no side

[PERFORM] postgres 8.2 seems to prefer Seq Scan

2007-04-06 Thread Alex Deucher
One more anomaly between 7.4 and 8.2. DB dumped from 7.4 and loaded onto 8.2, both have locale set to C. 8.2 seems to prefer Seq Scans for the first query while the ordering in the second query seems to perform worse on 8.2. I ran analyze. I've tried with the encoding set to UTF-8 and

Re: [PERFORM] SCSI vs SATA

2007-04-06 Thread Greg Smith
On Fri, 6 Apr 2007, Scott Marlowe wrote: Most server drives are rated for 55-60C environmental temperature operation, which means the drive would be even hotter. I chuckled when I dug into the details for the drives in my cheap PC; the consumer drives from Seagate:

Re: [PERFORM] SCSI vs SATA

2007-04-06 Thread Michael Stone
On Fri, Apr 06, 2007 at 03:37:08PM -0400, Ron wrote: studies. I respect that. Unfortunately the RW is too fast moving and too messy to wait for a laboratory style study to be completed before we are called on to make professional decisions on most issues we face within our work IME I have to

Re: [PERFORM] postgres 8.2 seems to prefer Seq Scan

2007-04-06 Thread Michael Fuhr
On Fri, Apr 06, 2007 at 04:38:33PM -0400, Alex Deucher wrote: One more anomaly between 7.4 and 8.2. DB dumped from 7.4 and loaded onto 8.2, both have locale set to C. 8.2 seems to prefer Seq Scans for the first query while the ordering in the second query seems to perform worse on 8.2. I

Re: [PERFORM] postgres 8.2 seems to prefer Seq Scan

2007-04-06 Thread Alex Deucher
On 4/6/07, Michael Fuhr [EMAIL PROTECTED] wrote: On Fri, Apr 06, 2007 at 04:38:33PM -0400, Alex Deucher wrote: One more anomaly between 7.4 and 8.2. DB dumped from 7.4 and loaded onto 8.2, both have locale set to C. 8.2 seems to prefer Seq Scans for the first query while the ordering in the

Re: [PERFORM] SCSI vs SATA

2007-04-06 Thread david
On Fri, 6 Apr 2007, Scott Marlowe wrote: Based on experience I think that on average server drives are more reliable than consumer grade drives, and can take more punishment. this I am not sure about But, the variables of manufacturer, model, and the batch often make even more difference

Re: [PERFORM] SCSI vs SATA

2007-04-06 Thread Charles Sprickman
On Fri, 6 Apr 2007, [EMAIL PROTECTED] wrote: On Fri, 6 Apr 2007, Scott Marlowe wrote: Based on experience I think that on average server drives are more reliable than consumer grade drives, and can take more punishment. this I am not sure about I think they should survey Tivo owners next

Re: [PERFORM] SCSI vs SATA

2007-04-06 Thread david
On Fri, 6 Apr 2007, Charles Sprickman wrote: On Fri, 6 Apr 2007, [EMAIL PROTECTED] wrote: On Fri, 6 Apr 2007, Scott Marlowe wrote: Based on experience I think that on average server drives are more reliable than consumer grade drives, and can take more punishment. this I am not sure

Re: [PERFORM] SCSI vs SATA

2007-04-06 Thread Andreas Kostyrka
* Charles Sprickman [EMAIL PROTECTED] [070407 00:49]: On Fri, 6 Apr 2007, [EMAIL PROTECTED] wrote: On Fri, 6 Apr 2007, Scott Marlowe wrote: Based on experience I think that on average server drives are more reliable than consumer grade drives, and can take more punishment. this I am not

Re: [PERFORM] SCSI vs SATA

2007-04-06 Thread Mark Kirkwood
Ron wrote: I read them as soon as they were available. Then I shrugged and noted YMMV to myself. 1= Those studies are valid for =those= users under =those= users' circumstances in =those= users' environments. How well do those circumstances and environments mimic anyone else's? Exactly,

Re: [PERFORM] SCSI vs SATA

2007-04-06 Thread Bruce Momjian
In summary, it seems one of these is true: 1. Drive manufacturers don't design server drives to be more reliable than consumer drive 2. Drive manufacturers _do_ design server drives to be more reliable than consumer drive, but the design doesn't yield significantly better