Re: [PERFORM] limiting performance impact of wal archiving.

2009-11-16 Thread Laurent Laborde
On Thu, Nov 12, 2009 at 3:21 PM, Laurent Laborde kerdez...@gmail.com wrote: Hi ! Here is my plan : - rebuilding a spare with ext3, raid10, without lvm - switch the slony master to this new node. Done 3 days ago : Problem solved ! It totally worked. \o/ -- ker2x sysadmin DBA @

Re: [PERFORM] Unexpected sequential scan on an indexed column

2009-11-16 Thread Eddy Escardo-Raffo
OK, I think that after reading this dochttp://www.postgresql.org/files/developer/optimizer.pdf (which I hadn't encountered before) about the optimizer, something clicked in my brain and I think I can answer my own question. I was basically thinking from my own perspective rather than from the

Re: [PERFORM] Manual vacs 5x faster than autovacs?

2009-11-16 Thread Wayne Beaver
Quoting Scott Marlowe scott.marl...@gmail.com: On Thu, Nov 12, 2009 at 9:58 AM, Wayne Beaver wa...@acedsl.com wrote: Quoting Scott Marlowe scott.marl...@gmail.com: On Thu, Nov 12, 2009 at 9:14 AM, Wayne Beaver wa...@acedsl.com wrote: I'd seen autovacs running for hours and had mis-attributed

Re: [PERFORM] Manual vacs 5x faster than autovacs?

2009-11-16 Thread Scott Marlowe
On Mon, Nov 16, 2009 at 9:13 AM, Wayne Beaver wa...@acedsl.com wrote: Quoting Scott Marlowe scott.marl...@gmail.com: On Thu, Nov 12, 2009 at 9:58 AM, Wayne Beaver wa...@acedsl.com wrote: Quoting Scott Marlowe scott.marl...@gmail.com: On Thu, Nov 12, 2009 at 9:14 AM, Wayne Beaver

Re: [PERFORM] Manual vacs 5x faster than autovacs?

2009-11-16 Thread Wayne Beaver
Quoting Scott Marlowe scott.marl...@gmail.com: On Mon, Nov 16, 2009 at 9:13 AM, Wayne Beaver wa...@acedsl.com wrote: Quoting Scott Marlowe scott.marl...@gmail.com: On Thu, Nov 12, 2009 at 9:58 AM, Wayne Beaver wa...@acedsl.com wrote: Quoting Scott Marlowe scott.marl...@gmail.com: On Thu,

Re: [PERFORM] Unexpected sequential scan on an indexed column

2009-11-16 Thread Dave Crooke
Hi Eddy Perhaps a slightly naive suggestion have you considered converting the query to a small stored procedure ('function' in Postgres speak)? You can pull the location values, and then iterate over a query like this: select userid from users where location=:x which is more-or-less

[PERFORM] Is Diskeeper Automatic Mode safe?

2009-11-16 Thread cb
I've got a pair of servers running PostgreSQL 8.0.4 on Windows. We have several tables that add and delete massive amounts of data in a single day and are increasingly having a problem with drive fragmentation and it appears to be giving us a decent performance hit. This is external

Re: [PERFORM] Is Diskeeper Automatic Mode safe?

2009-11-16 Thread Robert Haas
On Mon, Nov 16, 2009 at 12:14 PM, cb c...@mythtech.net wrote: I've got a pair of servers running PostgreSQL 8.0.4 on Windows. We have several tables that add and delete massive amounts of data in a single day and are increasingly having a problem with drive fragmentation and it appears to be

Re: [PERFORM] Is Diskeeper Automatic Mode safe?

2009-11-16 Thread Robert Schnabel
cb wrote: I'm curious if anyone else has used Diskeeper's Automatic Mode in combination with PostgreSQL to defrag and keep the drive defragged while PostgreSQL is actually running. Thanks! -chris www.mythtech.net I've been a Diskeeper customer for about 10 years now and consider it

Re: [PERFORM] Is Diskeeper Automatic Mode safe?

2009-11-16 Thread Robert Haas
On Mon, Nov 16, 2009 at 1:11 PM, Robert Schnabel schnab...@missouri.edu wrote: cb wrote: I'm curious if anyone else has used Diskeeper's Automatic Mode in  combination with PostgreSQL to defrag and keep the drive defragged  while PostgreSQL is actually running. Thanks! -chris

Re: [PERFORM] Is Diskeeper Automatic Mode safe?

2009-11-16 Thread Robert Schnabel
So the short answer is yes, I have it running with PostgreSQL and have not had any problems. Have you unplugged the power cord a few times in the middle of heavy write activity? ...Robert Nope. Forgive my ignorance but isn't that what a UPS is for anyway? Along with a BBU

Re: [PERFORM] Is Diskeeper Automatic Mode safe?

2009-11-16 Thread Scott Marlowe
On Mon, Nov 16, 2009 at 1:04 PM, Robert Schnabel schnab...@missouri.edu wrote:  So the short answer is yes, I have it running with PostgreSQL and have not had any problems. Have you unplugged the power cord a few times in the middle of heavy write activity? ...Robert Nope.  Forgive my

Re: [PERFORM] Is Diskeeper Automatic Mode safe?

2009-11-16 Thread Scott Marlowe
On Mon, Nov 16, 2009 at 1:12 PM, Scott Marlowe scott.marl...@gmail.com wrote: Power supplies / UPSes fail far more often than one might think.  And a db that doesn't come back up afterwards is not to be placed into production. Note that there are uses for databases that can lose everything and

Re: [PERFORM] Is Diskeeper Automatic Mode safe?

2009-11-16 Thread Greg Smith
Robert Schnabel wrote: Nope. Forgive my ignorance but isn't that what a UPS is for anyway? Along with a BBU controller. If you have a UPS *and* a BBU controller, then things are much better--those should have a write cache that insulates you from the worst of the problems. But just a UPS

Re: [PERFORM] Is Diskeeper Automatic Mode safe?

2009-11-16 Thread Karl Denninger
Greg Smith wrote: Robert Schnabel wrote: Nope. Forgive my ignorance but isn't that what a UPS is for anyway? Along with a BBU controller. If you have a UPS *and* a BBU controller, then things are much better--those should have a write cache that insulates you from the worst of the

Re: [PERFORM] Is Diskeeper Automatic Mode safe?

2009-11-16 Thread Dave Crooke
My reply about server failure was shwoing what could go wrong at the server level assuming a first-class, properly run data center, with fully redundant power, including a server with dual power supplies on separate cords fed by separate UPS'es etc. Unfortunately, *correctly* configured A/B

Re: [PERFORM] Is Diskeeper Automatic Mode safe?

2009-11-16 Thread Robert Schnabel
Scott Marlowe wrote: On Mon, Nov 16, 2009 at 1:04 PM, Robert Schnabel schnab...@missouri.edu wrote: So the short answer is yes, I have it running with PostgreSQL and have not had any problems. Have you unplugged the power cord a few times in the middle of heavy write activity?

Re: [PERFORM] Is Diskeeper Automatic Mode safe?

2009-11-16 Thread Karl Denninger
Dave Crooke wrote: My reply about server failure was shwoing what could go wrong at the server level assuming a first-class, properly run data center, with fully redundant power, including a server with dual power supplies on separate cords fed by separate UPS'es etc. Never had a

Re: [PERFORM] Unexpected sequential scan on an indexed column

2009-11-16 Thread Eddy Escardo-Raffo
Yeah this kind of thing would probably work. Doing this in java with separate queries would be easy to code but require multiple round trips. Doing it as a stored procedure would be nicer but I'd have to think a little more about how to refactor the java code around the query to make this happen.

Re: [PERFORM] Is Diskeeper Automatic Mode safe?

2009-11-16 Thread Scott Marlowe
On Mon, Nov 16, 2009 at 1:32 PM, Robert Schnabel schnab...@missouri.edu wrote: Ok, so you have sufficiently sparked my curiosity as to whether Diskeeper will in any way cause Postgres to fail the power chord test.  Unfortunately I have some deadlines to meet so won't be able to test this out

Re: [PERFORM] Is Diskeeper Automatic Mode safe?

2009-11-16 Thread Scott Marlowe
On Mon, Nov 16, 2009 at 1:32 PM, Karl Denninger k...@denninger.net wrote: Dave Crooke wrote: My reply about server failure was shwoing what could go wrong at the server level assuming a first-class, properly run data center, with fully redundant power, including a server with dual power

Re: [PERFORM] Unexpected sequential scan on an indexed column

2009-11-16 Thread Kenneth Marshall
On Mon, Nov 16, 2009 at 12:45:46PM -0800, Eddy Escardo-Raffo wrote: Yeah this kind of thing would probably work. Doing this in java with separate queries would be easy to code but require multiple round trips. Doing it as a stored procedure would be nicer but I'd have to think a little more

Re: [PERFORM] Is Diskeeper Automatic Mode safe?

2009-11-16 Thread Robert Schnabel
Scott Marlowe wrote: On Mon, Nov 16, 2009 at 1:32 PM, Robert Schnabel schnab...@missouri.edu wrote: Ok, so you have sufficiently sparked my curiosity as to whether Diskeeper will in any way cause Postgres to fail the power chord test. Unfortunately I have some deadlines to meet so

Re: [PERFORM] Is Diskeeper Automatic Mode safe?

2009-11-16 Thread Craig James
I also have backup software running that does complete drive imaging so I should be able to do this fairly safely. Here is the plan... 1) Shut down the Diskeeper service, run a query that is write heavy and then pull the chord on the box. Wait a few minutes then plug it back in and see if

Re: [PERFORM] Is Diskeeper Automatic Mode safe?

2009-11-16 Thread Alvaro Herrera
Craig James escribió: Do it more than once. This is a highly erratic test that can catch your system at a wide variety of points, some of which cause no problems, and some of which can be catastrophic. If you test and it fails, you know you have a problem. If you test and it doesn't fail,

Re: [PERFORM] Is Diskeeper Automatic Mode safe?

2009-11-16 Thread John Rouillard
On Mon, Nov 16, 2009 at 03:20:12PM -0500, Greg Smith wrote: Robert Schnabel wrote: Nope. Forgive my ignorance but isn't that what a UPS is for anyway? Along with a BBU controller. If you have a UPS *and* a BBU controller, then things are much better--those should have a write cache that

Re: [PERFORM] Unexpected sequential scan on an indexed column

2009-11-16 Thread Eddy Escardo-Raffo
This is incredibly helpful, Kenneth. I didn't know about the SETOF syntax at all. This could help minimize the amount of refactoring I need to do. Thanks! Eddy On Mon, Nov 16, 2009 at 12:55 PM, Kenneth Marshall k...@rice.edu wrote: On Mon, Nov 16, 2009 at 12:45:46PM -0800, Eddy Escardo-Raffo

Re: [PERFORM] Unexpected sequential scan on an indexed column

2009-11-16 Thread Dave Crooke
With Postgres, you can transparently replace a regular select with a function that takes the same types and returns a record iterator with the same columns. The only change needed is the SQL used to invoke it, you won't need any logic changes in your app code (Java or whatever), e.g. *select

Re: [PERFORM] Unexpected sequential scan on an indexed column

2009-11-16 Thread Eddy Escardo-Raffo
Thanks, Dave. Eddy On Mon, Nov 16, 2009 at 1:52 PM, Dave Crooke dcro...@gmail.com wrote: With Postgres, you can transparently replace a regular select with a function that takes the same types and returns a record iterator with the same columns. The only change needed is the SQL used to

Re: [PERFORM] Is Diskeeper Automatic Mode safe?

2009-11-16 Thread Scott Marlowe
On Mon, Nov 16, 2009 at 2:04 PM, Robert Schnabel schnab...@missouri.edu wrote: Granted, but the point of me testing this is to say whether or not the Diskeeper service could introduce a problem.  If the system recovers without Diskeeper running but does not recover while Diskeeper is actively

Re: [PERFORM] Is Diskeeper Automatic Mode safe?

2009-11-16 Thread cb
On Nov 16, 2009, at 1:09 PM, Robert Haas wrote: I'm not sure what the answer is to your actual question, but I'd highly recommend upgrading to 8.3 or 8.4. The performance is likely to be a lot better, and 8.0/8.1 are no longer supported on Windows. Ugh, yeah, I'd love to upgrade but the

Re: [PERFORM] Is Diskeeper Automatic Mode safe?

2009-11-16 Thread cb
On Nov 16, 2009, at 1:11 PM, Robert Schnabel wrote: I've been a Diskeeper customer for about 10 years now and consider it 'must have' software for Windows machines. snip So the short answer is yes, I have it running with PostgreSQL and have not had any problems. So that seems to be a

Re: [PERFORM] Is Diskeeper Automatic Mode safe?

2009-11-16 Thread Tom Lane
cb c...@mythtech.net writes: Ugh, yeah, I'd love to upgrade but the powers that get to make that decision have no interest in upgrading. So I'm stuck on 8.0.4, Make sure you're not in the line of fire when (not if) that version eats your data. Particularly on Windows, insisting on not

Re: [PERFORM] Is Diskeeper Automatic Mode safe?

2009-11-16 Thread Greg Smith
Tom Lane wrote: cb c...@mythtech.net writes: Ugh, yeah, I'd love to upgrade but the powers that get to make that decision have no interest in upgrading. So I'm stuck on 8.0.4, Make sure you're not in the line of fire when (not if) that version eats your data. Particularly on

Re: [PERFORM] Is Diskeeper Automatic Mode safe?

2009-11-16 Thread Scott Marlowe
On Mon, Nov 16, 2009 at 7:45 PM, Greg Smith g...@2ndquadrant.com wrote: Tom Lane wrote: cb c...@mythtech.net writes: Ugh, yeah, I'd love to upgrade but the powers that get to make that decision have no interest in upgrading. So I'm stuck on 8.0.4, Make sure you're not in the line of fire

Re: [PERFORM] Is Diskeeper Automatic Mode safe?

2009-11-16 Thread cb
On Nov 16, 2009, at 8:31 PM, Tom Lane wrote: Make sure you're not in the line of fire when (not if) that version eats your data. Particularly on Windows, insisting on not upgrading that version is unbelievably, irresponsibly stupid. There are a *large* number of known bugs. I hear ya, and