Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-05-05 Thread Greg Sabino Mullane
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 The pgsql-admin list has just seen another instance where careless use of prepared transactions brought down a database, and the DBA (who had no idea what a prepared transaction even was) had no idea how to fix it. Just as a followup (and I

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-05-05 Thread Tom Lane
Greg Sabino Mullane g...@turnstep.com writes: Just as a followup (and I already posted this on the pgsql-admin thread), the check_postgres script now has a specific check for this very case. It simply checks the age of entries in pg_prepared_xacts and gives a warning if the number is at or

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-05-05 Thread Bernd Helmle
--On Dienstag, Mai 05, 2009 15:29:15 + Greg Sabino Mullane g...@turnstep.com wrote: It simply checks the age of entries in pg_prepared_xacts and gives a warning if the number is at or over the given threshhold (defaults to 1 second). I'm still a heavy +1 on making the default Postgres

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-05-05 Thread Greg Sabino Mullane
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 1 seconds seems a very low default for me. I can imagine that most distributed transactions are taking longer than this to complete. One second means it is set by default to catch *all* prepared transactions. It's simply checking how long

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-28 Thread Robert Treat
On Wednesday 22 April 2009 15:49:32 Tom Lane wrote: I wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Configuration affects what can be tested in installcheck, that's quite natural. I would be happy with simply adding an alternative expected output file for

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-23 Thread Kevin Grittner
Joshua D. Drake j...@commandprompt.com wrote: I am not opposed to making the default zero. +1 making zero the default for 8.4 I am also +1 on adding the warnings. +1, but less urgent, lower priority -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

[HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Tom Lane
The pgsql-admin list has just seen another instance where careless use of prepared transactions brought down a database, and the DBA (who had no idea what a prepared transaction even was) had no idea how to fix it. It seems to me we need to do something about making that stuff less DBA-unfriendly.

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Alvaro Herrera
Tom Lane wrote: Anyway, maybe question zero is whether anyone else thinks this is important enough to justify extra work in the area. One thing that has already changed is that DROP DATABASE reports N users and M prepared transactions, so there is more of a hint. Another thing we could do is

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Joshua D. Drake
On Wed, 2009-04-22 at 13:48 -0400, Tom Lane wrote: One line of thought is just to raise the visibility of old prepared transactions somehow. I don't think I want to go as far as, say, making every session-start issue WARNINGs about every prepared xact that's more than a few minutes old. But

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Heikki Linnakangas
Joshua D. Drake wrote: On Wed, 2009-04-22 at 13:48 -0400, Tom Lane wrote: The main objection to just setting max_prepared_transactions to zero by default is that it would kill our ability to test the feature in the standard regression tests. That kills it for me. Unless we want to change

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Tom Lane
Joshua D. Drake j...@commandprompt.com writes: On Wed, 2009-04-22 at 13:48 -0400, Tom Lane wrote: Another line of thought is that prepared xacts are inherently a bad thing to be using if you have not done careful setup of a lot of external infrastructure (in particular, have a transaction

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Greg Sabino Mullane
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 Anyway, maybe question zero is whether anyone else thinks this is important enough to justify extra work in the area. Yes. For every user that complains on the list, there are a dozen other quiet ones who have been bit by the same. The

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: I think we should change the way we test it. Could we simply make max_prepared_transactions = 0 the default, but put max_prepared_transactions = 5 into the config file in make check? That only works for make check, not make

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Andrew Dunstan
Heikki Linnakangas wrote: I think we should change the way we test it. Could we simply make max_prepared_transactions = 0 the default, but put max_prepared_transactions = 5 into the config file in make check? That seems like the best alternative, it doesn't seem right to build extra

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Heikki Linnakangas
Tom Lane wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: I think we should change the way we test it. Could we simply make max_prepared_transactions = 0 the default, but put max_prepared_transactions = 5 into the config file in make check? That only works for make

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Tom Lane wrote: That only works for make check, not make installcheck. Configuration affects what can be tested in installcheck, that's quite natural. I would be happy with simply adding an alternative expected output file for

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Jeff Davis
On Wed, 2009-04-22 at 11:00 -0700, Joshua D. Drake wrote: Then perhaps a setting like max_stale_prepared_transaction_age and once that threshold is met it will autorollback? I think that defeats the safety of prepared transactions in many cases. Let's say you PREPARE TRANSACTION on two systems,

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Tom Lane
Jeff Davis pg...@j-davis.com writes: On Wed, 2009-04-22 at 11:00 -0700, Joshua D. Drake wrote: Then perhaps a setting like max_stale_prepared_transaction_age and once that threshold is met it will autorollback? I think that defeats the safety of prepared transactions in many cases. Let's say

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Jeff Davis
On Wed, 2009-04-22 at 13:53 -0400, Alvaro Herrera wrote: Another thing we could do is make autovacuum log something about those, similar to what it does to temp tables. And if one of them gets too near Xid wraparound, kill it. As I said in my reply to Joshua, I don't think killing a prepared

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Heikki Linnakangas
Tom Lane wrote: Does a prepared xact still block vacuum cleanup in HEAD, or has that been fixed since 8.2? It still does. A prepared xact is just like a idle-in-transaction backend as far as vacuum is concerned. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Jeff Davis
On Wed, 2009-04-22 at 21:58 +0300, Heikki Linnakangas wrote: Tom Lane wrote: Does a prepared xact still block vacuum cleanup in HEAD, or has that been fixed since 8.2? It still does. A prepared xact is just like a idle-in-transaction backend as far as vacuum is concerned. I thought idle

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Heikki Linnakangas
Jeff Davis wrote: On Wed, 2009-04-22 at 21:58 +0300, Heikki Linnakangas wrote: Tom Lane wrote: Does a prepared xact still block vacuum cleanup in HEAD, or has that been fixed since 8.2? It still does. A prepared xact is just like a idle-in-transaction backend as far as vacuum is concerned.

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Tom Lane
I wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Configuration affects what can be tested in installcheck, that's quite natural. I would be happy with simply adding an alternative expected output file for min_prepared_xacts=0 case. Like we've done for xml test cases,

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Joshua D. Drake
On Wed, 2009-04-22 at 15:49 -0400, Tom Lane wrote: I wrote: Warning about very old prepared transactions is something that we could think about doing as well; it doesn't have to be either-or. I think the need for it would decrease quite a bit if they weren't enabled by default, though.

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Tom Lane
Joshua D. Drake j...@commandprompt.com writes: On Wed, 2009-04-22 at 15:49 -0400, Tom Lane wrote: Comments? Anyone seriously opposed to making the default be zero? I am not opposed to making the default zero. I am also +1 on adding the warnings. What I think we could/should do about that

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Robert Haas
On Wed, Apr 22, 2009 at 2:58 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Tom Lane wrote: Does a prepared xact still block vacuum cleanup in HEAD, or has that been fixed since 8.2? It still does. A prepared xact is just like a idle-in-transaction backend as far as

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Wed, Apr 22, 2009 at 2:58 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: It still does. A prepared xact is just like a idle-in-transaction backend as far as vacuum is concerned. Is that really necessary? It's true that you can't

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Robert Haas
On Wed, Apr 22, 2009 at 5:44 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Apr 22, 2009 at 2:58 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: It still does. A prepared xact is just like a idle-in-transaction backend as far as

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Wed, Apr 22, 2009 at 5:44 PM, Tom Lane t...@sss.pgh.pa.us wrote: I think we've already milked what we can from that, since a prepared xact is treated exactly like an open one with no snapshot.  The point is that whatever rows it's written are still

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Robert Haas
On Wed, Apr 22, 2009 at 8:58 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Apr 22, 2009 at 5:44 PM, Tom Lane t...@sss.pgh.pa.us wrote: I think we've already milked what we can from that, since a prepared xact is treated exactly like an open one with

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Robert Haas
On Wed, Apr 22, 2009 at 9:21 PM, Robert Haas robertmh...@gmail.com wrote: Maybe I'm just dumb, but I don't get it.  If I start a transaction and do SELECT * FROM foo and then wait around for an hour or two while someone else makes changes to foo and then do SELECT * FROM foo again, I expect to

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Maybe I'm just dumb, but I don't get it. If I start a transaction and do SELECT * FROM foo and then wait around for an hour or two while someone else makes changes to foo and then do SELECT * FROM foo again, I expect to see the same rows I saw the