Re: [PATCHES] Async Commit, v21 (now: v22)

2007-08-03 Thread Simon Riggs
On Wed, 2007-08-01 at 18:54 -0400, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: Here's v23, including all suggested changes, plus some reworking of the transaction APIs to reduce the footprint of the patch. Applied with some editorialization --- Thanks I found a few bugs, as

Re: [PATCHES] Async Commit, v21 (now: v22)

2007-07-25 Thread Simon Riggs
On Tue, 2007-07-24 at 17:53 -0400, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: There is an explicit test for whether the transaction has modified files; if so the commit is always synchronous, even if explicitly requested otherwise. Also, utility commands never perform async

Re: [PATCHES] Async Commit, v21 (now: v22)

2007-07-24 Thread Gregory Stark
Tom Lane [EMAIL PROTECTED] writes: I wrote: (BTW, in case you can't tell from the drift of my questions, I've separated the patch into add background wal writer and add async commit, and am working on the first half.) I've committed the first half of that. Something that still needs

Re: [PATCHES] Async Commit, v21 (now: v22)

2007-07-24 Thread Simon Riggs
On Mon, 2007-07-23 at 20:02 -0400, Alvaro Herrera wrote: Simon Riggs wrote: Autovac is the most clean implementation of a special process, so seemed like a good prototype. I'd thought I'd combed out any pointless code though. What, you mean there's pointless code in autovac? Hey, be

Re: [PATCHES] Async Commit, v21 (now: v22)

2007-07-24 Thread Simon Riggs
On Tue, 2007-07-24 at 00:58 -0400, Tom Lane wrote: I wrote: (BTW, in case you can't tell from the drift of my questions, I've separated the patch into add background wal writer and add async commit, and am working on the first half.) I've committed the first half of that. Cool

Re: [PATCHES] Async Commit, v21 (now: v22)

2007-07-24 Thread Simon Riggs
On Mon, 2007-07-23 at 21:06 -0400, Tom Lane wrote: I came across another point worthy of mention: as given, the patch turns XLogWrite's flexible write logic into dead code, because there are no callers that pass flexible = true. We could rip that out, but it seems to me there's still some

Re: [PATCHES] Async Commit, v21 (now: v22)

2007-07-24 Thread Simon Riggs
On Mon, 2007-07-23 at 21:11 -0400, Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: What's the thing about doing the flush twice in a couple of comments in calls to XLogBackgroundFlush? Are they just leftover comments from older code? I was wondering that too --- they looked

Re: [PATCHES] Async Commit, v21 (now: v22)

2007-07-24 Thread Florian Weimer
+para + Asynchronous commit provides different behaviour to setting + varnamefsync/varname = off, which is a server-wide + setting that will alter the behaviour of all transactions, + overriding the setting of varnamesynchronous_commit/varname, + as well as risking much

Re: [PATCHES] Async Commit, v21 (now: v22)

2007-07-24 Thread Simon Riggs
On Tue, 2007-07-24 at 10:51 +0200, Florian Weimer wrote: +para + Asynchronous commit provides different behaviour to setting + varnamefsync/varname = off, which is a server-wide + setting that will alter the behaviour of all transactions, + overriding the setting of

Re: [PATCHES] Async Commit, v21 (now: v22)

2007-07-24 Thread Florian Weimer
* Simon Riggs: I think fsync=off also endagers metadata, while synchronous_commit=off should be perfectly safe as far as the metadata is concerned. Wouldn't this be worth mentioning as well? Well, I think wider data loss covers it for me, but I don't have a problem with people wanting to

Re: [PATCHES] Async Commit, v21 (now: v22)

2007-07-24 Thread Tom Lane
Gregory Stark [EMAIL PROTECTED] writes: Without async commits? Do we really want the walwriter doing the majority of the wal-flushing work for normal commits? It seems like that's not going to be any advantage over just having some random backend do the commit. Sure: the advantage is that the

Re: [PATCHES] Async Commit, v21 (now: v22)

2007-07-24 Thread Greg Smith
On Tue, 24 Jul 2007, Gregory Stark wrote: Do we really want the walwriter doing the majority of the wal-flushing work for normal commits? It seems like that's not going to be any advantage over just having some random backend do the commit. Might there be some advantage in

Re: [PATCHES] Async Commit, v21 (now: v22)

2007-07-24 Thread Heikki Linnakangas
Tom Lane wrote: Gregory Stark [EMAIL PROTECTED] writes: Without async commits? Do we really want the walwriter doing the majority of the wal-flushing work for normal commits? It seems like that's not going to be any advantage over just having some random backend do the commit. Sure: the

Re: [PATCHES] Async Commit, v21 (now: v22)

2007-07-24 Thread Gregory Stark
Tom Lane [EMAIL PROTECTED] writes: Gregory Stark [EMAIL PROTECTED] writes: Without async commits? Do we really want the walwriter doing the majority of the wal-flushing work for normal commits? It seems like that's not going to be any advantage over just having some random backend do the

Re: [PATCHES] Async Commit, v21 (now: v22)

2007-07-24 Thread Tom Lane
Gregory Stark [EMAIL PROTECTED] writes: Tom Lane [EMAIL PROTECTED] writes: Sure: the advantage is that the backends (ie, user query processing) don't get blocked on fsync's. This is not really different from the rationale for having the bgwriter. I'm puzzled though. How do they not get

Re: [PATCHES] Async Commit, v21 (now: v22)

2007-07-24 Thread Alvaro Herrera
Florian Weimer wrote: I think fsync=off also endagers metadata, while synchronous_commit=off should be perfectly safe as far as the metadata is concerned. Wouldn't this be worth mentioning as well? Is it true that a transaction is turned into sync commit as soon as it writes on a system

Re: [PATCHES] Async Commit, v21 (now: v22)

2007-07-24 Thread Jim C. Nasby
On Tue, Jul 24, 2007 at 02:08:00PM -0400, Alvaro Herrera wrote: Florian Weimer wrote: I think fsync=off also endagers metadata, while synchronous_commit=off should be perfectly safe as far as the metadata is concerned. Wouldn't this be worth mentioning as well? Is it true that a

Re: [PATCHES] Async Commit, v21 (now: v22)

2007-07-24 Thread Tom Lane
Jim C. Nasby [EMAIL PROTECTED] writes: On Tue, Jul 24, 2007 at 02:08:00PM -0400, Alvaro Herrera wrote: Is it true that a transaction is turned into sync commit as soon as it writes on a system catalog? Is it desirable to make it so? If we don't do that then regular users have the ability to

Re: [PATCHES] Async Commit, v21 (now: v22)

2007-07-24 Thread Simon Riggs
On Tue, 2007-07-24 at 16:25 -0400, Tom Lane wrote: Jim C. Nasby [EMAIL PROTECTED] writes: On Tue, Jul 24, 2007 at 02:08:00PM -0400, Alvaro Herrera wrote: Is it true that a transaction is turned into sync commit as soon as it writes on a system catalog? Is it desirable to make it so? If

Re: [PATCHES] Async Commit, v21 (now: v22)

2007-07-24 Thread Gregory Stark
Tom Lane [EMAIL PROTECTED] writes: If we don't do that then regular users have the ability to put the catalog (and by extension everything else) at risk... How do you arrive at that conclusion? The point of the async commit patch is that transactions might be lost, as in not really

Re: [PATCHES] Async Commit, v21 (now: v22)

2007-07-24 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: There is an explicit test for whether the transaction has modified files; if so the commit is always synchronous, even if explicitly requested otherwise. Also, utility commands never perform async commits, so overall there aren't that many of the commonly

Re: [PATCHES] Async Commit, v21 (now: v22)

2007-07-23 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: Thanks for reading. Updated version in new patch. What was the reasoning for basing walwriter.c on autovacuum (which needs to be able to execute transactions) rather than bgwriter (which does not)? The shutdown logic in particular seems all wrong; you can't

Re: [PATCHES] Async Commit, v21 (now: v22)

2007-07-23 Thread Simon Riggs
On Mon, 2007-07-23 at 18:59 -0400, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: Thanks for reading. Updated version in new patch. What was the reasoning for basing walwriter.c on autovacuum (which needs to be able to execute transactions) rather than bgwriter (which does not)?

Re: [PATCHES] Async Commit, v21 (now: v22)

2007-07-23 Thread Alvaro Herrera
Simon Riggs wrote: Autovac is the most clean implementation of a special process, so seemed like a good prototype. I'd thought I'd combed out any pointless code though. What, you mean there's pointless code in autovac? Hey, be sure to let me know about it! -- Alvaro Herrera

Re: [PATCHES] Async Commit, v21 (now: v22)

2007-07-23 Thread Alvaro Herrera
Simon Riggs wrote: Here's v23, including all suggested changes, plus some reworking of the transaction APIs to reduce the footprint of the patch. What's the thing about doing the flush twice in a couple of comments in calls to XLogBackgroundFlush? Are they just leftover comments from older

Re: [PATCHES] Async Commit, v21 (now: v22)

2007-07-23 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: On Mon, 2007-07-23 at 18:59 -0400, Tom Lane wrote: The shutdown logic in particular seems all wrong; you can't have a process connected to shared memory that is going to outlive the postmaster. It seemed to work cleanly when I tested it initially, but

Re: [PATCHES] Async Commit, v21 (now: v22)

2007-07-23 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: What's the thing about doing the flush twice in a couple of comments in calls to XLogBackgroundFlush? Are they just leftover comments from older code? I was wondering that too --- they looked like obsolete comments to me. My current thinking BTW is

Re: [PATCHES] Async Commit, v21 (now: v22)

2007-07-23 Thread Tom Lane
I wrote: (BTW, in case you can't tell from the drift of my questions, I've separated the patch into add background wal writer and add async commit, and am working on the first half.) I've committed the first half of that. Something that still needs investigation is what the default

Re: [PATCHES] Async Commit, v21 (now: v22)

2007-07-19 Thread Simon Riggs
On Wed, 2007-07-18 at 17:16 +0100, Simon Riggs wrote: On Wed, 2007-07-18 at 12:04 -0400, Alvaro Herrera wrote: Simon Riggs wrote: OK. Will do, thanks. Make sure to remove the bogus comment about pgstat considers our data as gone in walwriter.c as well (in the sigjmp block)

Re: [PATCHES] Async Commit, v21 (now: v22)

2007-07-18 Thread Peter Eisentraut
Am Dienstag, 17. Juli 2007 20:31 schrieb Simon Riggs: Here's the latest version. I've reviewed this to check that this does what I want it to do, re-written various comments and changed a few minor points in the code. I've also added a chunk to transam/README that describes the workings of

Re: [PATCHES] Async Commit, v21 (now: v22)

2007-07-18 Thread Heikki Linnakangas
Simon Riggs wrote: Here's the latest version. I've reviewed this to check that this does what I want it to do, re-written various comments and changed a few minor points in the code. I've also added a chunk to transam/README that describes the workings of the patch from a high level. Now

Re: [PATCHES] Async Commit, v21 (now: v22)

2007-07-18 Thread Simon Riggs
On Wed, 2007-07-18 at 16:44 +0100, Heikki Linnakangas wrote: Simon Riggs wrote: Here's the latest version. I've reviewed this to check that this does what I want it to do, re-written various comments and changed a few minor points in the code. I've also added a chunk to transam/README

Re: [PATCHES] Async Commit, v21 (now: v22)

2007-07-18 Thread Alvaro Herrera
Simon Riggs wrote: OK. Will do, thanks. Make sure to remove the bogus comment about pgstat considers our data as gone in walwriter.c as well (in the sigjmp block) -- Alvaro Herrera Developer, http://www.PostgreSQL.org/ Those who use electric razors are infidels

Re: [PATCHES] Async Commit, v21 (now: v22)

2007-07-18 Thread Simon Riggs
On Wed, 2007-07-18 at 12:04 -0400, Alvaro Herrera wrote: Simon Riggs wrote: OK. Will do, thanks. Make sure to remove the bogus comment about pgstat considers our data as gone in walwriter.c as well (in the sigjmp block) Thanks, its gone in v23. -- Simon Riggs EnterpriseDB

Re: [PATCHES] Async Commit, v21 (now: v22)

2007-07-17 Thread Simon Riggs
On Tue, 2007-07-17 at 01:28 -0400, Bruce Momjian wrote: Your patch has been added to the PostgreSQL unapplied patches list at: http://momjian.postgresql.org/cgi-bin/pgpatches It will be applied as soon as one of the PostgreSQL committers reviews and approves it.

Re: [PATCHES] Async Commit, v21 (now: v22)

2007-07-17 Thread Bruce Momjian
Your patch has been added to the PostgreSQL unapplied patches list at: http://momjian.postgresql.org/cgi-bin/pgpatches It will be applied as soon as one of the PostgreSQL committers reviews and approves it. ---