Re: [HACKERS] Keeping separate WAL segments for each database

2010-07-03 Thread Robert Haas
2010/6/30 Tom Lane t...@sss.pgh.pa.us: Robert Haas robertmh...@gmail.com writes: You need to make sure not only that you replay commit records in order, but also that, for example, you don't replay an XLOG_HEAP2_CLEAN record too early. Hm, good point.  That probably means that you *do* need

Re: [HACKERS] Keeping separate WAL segments for each database

2010-07-03 Thread Simon Riggs
On Wed, 2010-06-30 at 22:21 -0400, Tom Lane wrote: What about having a single WAL stream for all commit records (thereby avoiding any possible xact-serialization funnies) and other WAL records divided up among multiple streams in some fashion or other? A commit record would bear minimum-LSN

Re: [HACKERS] Keeping separate WAL segments for each database

2010-07-03 Thread Robert Haas
On Sat, Jul 3, 2010 at 10:46 AM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, 2010-06-30 at 22:21 -0400, Tom Lane wrote: What about having a single WAL stream for all commit records (thereby avoiding any possible xact-serialization funnies) and other WAL records divided up among multiple

[HACKERS] Keeping separate WAL segments for each database

2010-06-30 Thread Devrim GÜNDÜZ
I talked to Sybase people about their latest technologies. Many features that shipped with their latest ASE 15.5 product has been in PostgreSQL since many years (like functional indexes, etc). :) One of the things that interested me was parallel recovery feature. They said that they are keeping

Re: [HACKERS] Keeping separate WAL segments for each database

2010-06-30 Thread Alvaro Herrera
Excerpts from Devrim GÜNDÜZ's message of mié jun 30 14:54:06 -0400 2010: One of the things that interested me was parallel recovery feature. They said that they are keeping separate xlogs for each database, which speeds ups recovery in case of a crash. It also would increase performance,

Re: [HACKERS] Keeping separate WAL segments for each database

2010-06-30 Thread Robert Haas
2010/6/30 Alvaro Herrera alvhe...@commandprompt.com: Excerpts from Devrim GÜNDÜZ's message of mié jun 30 14:54:06 -0400 2010: One of the things that interested me was parallel recovery feature. They said that they are keeping separate xlogs for each database, which speeds ups recovery in case

Re: [HACKERS] Keeping separate WAL segments for each database

2010-06-30 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: I think one per database and one extra one for the shared catalogs would be enough. Most transactions would either touch either just the database, or just the shared catalogs, so you'd write the commit record in whichever stream was appropriate. If

Re: [HACKERS] Keeping separate WAL segments for each database

2010-06-30 Thread Robert Haas
2010/6/30 Tom Lane t...@sss.pgh.pa.us: Robert Haas robertmh...@gmail.com writes: I think one per database and one extra one for the shared catalogs would be enough.  Most transactions would either touch either just the database, or just the shared catalogs, so you'd write the commit record in

Re: [HACKERS] Keeping separate WAL segments for each database

2010-06-30 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: 2010/6/30 Tom Lane t...@sss.pgh.pa.us: Surely you'd have to roll back, not commit, in that situation.  You have no excuse for assuming that you've replayed all effects of the transaction. Hmm, good point. But you could make it work either way, I

Re: [HACKERS] Keeping separate WAL segments for each database

2010-06-30 Thread Robert Haas
2010/6/30 Tom Lane t...@sss.pgh.pa.us: (thinks some more...)  Maybe you don't even need the fencepoint record per se.  I think all it's doing for you is making sure you don't process commit records on different streams out-of-order.  There might be some other, more direct way to do that.

Re: [HACKERS] Keeping separate WAL segments for each database

2010-06-30 Thread Joe Conway
On 06/30/2010 05:52 PM, Robert Haas wrote: And at any rate, the per-database thing isn't really the design goal, anyway. FWIW, I've run into more than one client where PITR and/or warm standby on a per-database level would be a killer feature. Joe signature.asc Description: OpenPGP digital

Re: [HACKERS] Keeping separate WAL segments for each database

2010-06-30 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: You need to make sure not only that you replay commit records in order, but also that, for example, you don't replay an XLOG_HEAP2_CLEAN record too early. Hm, good point. That probably means that you *do* need fencepost records, and furthermore that