Re: [HACKERS] draft RFC: concept for partial, wal-based replication

2009-11-30 Thread Craig Ringer
On 30/11/2009 11:07 PM, Tom Lane wrote: Craig Ringer writes: Just a side note: in addition to its use for partial replication, this might have potential for performance-prioritizing databases or tablespaces. Being able to separate WAL logging so that different DBs, tablespaces, etc went to d

Re: [HACKERS] draft RFC: concept for partial, wal-based replication

2009-11-30 Thread Andres Freund
On Monday 30 November 2009 17:46:45 Hans-Jürgen Schönig wrote: > On Nov 30, 2009, at 10:32 AM, Stefan Kaltenbrunner wrote: > > the question is if filtering on the sending side is actually the > > "right thing" to do. > > It increases the overhead and the complexity on the master, > > especially if

Re: [HACKERS] draft RFC: concept for partial, wal-based replication

2009-11-30 Thread Hans-Jürgen Schönig
Just a side note: in addition to its use for partial replication, this might have potential for performance-prioritizing databases or tablespaces. hello ... this is an absolutely non-starter. the WAL is designed to be "hyper ordered" and hyper critical. once you fuck up order you will

Re: [HACKERS] draft RFC: concept for partial, wal-based replication

2009-11-30 Thread Hans-Jürgen Schönig
On Nov 30, 2009, at 10:32 AM, Stefan Kaltenbrunner wrote: Andres Freund wrote: On Monday 30 November 2009 03:57:11 Itagaki Takahiro wrote: Boszormenyi Zoltan wrote: we tried to discuss on a lower level what should be needed for a partial replication based on streaming replication. We need t

Re: [HACKERS] draft RFC: concept for partial, wal-based replication

2009-11-30 Thread Tom Lane
Craig Ringer writes: > Just a side note: in addition to its use for partial replication, this > might have potential for performance-prioritizing databases or tablespaces. > Being able to separate WAL logging so that different DBs, tablespaces, > etc went to different sets of WAL logs would allow

Re: [HACKERS] draft RFC: concept for partial, wal-based replication

2009-11-30 Thread Robert Haas
On Nov 30, 2009, at 1:55 AM, Craig Ringer wrote: Boszormenyi Zoltan wrote: c. splitting wal into different replication sets Just a side note: in addition to its use for partial replication, this might have potential for performance-prioritizing databases or tablespaces. Being able to

Re: [HACKERS] draft RFC: concept for partial, wal-based replication

2009-11-30 Thread Andres Freund
On Monday 30 November 2009 10:32:50 Stefan Kaltenbrunner wrote: > Andres Freund wrote: > > On Monday 30 November 2009 03:57:11 Itagaki Takahiro wrote: > >> Boszormenyi Zoltan wrote: > >>> we tried to discuss on a lower level what should be needed > >>> for a partial replication based on streaming

Re: [HACKERS] draft RFC: concept for partial, wal-based replication

2009-11-30 Thread Stefan Kaltenbrunner
Andres Freund wrote: On Monday 30 November 2009 03:57:11 Itagaki Takahiro wrote: Boszormenyi Zoltan wrote: we tried to discuss on a lower level what should be needed for a partial replication based on streaming replication. We need to discuss a "partial recovery" before the partial replicatio

Re: [HACKERS] draft RFC: concept for partial, wal-based replication

2009-11-30 Thread Itagaki Takahiro
Andres Freund wrote: > > We need to discuss a "partial recovery" before the partial replication. > If you do the filtering on the sending side you dont actually need partial > recover in the sense that you filter in the rmgr or similar. > > Or do I miss something? Sorry, I didn't explain well

Re: [HACKERS] draft RFC: concept for partial, wal-based replication

2009-11-30 Thread Andres Freund
On Monday 30 November 2009 03:57:11 Itagaki Takahiro wrote: > Boszormenyi Zoltan wrote: > > we tried to discuss on a lower level what should be needed > > for a partial replication based on streaming replication. > We need to discuss a "partial recovery" before the partial replication. If you do t

Re: [HACKERS] draft RFC: concept for partial, wal-based replication

2009-11-29 Thread Craig Ringer
Boszormenyi Zoltan wrote: > c. splitting wal into different replication sets Just a side note: in addition to its use for partial replication, this might have potential for performance-prioritizing databases or tablespaces. Being able to separate WAL logging so that different DBs, tablespaces, e

Re: [HACKERS] draft RFC: concept for partial, wal-based replication

2009-11-29 Thread Fujii Masao
On Mon, Nov 30, 2009 at 4:56 AM, Boszormenyi Zoltan wrote: > I don't think its reasonable trying to discuss and implement this all > in one huge patch so I propose implementing at least 1) as a seperate > patch. I agree with this development plan. > Now there unforatunately are two different sch

Re: [HACKERS] draft RFC: concept for partial, wal-based replication

2009-11-29 Thread Itagaki Takahiro
Boszormenyi Zoltan wrote: > we tried to discuss on a lower level what should be needed > for a partial replication based on streaming replication. We need to discuss a "partial recovery" before the partial replication. There are some related items in out ToDo list and previous discussions: ht

Re: [HACKERS] draft RFC: concept for partial, wal-based replication

2009-11-29 Thread Boszormenyi Zoltan
Hi, we tried to discuss on a lower level what should be needed for a partial replication based on streaming replication. a. transferring base data after a slave got added a relation/index/... (and initial setup) b. communicating the the slaves which relations they currently should have available

[HACKERS] draft RFC: concept for partial, wal-based replication

2009-10-30 Thread Hans-Juergen Schoenig -- PostgreSQL
hello ... as my day has worked out quite nicely up to know i thought to f... it up and post a new concept which has been requested by a customer. the goal is to collect some feedback, ideas and so on (not to be mixed up with "flames"). we have funding for this and we are trying to sort out how