Re: [HACKERS] [PATCH 08/16] Introduce the ApplyCache module which can reassemble transactions from a stream of interspersed changes

2012-07-03 Thread Robert Haas
On Thu, Jun 28, 2012 at 12:04 PM, Andres Freund and...@2ndquadrant.com wrote: On Thursday, June 28, 2012 06:01:10 PM Robert Haas wrote: On Tue, Jun 26, 2012 at 8:13 PM, Andres Freund and...@2ndquadrant.com wrote: It even can be significantly higher than max_connections because

Re: [HACKERS] [PATCH 08/16] Introduce the ApplyCache module which can reassemble transactions from a stream of interspersed changes

2012-06-28 Thread Robert Haas
On Tue, Jun 26, 2012 at 8:13 PM, Andres Freund and...@2ndquadrant.com wrote: It even can be significantly higher than max_connections because subtransactions are only recognizable as part of their parent transaction uppon commit. I've been wondering whether sub-XID assignment was going to end

Re: [HACKERS] [PATCH 08/16] Introduce the ApplyCache module which can reassemble transactions from a stream of interspersed changes

2012-06-28 Thread Andres Freund
On Thursday, June 28, 2012 06:01:10 PM Robert Haas wrote: On Tue, Jun 26, 2012 at 8:13 PM, Andres Freund and...@2ndquadrant.com wrote: It even can be significantly higher than max_connections because subtransactions are only recognizable as part of their parent transaction uppon commit.

Re: [HACKERS] [PATCH 08/16] Introduce the ApplyCache module which can reassemble transactions from a stream of interspersed changes

2012-06-26 Thread Andres Freund
Hi Steve, On Tuesday, June 26, 2012 02:14:22 AM Steve Singer wrote: I planned to have some cutoff 'max_changes_in_memory_per_txn' value. If it has been reached for one transaction all existing changes are spilled to disk. New changes again can be kept in memory till its reached again. Do

Re: [HACKERS] [PATCH 08/16] Introduce the ApplyCache module which can reassemble transactions from a stream of interspersed changes

2012-06-25 Thread Steve Singer
On 12-06-21 04:37 AM, Andres Freund wrote: Hi Steve, Thanks! Attached is a detailed review of the patch. Very good analysis, thanks! Another reasons why we cannot easily do 1) is that subtransactions aren't discernible from top-level transactions before the top-level commit happens, we can

Re: [HACKERS] [PATCH 08/16] Introduce the ApplyCache module which can reassemble transactions from a stream of interspersed changes

2012-06-21 Thread Andres Freund
Hi Steve, On Thursday, June 21, 2012 02:16:57 AM Steve Singer wrote: On 12-06-13 07:28 AM, Andres Freund wrote: From: Andres Freundand...@anarazel.de The individual changes need to be identified by an xid. The xid can be a subtransaction or a toplevel one, at commit those can be

Re: [HACKERS] [PATCH 08/16] Introduce the ApplyCache module which can reassemble transactions from a stream of interspersed changes

2012-06-20 Thread Steve Singer
On 12-06-13 07:28 AM, Andres Freund wrote: From: Andres Freundand...@anarazel.de The individual changes need to be identified by an xid. The xid can be a subtransaction or a toplevel one, at commit those can be reintegrated by doing a k-way mergesort between the individual transaction.

[HACKERS] [PATCH 08/16] Introduce the ApplyCache module which can reassemble transactions from a stream of interspersed changes

2012-06-13 Thread Andres Freund
From: Andres Freund and...@anarazel.de The individual changes need to be identified by an xid. The xid can be a subtransaction or a toplevel one, at commit those can be reintegrated by doing a k-way mergesort between the individual transaction. Callbacks for apply_begin, apply_change and