On Wed, Nov 21, 2012 at 4:30 PM, Andres Freund <and...@2ndquadrant.com>wrote:
> On 2012-11-21 15:28:30 +0900, Michael Paquier wrote: > > On Tue, Nov 20, 2012 at 8:22 PM, Andres Freund <and...@2ndquadrant.com > >wrote: > > > > > On 2012-11-20 09:30:40 +0900, Michael Paquier wrote: > > > > Btw, here are some extra comments based on my progress, hope it will > be > > > > useful for other people playing around with your patches. > > > > 1) Necessary to install the contrib module test_decoding on server > side > > > or > > > > the test case will not work. > > > > 2) Obtention of the following logs on server: > > > > LOG: forced to assume catalog changes for xid 1370 because it was > > > running > > > > to early > > > > WARNING: ABORT 1370 > > > > Actually I saw that there are many warnings like this. > > > > > > Those aren't unexpected. Perhaps I should not make it a warning then... > > > > > A NOTICE would be more adapted, a WARNING means that something that may > > endanger the system has happened, but as far as I understand from your > > explanation this is not the case. > > I think it should go DEBUG2 or so once were a bit more confident about > the code. > > > > A short explanation: > > > > > > We can only decode tuples we see in the WAL when we already have a > > > timetravel catalog snapshot before that transaction started. To build > > > such a snapshot we need to collect information about committed which > > > changed the catalog. Unfortunately we can't diagnose whether a txn > > > changed the catalog without a snapshot so we just assume all committed > > > ones do - it just costs a bit of memory. Thats the background of the > > > "forced to assume catalog changes for ..." message. > > > > > OK, so this snapshot only needs to include the XIDs of transactions that > > have modified the catalogs. Do I get it right? This way you are able to > > fetch the correct relation definition for replication decoding. > > Yes. We only carry those between (recenXmin, newestCatalogModifyingTxn), > so its not all of them. Normal snapshots carry all in-progress > transactionids instead of the committed ones, but that would have been > far more in our case (only a minority of txn's touch the catalog) and it > has problems with subtransaction tracking. > Hum. I might have missed something but what is the variable tracking the newest XID that modified catalogs. I can see of course recentXmin in snapmgr.c but nothing related to what you describe. > > > Just thinking but... It looks to be a waste to store the transactions > XIDs > > of all the committed transactions, but on the other hand there is no way > to > > track the XIDs of transactions that modified a catalog in current core > > code. So yes this approach is better as refining the transaction XID > > tracking for snapshot reconstruction is something that could be improved > > later. Those are only thoughts though... > > We actually only track xids of catalog modifying transactions once we > hit the CONSISTENT state. Before the initial snapshot we can't detect > that. > How to you track them? I think I need to go deeper in the code before asking more... -- Michael Paquier http://michael.otacoo.com
