Hi, > Having this as responsibility of plugin sounds interesting. It certainly > narrows the scope for which we need to solve the abort issue. For 2PC > that may be okay as we need to somehow interact with transaction manager > as Simon noted. I am not sure if this helps streaming use-case though as > there is not going to be any external transaction management involved there. > > In any case all this interlocking could potentially be made less > impact-full by only doing it when we know the transaction did catalog > changes prior to currently decoded change (which we do during decoding) > since that's the only time we are interested in if it aborted or not. > > This all leads me to another idea. What if logical decoding provided API > for "locking/unlocking" the currently decoded transaction against abort. > This function would then be called by both decoding and output plugin > before any catalog read. The function can be smart enough to be NOOP if > the transaction is not running (ie we are not doing 2PC decoding or > streaming) or when the transaction didn't do any catalog modifications > (we already have that info easily accessible as bool). > > That would mean we'd never do any kind of heavy locking for prolonged > periods of time (ie network calls) but only during catalog access and > only when needed. It would also solve this for both 2PC and streaming > and it would be easy to use by plugin authors. Just document that some > call should be done before catalog access when in output plugin, can > even be Asserted that the call was done probably. > > Thoughts? >
Yeah, this might work. We already have SET_LOCKTAG_TRANSACTION() via which every transaction takes an exclusive lock on its own transactionid when it starts, for example. We ideally want a single solution to handle 2PC and ongoing (streaming) transactions. We could introduce a new SET_LOCKTAG_LOGICALTRANSACTION(). The logical decoding process could take a SHARED lock on this, check if the XID is still ok to decode, read the catalog and unlock. Abort/Commit transaction processing could take this in EXCLUSIVE mode. As mentioned above, the plugin API which takes this lock will be smart enough to be a NOOP if the transaction is not running (i.e we are not doing 2PC decoding or streaming) or when the transaction didn't do any catalog modifications. Thoughts? Regards, Nikhils -- Nikhil Sontakke http://www.2ndQuadrant.com/ PostgreSQL/Postgres-XL Development, 24x7 Support, Training & Services
