On Fri, Nov 27, 2015 at 1:45 PM, Radim Vansa <[email protected]> wrote: > On 11/27/2015 11:48 AM, Bela Ban wrote: >> You're talking about the case where P applies the PUT, and sends an ACK >> back to O, but the async updates to the Bs are received by only a subset >> (or none) of the Bs, and then P crashes. >> >> As I was referring about the non-transactional case, wouldn't this be >> fine? Or do we want the *non-transactional* case to be an atomic update >> of P and all Bs? IMO, the latter should be done as part of a TX, not for >> the non-transactional case. > > You're not talking about non-transactional mode but mongo mode :)
Indeed! There's a reason we don't recommend using DIST_ASYNC, which is quite similar. In fact, I wouldn't be opposed to changing DIST_ASYNC so that the O -> P communication is synchronous, and only the P -> B communication is asynchronous. > > Non-transactional mode still guarantees that the data will be reliably > stored, but it does not allow any consistency between two keys. > Transactional mode allows you to change all keys or none of them. > > The atomicity is rather discutable. All writes are atomic with respect > to writes, but reads just come and read something, and there's no way to > make sure that two transactional reads read the same value. > > Due to the two-armies problem, in case that an error is encountered, > it's possible that the cluster will end up in inconsistent state - in > non-tx mode this is the updated B and P not applying the update. In > transactional case, if the second phase (CommitCommand) gets executed on > a subset of nodes and the others don't reply, the rollback sent cannot > undo the already committed transactions. In that case, Infinispan is > obliged to throw an exception to the user (tx mode requires > useSynchronizations=false to do this) but it cannot prevent/resolve it. Right, the biggest source of inconsistencies right now is replication timeouts, because those are not retried. As Radim says, transactional mode is not necessarily safer when timeouts are involved. I've argued before that we should remove the replication timeout and rely on FD exclusively, because suspected nodes are handled much better. There are also problems when the originator crashes after sending the commit command to some owners, but before all of them got it [1]. With pessimistic locking, the same can happen if the originator crashes after only some of the owners received the prepare command. We've made some progress handling crashed originators when partition handling is enabled, but we aren't covering all the cases yet. Still, there's a difference between having an inconsistency after reporting an error to the application, and having an inconsistency after reporting that everything's A-OK. [1] https://issues.jboss.org/browse/ISPN-3421 > >> >> So I think we need to come up with a concise definition of what the >> transactional versus non-transaction semantics are. >> >> But even if we go with a design where O waits for ACKs from *all* Bs, we >> can still end up with inconsistencies; e.g. when not all Bs received the >> updates. O will fail the PUT, but the question is what do we do in such >> a case? Re-submit the PUT? > > Throw exception and provide API to asses the situation. The "API to assess the situation" part is kind of lacking at the moment, but yeah, that's the idea. I doubt we'll ever get to a point where we can say the cache will *never* become inconsistent, but the application should at least be notified when something goes wrong. Cheers Dan _______________________________________________ infinispan-dev mailing list [email protected] https://lists.jboss.org/mailman/listinfo/infinispan-dev
