That doesn't sound right, we don't keep any lock for the duration of the replication. In non-tx mode, we have to do a RPC to the primary owner before we acquire any key. So there's nothing stopping the PFER from writing its value after a regular (sync) put when the put was initiated after the PFER.
On Fri, Nov 22, 2013 at 2:49 PM, William Burns <[email protected]> wrote: > I wonder if we are over analyzing this. It seems the main issue is > that the replication is done asynchronously. Infinispan has many ways > to be make something asynchronous. In my opinion we just chose the > wrong way. Wouldn't it be easier to just change the PFER to instead > of passing along the FORCE_ASYNCHRONOUS flag we instead just have the > operation performed asynchronous using putIfAbsentAsync ? This way > the lock is held during the duration of the replication and should be > consistent with other operations. Also the user can regain control > back faster as it doesn't even have to process the local interceptor > chain. We could also change the putForExternalRead method declaration > to also return a NotifiableFuture<Void> or something so they know when > the operation is completed (if they want). > > - Will > > On Thu, Nov 21, 2013 at 9:54 AM, Dan Berindei <[email protected]> > wrote: > > > > > > > > On Thu, Nov 21, 2013 at 12:35 PM, Galder Zamarreño <[email protected]> > > wrote: > >> > >> > >> On Nov 18, 2013, at 12:42 PM, Dan Berindei <[email protected]> > wrote: > >> > >> > > >> > > >> > > >> > On Mon, Nov 18, 2013 at 9:43 AM, Galder Zamarreño <[email protected]> > >> > wrote: > >> > > >> > On Nov 14, 2013, at 1:20 PM, Pedro Ruivo <[email protected]> > wrote: > >> > > >> > > Hi, > >> > > > >> > > Simple question: shouldn't PFER ensure some consistency? > >> > > > >> > > I know that PFER is asynchronous but (IMO) it can create > >> > > inconsistencies > >> > > in the data. the primary owner replicates the PFER follow by a PUT > >> > > (PFER > >> > > is sent async log the lock is released immediately) for the same > key, > >> > > we > >> > > have no way to be sure if the PFER is delivered after or before in > all > >> > > the backup owners. > >> > > > >> > > comments? > >> > > >> > Assuming that PFER and PUT happen in the same thread, we're normally > >> > relying on the JGroups sequence of events to send the first, wait no > >> > response, and then send the second put. That should guarantee order > in which > >> > puts are received in the other nodes, but after that yeah, there's a > risk > >> > that it could happen. PFER and PUT for a given key normally happen in > the > >> > same thread in cache heavy use cases such as Hibernate 2LC, but > there's no > >> > guarantee. > >> > > >> > I don't think that's correct. If the cache is synchronous, the PUT > will > >> > be sent as an OOB message, and as such it can be delivered on the > target > >> > before the previous PFER command. That's regardless of whether the > PFER > >> > command was sent as a regular or as an OOB message. > >> > >> ^ Hmmmm, that's definitely risky. I think we should make PFER local > only. > >> > >> The fact that PFER is asynchronous is nice to have. IOW, if you read a > >> value from a database and you want to store it in the cache for later > read, > >> the fact that it's replicated asynchronously is just so that other > nodes can > >> take advantage of the value being in the cache. Since it's asynchronous > some > >> nodes could fail to apply, but that's fine since you can go to the > database > >> and re-retrieve it from there. So, making PFER local only would be the > >> degenerate case, where all nodes fail to apply except the local node, > which > >> is fine. This is better than having the reordering above. > >> > >> In a chat I had with Dan, he pointed out that having PFER local only > would > >> be problematic for DIST mode w/ L1 enabled, since the local write would > not > >> invalidate other nodes, but this is fine because PFER only really makes > >> sense for situations where the Infinispan is used as a cache. So, if the > >> data is in the DB, you might as well go there (1 network trip), as > opposed > >> to askign the other nodes for data and the database in the worst case (2 > >> network trips). > >> > >> PFER is really designed for replication or invalidation use cases, which > >> are precisely the ones configured for Hibernate 2LC. > >> > >> Thoughts? > >> > > > > +1 to make PFER local-only in replicated caches, but I now think we > should > > go all the way and disallow PFER completely in dist caches. > > > > I still think having L1 enabled would be a problem, because a regular > put() > > won't invalidate the entry on all the nodes that did a PFER for that key > > (there are no requestors, and even if we assume that we do a remote get > > before the PFER we'd still have race conditions). > > > > With L1 disabled, we have the problem that you mentioned: we're trying to > > read the value from the proper owners, but we never write it to the > proper > > owners, so the hit ratio will be pretty bad. Using the SKIP_REMOTE_LOOKUP > > flag on reads, we'll avoid the extra RPC in Infinispan, but that will > make > > the hit ratio even worse. E.g. in a 4-nodes cluster with numOwners=2, the > > hit ratio will never go above 50%. > > > > I don't think anyone would use a cache knowing that its hit ratio can > never > > get above 50%, so we should just save ourselves some effort and stop > > supporting PFER in DIST mode. > > > > Cheers > > Dan > > > > > > _______________________________________________ > > infinispan-dev mailing list > > [email protected] > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > _______________________________________________ > infinispan-dev mailing list > [email protected] > https://lists.jboss.org/mailman/listinfo/infinispan-dev >
_______________________________________________ infinispan-dev mailing list [email protected] https://lists.jboss.org/mailman/listinfo/infinispan-dev
