I think Will has a point here. if we make the PFER sync but performed by other thread we make it available for REPL and DIST and will not destroy the consistency...
On 11/22/2013 12:49 PM, William Burns wrote: > I wonder if we are over analyzing this. It seems the main issue is > that the replication is done asynchronously. Infinispan has many ways > to be make something asynchronous. In my opinion we just chose the > wrong way. Wouldn't it be easier to just change the PFER to instead > of passing along the FORCE_ASYNCHRONOUS flag we instead just have the > operation performed asynchronous using putIfAbsentAsync ? This way > the lock is held during the duration of the replication and should be > consistent with other operations. Also the user can regain control > back faster as it doesn't even have to process the local interceptor > chain. We could also change the putForExternalRead method declaration > to also return a NotifiableFuture<Void> or something so they know when > the operation is completed (if they want). > > - Will > > On Thu, Nov 21, 2013 at 9:54 AM, Dan Berindei <[email protected]> wrote: >> >> >> >> On Thu, Nov 21, 2013 at 12:35 PM, Galder Zamarreño <[email protected]> >> wrote: >>> >>> >>> On Nov 18, 2013, at 12:42 PM, Dan Berindei <[email protected]> wrote: >>> >>>> >>>> >>>> >>>> On Mon, Nov 18, 2013 at 9:43 AM, Galder Zamarreño <[email protected]> >>>> wrote: >>>> >>>> On Nov 14, 2013, at 1:20 PM, Pedro Ruivo <[email protected]> wrote: >>>> >>>>> Hi, >>>>> >>>>> Simple question: shouldn't PFER ensure some consistency? >>>>> >>>>> I know that PFER is asynchronous but (IMO) it can create >>>>> inconsistencies >>>>> in the data. the primary owner replicates the PFER follow by a PUT >>>>> (PFER >>>>> is sent async log the lock is released immediately) for the same key, >>>>> we >>>>> have no way to be sure if the PFER is delivered after or before in all >>>>> the backup owners. >>>>> >>>>> comments? >>>> >>>> Assuming that PFER and PUT happen in the same thread, we're normally >>>> relying on the JGroups sequence of events to send the first, wait no >>>> response, and then send the second put. That should guarantee order in >>>> which >>>> puts are received in the other nodes, but after that yeah, there's a risk >>>> that it could happen. PFER and PUT for a given key normally happen in the >>>> same thread in cache heavy use cases such as Hibernate 2LC, but there's no >>>> guarantee. >>>> >>>> I don't think that's correct. If the cache is synchronous, the PUT will >>>> be sent as an OOB message, and as such it can be delivered on the target >>>> before the previous PFER command. That's regardless of whether the PFER >>>> command was sent as a regular or as an OOB message. >>> >>> ^ Hmmmm, that's definitely risky. I think we should make PFER local only. >>> >>> The fact that PFER is asynchronous is nice to have. IOW, if you read a >>> value from a database and you want to store it in the cache for later read, >>> the fact that it's replicated asynchronously is just so that other nodes can >>> take advantage of the value being in the cache. Since it's asynchronous some >>> nodes could fail to apply, but that's fine since you can go to the database >>> and re-retrieve it from there. So, making PFER local only would be the >>> degenerate case, where all nodes fail to apply except the local node, which >>> is fine. This is better than having the reordering above. >>> >>> In a chat I had with Dan, he pointed out that having PFER local only would >>> be problematic for DIST mode w/ L1 enabled, since the local write would not >>> invalidate other nodes, but this is fine because PFER only really makes >>> sense for situations where the Infinispan is used as a cache. So, if the >>> data is in the DB, you might as well go there (1 network trip), as opposed >>> to askign the other nodes for data and the database in the worst case (2 >>> network trips). >>> >>> PFER is really designed for replication or invalidation use cases, which >>> are precisely the ones configured for Hibernate 2LC. >>> >>> Thoughts? >>> >> >> +1 to make PFER local-only in replicated caches, but I now think we should >> go all the way and disallow PFER completely in dist caches. >> >> I still think having L1 enabled would be a problem, because a regular put() >> won't invalidate the entry on all the nodes that did a PFER for that key >> (there are no requestors, and even if we assume that we do a remote get >> before the PFER we'd still have race conditions). >> >> With L1 disabled, we have the problem that you mentioned: we're trying to >> read the value from the proper owners, but we never write it to the proper >> owners, so the hit ratio will be pretty bad. Using the SKIP_REMOTE_LOOKUP >> flag on reads, we'll avoid the extra RPC in Infinispan, but that will make >> the hit ratio even worse. E.g. in a 4-nodes cluster with numOwners=2, the >> hit ratio will never go above 50%. >> >> I don't think anyone would use a cache knowing that its hit ratio can never >> get above 50%, so we should just save ourselves some effort and stop >> supporting PFER in DIST mode. >> >> Cheers >> Dan >> >> >> _______________________________________________ >> infinispan-dev mailing list >> [email protected] >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > _______________________________________________ > infinispan-dev mailing list > [email protected] > https://lists.jboss.org/mailman/listinfo/infinispan-dev > _______________________________________________ infinispan-dev mailing list [email protected] https://lists.jboss.org/mailman/listinfo/infinispan-dev
