[infinispan-dev] Testing ISPN 200
Guys, I was thinking about test ISPN-200 in a real environment. I want to compare the distributed queries with local queries in different scenarios. Do you have any thought about this? Any suggestions? My first idea is to create an application that puts and search "n" values (of size "x") in a cache with "k" nodes... What do you think about this? thanks! Israel ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] What should be the default transaction mode: transactional or non-transactional?
On 28 September 2011 19:08, Pete Muir wrote: > I would say non-TX by default. +1, as it's the current default. I see many advantages in using TX instead, but if we change that I'd expect that to happen in a mayor release. Sanne ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] What should be the default transaction mode: transactional or non-transactional?
I would say non-TX by default. On 28 Sep 2011, at 17:10, Mircea Markus wrote: > Hi, > > ATM the default cache transaction model in 5.1 is transactional. The main > reason transactional cache was chosen by default was backward compatibility: > existing code that access the cache in a mixed way (i.e. both transactional > and non transactional) would fail if the default mode is non transactional. > By fail I don't mean a runtime exception being thrown, but the the operations > in transaction's scope being executed individually and the transaction > ignored. > > As Galder highlighted, the problem with being transactional by default is > that you pay a penalti if you take Infinispan and run out of the box as a > simple local cache for example. In theory, in only makes sense to enable > transactional behaivour if you want transactions. [1] > > Shall we switch to non-tx caches by default? Or implement ISPN-61 in 5.1, > measure performance and if perf is ok stick with transactional cache? [2] > Opinions? > > Cheers, > Mircea > > > [1] > a). The performance for a get should be the same for a tx and non tx cache. > b) Put's performance is better now: 1RPC for non-tx and 2 RPC for tx. > However this will change once https://issues.jboss.org/browse/ISPN-61. With > ISPN-61 implemented, I expect the performance of both tx and non-tx caches to > be roughly the same. > > [2] Implementing first optimisation described in ISPN-61 is rather trivial > right now. > ___ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] ISPN-1384 - InboundInvocationHandlerImpl should wait for cache to be started? (not just defined)
On Wed, Sep 28, 2011 at 6:21 PM, Galder Zamarreño wrote: > > On Sep 28, 2011, at 4:06 PM, Galder Zamarreño wrote: > >> >> On Sep 27, 2011, at 6:50 PM, Dan Berindei wrote: >> >>> On Tue, Sep 27, 2011 at 5:59 PM, Galder Zamarreño wrote: Hi, Re: https://issues.jboss.org/browse/ISPN-1384 I've had a look to this and this race condition could, in theory, be resolved by making InboundInvocationHandlerImpl.handle() waiting for cache not only to be defined, but to be started. Otherwise there'll always be a potential race condition like the one showed in the log. >>> >>> Actually I think it would be enough to wait until the cache has >>> started joining (StateTransferManager.waitForJoinToStart()), that >>> means all the other components have finished starting. >> >> Sounds good to me. > > Couple of things: > > 1. In this particular case, waiting for that would work because this > clustered get is a cluster wide one, so there's no need for this node to wait > for state transfer to occur. InboundInvocationHandlerImpl.handle could be > hacked to check on the cache's StateTransferManager.waitForJoinToStart() but > seems a bit hacky. > It's already calling StateTransferManager.waitForJoinToComplete(), I don't think calling another method would make it hackier... > 2. In BaseStateTransferManagerImpl.start it says: > > " // needs to be AFTER the DistributionManager and *after* the cache loader > manager (if any) inits and preloads" > > CacheLoaderManagerImpl.preload has priority 56, compared to > BaseStateTransferManagerImpl.start which has 21, so state transfer could > start before preloading has occurred? > I missed this, I got the comment from the old StateTransferManager but I didn't check the actual priorities. I'm not sure if the comment is valid though, since the old StateTransferManager had priority 55 and it also cleared the data container before applying the state from the coordinator. I'm not sure how preloading and state transfer are supposed to interact, maybe Manik can help clear this up? > >> >>> >>> With the asymmetric cache support in place I think we shouldn't have >>> to wait at all, since we'll send the join request only after all the >>> components have been started. We could either apply the commands (if >>> we implement the non-blocking state transfer option) or reject them >>> and tell the originator to retry after the state transfer is done (if >>> we keep the blocking state transfer, since the other nodes shouldn't >>> be sending commands to us anyway). >>> In this particular case, this is clustered get command being received from a clustered cache loader, which is arriving in the cache before this is started (and the cache loader has been created, hence the NPE). Another question, is there any reason why CacheLoader is not a named cache component which can be initalised with a corresponding factory and to which other components can be injected (i.e. marshaller, cache...etc)? In this particular case, this would also resolve the issue because ClusterCacheLoader.start() does nothing, so all the interceptor needs is a proper instance of ClusterCacheLoader available. The factory makes these available bejore inject. Thoughts? Cheers, p.s. Dan, I am aware of https://issues.jboss.org/browse/ISPN-1324, maybe you're solving this indirectly with the work for that JIRA? >>> >>> With asymmetric caches in place I don't think we'll need/want >>> ISPN-1324 any more. >> >> +1 >> >>> >>> Cheers >>> Dan >>> >>> -- Galder Zamarreño Sr. Software Engineer Infinispan, JBoss Cache ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >>> ___ >>> infinispan-dev mailing list >>> infinispan-dev@lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> -- >> Galder Zamarreño >> Sr. Software Engineer >> Infinispan, JBoss Cache >> >> >> ___ >> infinispan-dev mailing list >> infinispan-dev@lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > -- > Galder Zamarreño > Sr. Software Engineer > Infinispan, JBoss Cache > > > ___ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] What should be the default transaction mode: transactional or non-transactional?
2 use cases: - existing users already have this in place - non exiting users: defaults to DummyTM. On 28 Sep 2011, at 17:44, Manik Surtani wrote: > Don't transactional caches require a JTA cache manager being present, > configured and running? > > On 28 Sep 2011, at 17:10, Mircea Markus wrote: > >> Hi, >> >> ATM the default cache transaction model in 5.1 is transactional. The main >> reason transactional cache was chosen by default was backward compatibility: >> existing code that access the cache in a mixed way (i.e. both transactional >> and non transactional) would fail if the default mode is non transactional. >> By fail I don't mean a runtime exception being thrown, but the the >> operations in transaction's scope being executed individually and the >> transaction ignored. >> >> As Galder highlighted, the problem with being transactional by default is >> that you pay a penalti if you take Infinispan and run out of the box as a >> simple local cache for example. In theory, in only makes sense to enable >> transactional behaivour if you want transactions. [1] >> >> Shall we switch to non-tx caches by default? Or implement ISPN-61 in 5.1, >> measure performance and if perf is ok stick with transactional cache? [2] >> Opinions? >> >> Cheers, >> Mircea >> >> >> [1] >> a). The performance for a get should be the same for a tx and non tx cache. >> b) Put's performance is better now: 1RPC for non-tx and 2 RPC for tx. >> However this will change once https://issues.jboss.org/browse/ISPN-61. With >> ISPN-61 implemented, I expect the performance of both tx and non-tx caches >> to be roughly the same. >> >> [2] Implementing first optimisation described in ISPN-61 is rather trivial >> right now. >> ___ >> infinispan-dev mailing list >> infinispan-dev@lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > -- > Manik Surtani > ma...@jboss.org > twitter.com/maniksurtani > > Lead, Infinispan > http://www.infinispan.org > > > > ___ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] What should be the default transaction mode: transactional or non-transactional?
Don't transactional caches require a JTA cache manager being present, configured and running? On 28 Sep 2011, at 17:10, Mircea Markus wrote: > Hi, > > ATM the default cache transaction model in 5.1 is transactional. The main > reason transactional cache was chosen by default was backward compatibility: > existing code that access the cache in a mixed way (i.e. both transactional > and non transactional) would fail if the default mode is non transactional. > By fail I don't mean a runtime exception being thrown, but the the operations > in transaction's scope being executed individually and the transaction > ignored. > > As Galder highlighted, the problem with being transactional by default is > that you pay a penalti if you take Infinispan and run out of the box as a > simple local cache for example. In theory, in only makes sense to enable > transactional behaivour if you want transactions. [1] > > Shall we switch to non-tx caches by default? Or implement ISPN-61 in 5.1, > measure performance and if perf is ok stick with transactional cache? [2] > Opinions? > > Cheers, > Mircea > > > [1] > a). The performance for a get should be the same for a tx and non tx cache. > b) Put's performance is better now: 1RPC for non-tx and 2 RPC for tx. > However this will change once https://issues.jboss.org/browse/ISPN-61. With > ISPN-61 implemented, I expect the performance of both tx and non-tx caches to > be roughly the same. > > [2] Implementing first optimisation described in ISPN-61 is rather trivial > right now. > ___ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Lead, Infinispan http://www.infinispan.org ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Asymmetric caches and manual rehashing design
On 28 Sep 2011, at 12:48, Dan Berindei wrote: >> >> - Do we need the complex PREPARE_VIEW / ROLLBACK_VIEW / COMMIT_VIEW 2PC >> handling ? This adds a lot of complexity. Is it only used when we have a >> transactional cache ? >> > > Nope, this doesn't have anything to do with transactional caches, > instead it is all about computing the owner that will push the key > during the rebalance operation. Yes this is what I understood as well. 2-phase view installation to ensure consistency. Nothing to do with operational transactions. -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Lead, Infinispan http://www.infinispan.org ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] trace vs log.isTraceEnabled
On 09/28/2011 11:35 AM, Mircea Markus wrote:
> Hi,
>
> I'm not aware of any convention on using trace vs log.isTraceEnabled() to
> guard the trace statements.
>
> if (trace) log.trace("some request related stuff")
> vs
> if (log.isTraceEnabled()) log.trace("some request related stuff");
>
> The former is more efficient, but cannot be managed at runtime. It seems to
> be the preferred form, so shall we stick with it?
If you're using the jboss-logging API, and your log statement does not
do any interpolation, then it is just as fast to do any of the following
(with no if):
log.trace("blah");
log.tracef("the %s happened to %s", foo, bar);
log.tracev("the {0} happened to {1}", foo, bar);
In the case where trace logging is disabled, these are exactly as
efficient as the if (log.isTraceEnabled()) variants. In the case where
it is enabled, it is marginally more efficient (though the trace log
itself is substantially more expensive of course).
Overall I'd avoid the "if" forms unless you're doing complex interpolation:
log.trace("Foo " + bar + " baz " + zap);
log.tracef("the %s happened to %s", fooMethod().barMethod(), bar);
...both of which incur the expense of the expression resolution even if
the log message is ultimately discarded.
--
- DML
___
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
[infinispan-dev] trace vs log.isTraceEnabled
Hi,
I'm not aware of any convention on using trace vs log.isTraceEnabled() to guard
the trace statements.
if (trace) log.trace("some request related stuff")
vs
if (log.isTraceEnabled()) log.trace("some request related stuff");
The former is more efficient, but cannot be managed at runtime. It seems to be
the preferred form, so shall we stick with it?
Cheers,
Mircea
___
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
[infinispan-dev] What should be the default transaction mode: transactional or non-transactional?
Hi, ATM the default cache transaction model in 5.1 is transactional. The main reason transactional cache was chosen by default was backward compatibility: existing code that access the cache in a mixed way (i.e. both transactional and non transactional) would fail if the default mode is non transactional. By fail I don't mean a runtime exception being thrown, but the the operations in transaction's scope being executed individually and the transaction ignored. As Galder highlighted, the problem with being transactional by default is that you pay a penalti if you take Infinispan and run out of the box as a simple local cache for example. In theory, in only makes sense to enable transactional behaivour if you want transactions. [1] Shall we switch to non-tx caches by default? Or implement ISPN-61 in 5.1, measure performance and if perf is ok stick with transactional cache? [2] Opinions? Cheers, Mircea [1] a). The performance for a get should be the same for a tx and non tx cache. b) Put's performance is better now: 1RPC for non-tx and 2 RPC for tx. However this will change once https://issues.jboss.org/browse/ISPN-61. With ISPN-61 implemented, I expect the performance of both tx and non-tx caches to be roughly the same. [2] Implementing first optimisation described in ISPN-61 is rather trivial right now.___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] ISPN-1384 - InboundInvocationHandlerImpl should wait for cache to be started? (not just defined)
On Sep 28, 2011, at 4:06 PM, Galder Zamarreño wrote: > > On Sep 27, 2011, at 6:50 PM, Dan Berindei wrote: > >> On Tue, Sep 27, 2011 at 5:59 PM, Galder Zamarreño wrote: >>> Hi, >>> >>> Re: https://issues.jboss.org/browse/ISPN-1384 >>> >>> I've had a look to this and this race condition could, in theory, be >>> resolved by making InboundInvocationHandlerImpl.handle() waiting for cache >>> not only to be defined, but to be started. Otherwise there'll always be a >>> potential race condition like the one showed in the log. >>> >> >> Actually I think it would be enough to wait until the cache has >> started joining (StateTransferManager.waitForJoinToStart()), that >> means all the other components have finished starting. > > Sounds good to me. Couple of things: 1. In this particular case, waiting for that would work because this clustered get is a cluster wide one, so there's no need for this node to wait for state transfer to occur. InboundInvocationHandlerImpl.handle could be hacked to check on the cache's StateTransferManager.waitForJoinToStart() but seems a bit hacky. 2. In BaseStateTransferManagerImpl.start it says: " // needs to be AFTER the DistributionManager and *after* the cache loader manager (if any) inits and preloads" CacheLoaderManagerImpl.preload has priority 56, compared to BaseStateTransferManagerImpl.start which has 21, so state transfer could start before preloading has occurred? > >> >> With the asymmetric cache support in place I think we shouldn't have >> to wait at all, since we'll send the join request only after all the >> components have been started. We could either apply the commands (if >> we implement the non-blocking state transfer option) or reject them >> and tell the originator to retry after the state transfer is done (if >> we keep the blocking state transfer, since the other nodes shouldn't >> be sending commands to us anyway). >> >>> In this particular case, this is clustered get command being received from >>> a clustered cache loader, which is arriving in the cache before this is >>> started (and the cache loader has been created, hence the NPE). >>> >>> Another question, is there any reason why CacheLoader is not a named cache >>> component which can be initalised with a corresponding factory and to which >>> other components can be injected (i.e. marshaller, cache...etc)? >>> >>> In this particular case, this would also resolve the issue because >>> ClusterCacheLoader.start() does nothing, so all the interceptor needs is a >>> proper instance of ClusterCacheLoader available. The factory makes these >>> available bejore inject. >>> >>> Thoughts? >>> >>> Cheers, >>> >>> p.s. Dan, I am aware of https://issues.jboss.org/browse/ISPN-1324, maybe >>> you're solving this indirectly with the work for that JIRA? >> >> With asymmetric caches in place I don't think we'll need/want >> ISPN-1324 any more. > > +1 > >> >> Cheers >> Dan >> >> >>> -- >>> Galder Zamarreño >>> Sr. Software Engineer >>> Infinispan, JBoss Cache >>> >>> >>> ___ >>> infinispan-dev mailing list >>> infinispan-dev@lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >> >> ___ >> infinispan-dev mailing list >> infinispan-dev@lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > -- > Galder Zamarreño > Sr. Software Engineer > Infinispan, JBoss Cache > > > ___ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Galder Zamarreño Sr. Software Engineer Infinispan, JBoss Cache ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Asymmetric caches and manual rehashing design
On Sep 28, 2011 5:24 PM, "Mircea Markus" wrote: > > Nice stuff. > > Some q: > > - When a node CM that has c1 and c2 caches running on it goes down, you'll end up having REQUEST_LEAVE(c1) and REQUEST_LEAVE(c2) at the same time. Is this handled in serial or in parallel? > In parallel, although that shouldn't matter because the rebalance is triggerred asynchronously. > - The view updates can be disabled dynamically (DISABLE_VIEW_UPDATES). Shouldn't we also be able to re-enabled them manually after that? > > The view accept handler updates the pending view acquires the view change lock in exclusive mode, then releases it immediately > Shouldn't this be: > - acquire exclusive lock > - updates the pending view > - release lock > ? Right, what I meant to say is that it doesn't hold the lock for the entire duration of the rebalance task. > > On 27 Sep 2011, at 17:22, Dan Berindei wrote: > >> Following the discussions last week I've written up a wiki page >> describing the strategies for cache view management and state transfer >> that will enable asymmetric caches and manual rehashing: >> http://community.jboss.org/wiki/AsymmetricCachesAndManualRehashingDesign >> >> The state transfer part is not very detailed, as you'll see we want to >> go with a non-blocking approach but I'm not sure we can finish that >> for 5.1 so we're keeping a blocking fallback option. >> >> Your comments are welcome, either on the list or on the wiki page. >> >> Dan >> ___ >> infinispan-dev mailing list >> infinispan-dev@lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > ___ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Asymmetric caches and manual rehashing design
Nice stuff. Some q: - When a node CM that has c1 and c2 caches running on it goes down, you'll end up having REQUEST_LEAVE(c1) and REQUEST_LEAVE(c2) at the same time. Is this handled in serial or in parallel? - The view updates can be disabled dynamically (DISABLE_VIEW_UPDATES). Shouldn't we also be able to re-enabled them manually after that? The view accept handler updates the pending view acquires the view change lock in exclusive mode, then releases it immediately Shouldn't this be: - acquire exclusive lock - updates the pending view - release lock ? On 27 Sep 2011, at 17:22, Dan Berindei wrote: > Following the discussions last week I've written up a wiki page > describing the strategies for cache view management and state transfer > that will enable asymmetric caches and manual rehashing: > http://community.jboss.org/wiki/AsymmetricCachesAndManualRehashingDesign > > The state transfer part is not very detailed, as you'll see we want to > go with a non-blocking approach but I'm not sure we can finish that > for 5.1 so we're keeping a blocking fallback option. > > Your comments are welcome, either on the list or on the wiki page. > > Dan > ___ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] optimistic locking - the time has come
On 28 Sep 2011, at 15:02, Galder Zamarreño wrote: > Thanks Mircea :) > > To all: I've had a look and made some comments, but this patch is definitely > worth another look by at least another person. +1 Manik and Sanne are good candidates. ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] ISPN-1384 - InboundInvocationHandlerImpl should wait for cache to be started? (not just defined)
On Sep 27, 2011, at 6:50 PM, Dan Berindei wrote: > On Tue, Sep 27, 2011 at 5:59 PM, Galder Zamarreño wrote: >> Hi, >> >> Re: https://issues.jboss.org/browse/ISPN-1384 >> >> I've had a look to this and this race condition could, in theory, be >> resolved by making InboundInvocationHandlerImpl.handle() waiting for cache >> not only to be defined, but to be started. Otherwise there'll always be a >> potential race condition like the one showed in the log. >> > > Actually I think it would be enough to wait until the cache has > started joining (StateTransferManager.waitForJoinToStart()), that > means all the other components have finished starting. Sounds good to me. > > With the asymmetric cache support in place I think we shouldn't have > to wait at all, since we'll send the join request only after all the > components have been started. We could either apply the commands (if > we implement the non-blocking state transfer option) or reject them > and tell the originator to retry after the state transfer is done (if > we keep the blocking state transfer, since the other nodes shouldn't > be sending commands to us anyway). > >> In this particular case, this is clustered get command being received from a >> clustered cache loader, which is arriving in the cache before this is >> started (and the cache loader has been created, hence the NPE). >> >> Another question, is there any reason why CacheLoader is not a named cache >> component which can be initalised with a corresponding factory and to which >> other components can be injected (i.e. marshaller, cache...etc)? >> >> In this particular case, this would also resolve the issue because >> ClusterCacheLoader.start() does nothing, so all the interceptor needs is a >> proper instance of ClusterCacheLoader available. The factory makes these >> available bejore inject. >> >> Thoughts? >> >> Cheers, >> >> p.s. Dan, I am aware of https://issues.jboss.org/browse/ISPN-1324, maybe >> you're solving this indirectly with the work for that JIRA? > > With asymmetric caches in place I don't think we'll need/want > ISPN-1324 any more. +1 > > Cheers > Dan > > >> -- >> Galder Zamarreño >> Sr. Software Engineer >> Infinispan, JBoss Cache >> >> >> ___ >> infinispan-dev mailing list >> infinispan-dev@lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > > ___ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Galder Zamarreño Sr. Software Engineer Infinispan, JBoss Cache ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] optimistic locking - the time has come
Thanks Mircea :) To all: I've had a look and made some comments, but this patch is definitely worth another look by at least another person. On Sep 28, 2011, at 1:51 PM, Mircea Markus wrote: > > On 28 Sep 2011, at 10:05, Galder Zamarreño wrote: > >> Thanks Mircea. I've made some comments. > > Thanks, I've replied and updated the pull request. > Also added a summary: https://github.com/infinispan/infinispan/pull/551 > >> >> To all: >> >> If you're gonna submit a big pull req, please use the description to >> *summarise the changes* in order to help reviewers. This helps focus the >> attention of the reviewer to the pieces of code that are most relevant. > > +1 > > ___ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Galder Zamarreño Sr. Software Engineer Infinispan, JBoss Cache ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Asymmetric caches and manual rehashing design
Hi Bela After writing my reply I realized there is a small problem with my argument: > > In order to do it deterministically we need to have a common "last > good consistent hash" for the last rebalance that finished > successfully, and each node must determine if it should push a key or > not based on that last good CH. > If there was a merge, the nodes in each partition have different ideas of what their old view was, so it's not possible to have a common "last good CH" for the entire cluster unless we pick the coordinator's view as the "winner" and discard everything in the loser partition. We will have instead a last good CH for each partition, and each node will determine whether to push a key or not based on the CH of its partition and the CH of the united cluster. Cheers Dan ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] optimistic locking - the time has come
On 28 Sep 2011, at 10:05, Galder Zamarreño wrote: > Thanks Mircea. I've made some comments. Thanks, I've replied and updated the pull request. Also added a summary: https://github.com/infinispan/infinispan/pull/551 > > To all: > > If you're gonna submit a big pull req, please use the description to > *summarise the changes* in order to help reviewers. This helps focus the > attention of the reviewer to the pieces of code that are most relevant. +1 ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Asymmetric caches and manual rehashing design
On Wed, Sep 28, 2011 at 12:59 PM, Bela Ban wrote: > My 5 cents: > > Virtual Cache Views: > > > > - The REQUEST_VIEW_UPDATE is a good idea to manually trigger flushing of > pending views. However, I'm not sure what the goal of > DISABLE_VIEW_UPDATES is. Since I assume this is for the entire cluster, > can 1 member disable view delivery for the entire cluster ? What > scenario is this for ? > Indeed, one member could disable view delivery for the entire cluster. The main scenario I had in mind for this was stopping the entire cluster. A good rebalancing policy might handle such cases automatically, so I'm not 100% sure we need this. > - Are you clubbing (virtual) view updates and rebalancing together ? And > if so (I should probably read on first...), can't you have view > installations *without* rebalancing ? > I'm not sure what benefits you would get from joining the cache view but not receiving state compared to not sending the join request at all. Non-members are allowed to send/receive commands, so in the future we could even have separate "server" nodes (that join the cache view) and "client" nodes (joining the JGroups cluster to send commands, but not the cache view and so not holding any data except L1). My idea was that the cache view was a representation of the caches that are able to service requests, so it doesn't make sense to include in the view caches that don't hold data. As a practical note, if we add nodes to the consistent hash as they come along, a key will have one set of owners if written to before the join and another set of owner if written to after the join. How should another node determine the current location of the key, especially considering that there can be many view changes before rebalancing is triggerred? > - Do we need the complex PREPARE_VIEW / ROLLBACK_VIEW / COMMIT_VIEW 2PC > handling ? This adds a lot of complexity. Is it only used when we have a > transactional cache ? > Nope, this doesn't have anything to do with transactional caches, instead it is all about computing the owner that will push the key during the rebalance operation. In order to do it deterministically we need to have a common "last good consistent hash" for the last rebalance that finished successfully, and each node must determine if it should push a key or not based on that last good CH. A rebalance operation can also fail for various reasons (e.g. the coordinator died). If that happens the new owners won't have all the state, so they should not receive requests for the state that they would have had in the pending CH. > - State is to be transferred *within* this 2PC time frame. Hmm, again, > this ties rebalancing and view installation together (see my argument > above)... > If view installation wasn't tied to state transfer then we'd have to keep yet the last rebalanced view somewhere else. We would hold the "last pending view" (pending rebalance, that is) in the CacheViewsManager and the "last rebalanced view" in another component, and that component would have it's own mechanism for synchronizing the "last rebalanced view" among cache members. So I think the 2PC approach in CacheViewsManager actually simplifies things. > > On 9/27/11 6:22 PM, Dan Berindei wrote: >> Following the discussions last week I've written up a wiki page >> describing the strategies for cache view management and state transfer >> that will enable asymmetric caches and manual rehashing: >> http://community.jboss.org/wiki/AsymmetricCachesAndManualRehashingDesign >> >> The state transfer part is not very detailed, as you'll see we want to >> go with a non-blocking approach but I'm not sure we can finish that >> for 5.1 so we're keeping a blocking fallback option. >> >> Your comments are welcome, either on the list or on the wiki page. > > > -- > Bela Ban > Lead JGroups (http://www.jgroups.org) > JBoss / Red Hat > ___ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > Cheers Dan ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] optimistic locking - the time has come
On 28 Sep 2011, at 11:31, Sanne Grinovero wrote: > Agreed, even better if we could use the commit description so it stays > with the code, and you can describe a step at a time. > > We usually use a single-liner description, but for the more complex > patches we should add a paragraph. I did it occasionally, not sure > anybody noticed :) I did notice - and I think that approach works really well. -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Lead, Infinispan http://www.infinispan.org ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] optimistic locking - the time has come
Agreed, even better if we could use the commit description so it stays with the code, and you can describe a step at a time. We usually use a single-liner description, but for the more complex patches we should add a paragraph. I did it occasionally, not sure anybody noticed :) Sanne On 28 September 2011 11:27, Manik Surtani wrote: > > On 28 Sep 2011, at 10:05, Galder Zamarreño wrote: > > If you're gonna submit a big pull req, please use the description to > *summarise the changes* in order to help reviewers. This helps focus the > attention of the reviewer to the pieces of code that are most relevant. > > +1! > -- > Manik Surtani > ma...@jboss.org > twitter.com/maniksurtani > Lead, Infinispan > http://www.infinispan.org > > > > ___ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] optimistic locking - the time has come
On 28 Sep 2011, at 10:05, Galder Zamarreño wrote: > > If you're gonna submit a big pull req, please use the description to > *summarise the changes* in order to help reviewers. This helps focus the > attention of the reviewer to the pieces of code that are most relevant. +1! -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Lead, Infinispan http://www.infinispan.org ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Asymmetric caches and manual rehashing design
My 5 cents: Virtual Cache Views: - The REQUEST_VIEW_UPDATE is a good idea to manually trigger flushing of pending views. However, I'm not sure what the goal of DISABLE_VIEW_UPDATES is. Since I assume this is for the entire cluster, can 1 member disable view delivery for the entire cluster ? What scenario is this for ? - Are you clubbing (virtual) view updates and rebalancing together ? And if so (I should probably read on first...), can't you have view installations *without* rebalancing ? - Do we need the complex PREPARE_VIEW / ROLLBACK_VIEW / COMMIT_VIEW 2PC handling ? This adds a lot of complexity. Is it only used when we have a transactional cache ? - State is to be transferred *within* this 2PC time frame. Hmm, again, this ties rebalancing and view installation together (see my argument above)... On 9/27/11 6:22 PM, Dan Berindei wrote: > Following the discussions last week I've written up a wiki page > describing the strategies for cache view management and state transfer > that will enable asymmetric caches and manual rehashing: > http://community.jboss.org/wiki/AsymmetricCachesAndManualRehashingDesign > > The state transfer part is not very detailed, as you'll see we want to > go with a non-blocking approach but I'm not sure we can finish that > for 5.1 so we're keeping a blocking fallback option. > > Your comments are welcome, either on the list or on the wiki page. -- Bela Ban Lead JGroups (http://www.jgroups.org) JBoss / Red Hat ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Asymmetric caches and manual rehashing design
I'm commenting on the wiki page. On 27 Sep 2011, at 17:22, Dan Berindei wrote: > Following the discussions last week I've written up a wiki page > describing the strategies for cache view management and state transfer > that will enable asymmetric caches and manual rehashing: > http://community.jboss.org/wiki/AsymmetricCachesAndManualRehashingDesign > > The state transfer part is not very detailed, as you'll see we want to > go with a non-blocking approach but I'm not sure we can finish that > for 5.1 so we're keeping a blocking fallback option. > > Your comments are welcome, either on the list or on the wiki page. > > Dan > ___ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Lead, Infinispan http://www.infinispan.org ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] optimistic locking - the time has come
Thanks Mircea. I've made some comments. To all: If you're gonna submit a big pull req, please use the description to *summarise the changes* in order to help reviewers. This helps focus the attention of the reviewer to the pieces of code that are most relevant. On Sep 27, 2011, at 7:10 PM, Mircea Markus wrote: > Hi, > > I've issued a pull request for SPN-1131 (Add support for optimistic locking). > My last run brought the same number of test failures in master and in my > branch, so might be a good idea to integrate it first and then continue > fixing tests. > > Cheers, > Mircea > > > ___ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Galder Zamarreño Sr. Software Engineer Infinispan, JBoss Cache ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
