Re: [infinispan-dev] 5.3.0. release name: Tactical Nuclear Penguin
Oh boy! Now we are serious competition to Fedora codenames. Their last name one is Spherical Cow. Rado On 11/02/13 17:50, Mircea Markus wrote: Sometimes democracy gets funny: Tactical Nuclear Penguin is the name of our next 5.3 release[1]. It's worth mentioning we have a tie: Hopocalypsebeing the alternative. Let's use this one for our next release. [1] https://community.jboss.org/polls/1115 Cheers, -- Mircea Markus Infinispan lead (www.infinispan.org http://www.infinispan.org) ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Trigger NBST for Auto-Placer
Hi Dan, On 2/12/13 3:12 PM, Dan Berindei wrote: Hi Pedro When I split off the RebalancePolicy I was thinking that when a RebalancePolicy needs to collaborate with a ConsistentHashFactory, they should do so via another cache manager-scoped component. But that doesn't really work (yet?), because ConsistentHashFactory can't access any components. I didn't understand the previous sentence... Do I need to invoke anything in the RebalancePolicy? So far, I'm invoking directly in the ClusterTopologyManager: https://github.com/pruivo/infinispan/blob/cloudtm_v2/core/src/main/java/org/infinispan/dataplacement/DataPlacementManager.java#L246 Thanks! Cheers, Pedro I think it would be better to extend ClusterTopologyManager.triggerRebalance (and ConsistentHashFactory.rebalance) to accept an arbitrary Object parameter. Then RebalancePolicy could use this parameter to pass extra information to the CHF, like your Mappings object, and then when ClusterTopologyManagerImpl asks for a balanced CH, the CHF will include the Mappings in the result CH. What do you think? In order to trigger the rebalance you have to call startRebalance, and the new (balanced) consistent hash must not be equal to the existing consistent hash. See https://github.com/infinispan/infinispan/blob/master/core/src/main/java/org/infinispan/topology/ClusterTopologyManagerImpl.java#L389 Cheers Dan On Thu, Feb 7, 2013 at 10:05 PM, Pedro Ruivo pru...@gsd.inesc-id.pt mailto:pru...@gsd.inesc-id.pt wrote: Hi, I'm working in a way to rebase auto-placer on top of NBST and I have one question... If you have already forgot, auto-placer analyzes the workload and tries to move the most remote accessed keys to the corresponding requester. After calculating the new mappings, I want to trigger the NBST with this mapping. I'm thinking to add a new method in the ClusterTopologyManager, something like: triggerAutoPlacer(String cacheName, Mappings newMappings); and this method it will be a duplicate of triggerRebalance but instead of doing chFactory.rebalance(CH) (in the startRebalance() method) I'm thinking to do chFactory.autoPlacer(CH, Mappings). The last method will override the defautl CH location. Question: will this solution trigger the NBST or do I have to create the triggerAutoPlacer() method in another class? ps. forget the methods names... I will think in better names later Thanks!! Cheers, Pedro ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org mailto:infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Trigger NBST for Auto-Placer
Sorry, I didn't read your code so I just assumed you're writing your own RebalancePolicy. I think you need to implement your own RebalancePolicy, because ClusterTopologyManagerImpl by itself doesn't remember that a rebalance was triggerred. So if you call startRebalance, but there is already a rebalance in progress, it is just ignored. When the in-progress rebalance finishes, it calls RebalancePolicy.updateCacheStatus, and it's the RebalancePolicy implementation's job to start a new rebalance if needed. On Tue, Feb 12, 2013 at 5:28 PM, Pedro Ruivo pru...@gsd.inesc-id.pt wrote: ** Hi Dan, On 2/12/13 3:12 PM, Dan Berindei wrote: Hi Pedro When I split off the RebalancePolicy I was thinking that when a RebalancePolicy needs to collaborate with a ConsistentHashFactory, they should do so via another cache manager-scoped component. But that doesn't really work (yet?), because ConsistentHashFactory can't access any components. I didn't understand the previous sentence... Do I need to invoke anything in the RebalancePolicy? So far, I'm invoking directly in the ClusterTopologyManager: https://github.com/pruivo/infinispan/blob/cloudtm_v2/core/src/main/java/org/infinispan/dataplacement/DataPlacementManager.java#L246 Thanks! Cheers, Pedro I think it would be better to extend ClusterTopologyManager.triggerRebalance (and ConsistentHashFactory.rebalance) to accept an arbitrary Object parameter. Then RebalancePolicy could use this parameter to pass extra information to the CHF, like your Mappings object, and then when ClusterTopologyManagerImpl asks for a balanced CH, the CHF will include the Mappings in the result CH. What do you think? In order to trigger the rebalance you have to call startRebalance, and the new (balanced) consistent hash must not be equal to the existing consistent hash. See https://github.com/infinispan/infinispan/blob/master/core/src/main/java/org/infinispan/topology/ClusterTopologyManagerImpl.java#L389 Cheers Dan On Thu, Feb 7, 2013 at 10:05 PM, Pedro Ruivo pru...@gsd.inesc-id.ptwrote: Hi, I'm working in a way to rebase auto-placer on top of NBST and I have one question... If you have already forgot, auto-placer analyzes the workload and tries to move the most remote accessed keys to the corresponding requester. After calculating the new mappings, I want to trigger the NBST with this mapping. I'm thinking to add a new method in the ClusterTopologyManager, something like: triggerAutoPlacer(String cacheName, Mappings newMappings); and this method it will be a duplicate of triggerRebalance but instead of doing chFactory.rebalance(CH) (in the startRebalance() method) I'm thinking to do chFactory.autoPlacer(CH, Mappings). The last method will override the defautl CH location. Question: will this solution trigger the NBST or do I have to create the triggerAutoPlacer() method in another class? ps. forget the methods names... I will think in better names later Thanks!! Cheers, Pedro ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing listinfinispan-dev@lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Trigger NBST for Auto-Placer
Actually I'd like regular users to be able to define their own
RebalancePolicies and ConsistentHashFactories, without modifying
ClusterTopologyManagerImpl/ClusterCacheStatus, so I see this as a good
opportunity to modify our implementation to allow it.
On Tue, Feb 12, 2013 at 6:04 PM, Pedro Ruivo pru...@gsd.inesc-id.pt wrote:
**
Can I modify the existing one?
I'm thinking in the following:
ClusterTopologyManagerImpl.handleNewMappings(...) { //new method
ClusterCacheStatus status = //get status for cache name
status.setNewMappings(...) //synchronized of course
rebalancePolicy.updateCacheStatus(...);
}
Like I said, I'd like to keep ClusterTopologyManager as generic as possible
wrt rebalance strategies, so I think your DataPlacementManager should call
your custom RebalancePolicy directly.
The rebalance policy could keep the new mappings in a map on its own,
although maybe it would be a nice touch to allow storing custom state in
ClusterCacheStatus.
DefaultRebalancePolicy.updateCacheStatus(...) { //modified
...
if (!status.hasJoiners() isBalanced(...) !status.hasNewMappings())
{ //added last condition
return;
}
...
}
I guess you'd also need to clean up the old new mappings here after the
rebalance is done.
ClusterTopologyManagerImpl.startRebalance(...) { //modifed
...
chFactory.rebalance(ch);
chFactory.applyMappings(ch, status.getNewMappings()); //added.
... //if it is the same ch, no state transfer is triggered
}
This would require ClusterTopologyManagerImpl to know about your custom
ConsistentHashFactory, and it wouldn't work with another
ConsistentHashFactory that requires different custom data. So I'd rather we
add a generic parameter to ConsistentHashFactory.rebalance.
What do you think?
Thanks,
Pedro
On 2/12/13 3:39 PM, Dan Berindei wrote:
Sorry, I didn't read your code so I just assumed you're writing your own
RebalancePolicy.
I think you need to implement your own RebalancePolicy, because
ClusterTopologyManagerImpl by itself doesn't remember that a rebalance was
triggerred. So if you call startRebalance, but there is already a rebalance
in progress, it is just ignored. When the in-progress rebalance finishes,
it calls RebalancePolicy.updateCacheStatus, and it's the RebalancePolicy
implementation's job to start a new rebalance if needed.
On Tue, Feb 12, 2013 at 5:28 PM, Pedro Ruivo pru...@gsd.inesc-id.ptwrote:
Hi Dan,
On 2/12/13 3:12 PM, Dan Berindei wrote:
Hi Pedro
When I split off the RebalancePolicy I was thinking that when a
RebalancePolicy needs to collaborate with a ConsistentHashFactory, they
should do so via another cache manager-scoped component. But that doesn't
really work (yet?), because ConsistentHashFactory can't access any
components.
I didn't understand the previous sentence... Do I need to invoke
anything in the RebalancePolicy?
So far, I'm invoking directly in the ClusterTopologyManager:
https://github.com/pruivo/infinispan/blob/cloudtm_v2/core/src/main/java/org/infinispan/dataplacement/DataPlacementManager.java#L246
Thanks!
Cheers,
Pedro
I think it would be better to extend
ClusterTopologyManager.triggerRebalance (and
ConsistentHashFactory.rebalance) to accept an arbitrary Object parameter.
Then RebalancePolicy could use this parameter to pass extra information to
the CHF, like your Mappings object, and then when
ClusterTopologyManagerImpl asks for a balanced CH, the CHF will include the
Mappings in the result CH. What do you think?
In order to trigger the rebalance you have to call startRebalance, and
the new (balanced) consistent hash must not be equal to the existing
consistent hash. See
https://github.com/infinispan/infinispan/blob/master/core/src/main/java/org/infinispan/topology/ClusterTopologyManagerImpl.java#L389
Cheers
Dan
On Thu, Feb 7, 2013 at 10:05 PM, Pedro Ruivo pru...@gsd.inesc-id.ptwrote:
Hi,
I'm working in a way to rebase auto-placer on top of NBST and I have one
question...
If you have already forgot, auto-placer analyzes the workload and tries
to move the most remote accessed keys to the corresponding requester.
After calculating the new mappings, I want to trigger the NBST with this
mapping. I'm thinking to add a new method in the ClusterTopologyManager,
something like:
triggerAutoPlacer(String cacheName, Mappings newMappings);
and this method it will be a duplicate of triggerRebalance but instead
of doing chFactory.rebalance(CH) (in the startRebalance() method) I'm
thinking to do chFactory.autoPlacer(CH, Mappings). The last method will
override the defautl CH location.
Question: will this solution trigger the NBST or do I have to create the
triggerAutoPlacer() method in another class?
ps. forget the methods names... I will think in better names later
Thanks!!
Cheers,
Pedro
___
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
