Re: Can "resource weight" be taken into consideration when load-balancing using Helix?

2018-07-12 Thread kishore g
I mostly agree with the solution proposed by Asmund. One addition would be that you can have replicas for each partition and that would give you fault tolerance. You can use FULL AUTO mode and let Helix manage everything. Regarding the statement "My understanding of helix is that it isn't trivial

Re: Can "resource weight" be taken into consideration when load-balancing using Helix?

2018-07-12 Thread Åsmund Tokheim
Hi I'm not that experienced with helix, so wait and see if anyone offers any corrections. My understanding of helix is that it isn't trivial to dynamically add a partition, and you in any case wouldn't want thousands of partitions or 'tasks'. For a problem like yours, I would define one task

Can "resource weight" be taken into consideration when load-balancing using Helix?

2018-07-12 Thread Diot Sébastien
Hi, First message. I've just discovered Apache Helix, while looking at Pinterest Rocksplicator. I was wandering if Helix could replace our home-grown load-balancer. We have a productive cluster of 12 "large" Java application servers, with a home-grown Java load-balancer, that acts as a