Actually, can someone explain the replace REST API.
curl -i -b ~/cookiejar.txt -c ~/cookiejar.txt -X POST
-H "Content-Type: application/json"
-d '{"rebalanceOptions": {"numWorkers": 2, "executors": { "spout" :
"5", "split": 7, "count": 5 }}, "callback":"foo"}'
http://localhost:8080/api/v1/topology/wordcount-1-1420308665/rebalance/0
I take it /0 is a reference to a topology? How do I know what values to
use for the executors and numworkers?
Through the Storm UI, it's just a button. Anyone know how it determines
these values?
Thanks
On Tue, Jun 30, 2015 at 5:26 PM, Dillian Murphey <[email protected]>
wrote:
> Anyone have any updates on automatically re-balancing storm workers when
> using an autoscaling group?
>
> On Mon, Jun 22, 2015 at 2:50 AM, Andrew Wu <[email protected]> wrote:
>
>> I agree with you. I think there are 2 levels of scaling issue
>> 1) Topology level, mainly on number of workers. we are setting 2 twice
>> amount of workers than number of supervisors so that even we double number
>> of supervisors there will be at least 1 vm on each supervisor. it works on
>> sharing cpu load but does not work on memory.
>> 2) Cluster level, storm will not automatically use new supervisor until
>> topology is "rebalanced”.
>>
>> I have investigated this a bit and here’s something may help
>> This can be a solution for 2)
>> https://github.com/apache/storm/blob/master/STORM-UI-REST-API.md. you
>> can have a saperate monitor program to detect cluster changes and rebalance
>> topologies programatically.
>> 1) is a bit hard to solve, I*d like to know if there is any solution out
>> there as well.
>>
>>
>>
>> *Andrew Ge Wu*
>> Backend Developer & Integration Tech Lead
>> Mobile: +46 76 88 99 970
>>
>> Website: www.innometrics.com
>>
>>
>>
>> On 02 Jun 2015, at 17:31, Dmitry Goldenberg <[email protected]>
>> wrote:
>>
>> Hi,
>>
>> What's the latest perspective on cluster autoscaling in Storm? We want
>> to be able to have machines provisioned/added or decommissioned via AWS
>> based on the system load.
>>
>> E.g. this post:
>> https://groups.google.com/forum/#!searchin/storm-user/autoscaling/storm-user/HLpFAZvbwgU/14RTS3ltiwsJ
>> didn't seem to get an answer.
>>
>> Is there a way to pre-provision a pool of machines ready to go but not
>> fully in use, then use them as needed?
>>
>> I see
>> https://issues.apache.org/jira/browse/STORM-594
>> marked as Minor - ?
>>
>> This writeup seems to indicate there's autoscaling in Storm:
>>
>> https://mail-archives.apache.org/mod_mbox/storm-user/201501.mbox/%[email protected]%3E
>>
>> Any thoughts/recommendations? Thanks.
>>
>>
>>
>
