How can one calculate how much memory is needed? On Tue, Jul 19, 2016 at 1:05 AM, Navin Ipe <[email protected]> wrote:
> I've figured out the answer to this. A slot is used by a worker. A worker > is a JVM. So each JVM would require a clump of heap memory of its own. So a > default of 4 slots would use 4*x amount of memory, where x is the memory > used by a worker JVM. > Now obviously if you declare more than 4 ports, it'll take up that much > more memory. > The problem with taking up too much memory, is that your topologies will > suddenly crash with a GC overhead limit exceeded exception and the spout or > bolt will get re-started constantly. > As I understand, you'd be better off with increasing the number of servers > or RAM on the existing server, if you want to have many workers/topologies. > > > On Mon, Jul 18, 2016 at 3:43 PM, Navin Ipe < > [email protected]> wrote: > >> Ok, if there's an answer to the first question, then anyone who knows >> details about Storm's design, please help in this thread. For the second >> question, I'll be starting a separate thread, since there would be people >> who'd have experience with running multiple topologies. >> >> On Mon, Jul 18, 2016 at 12:12 PM, Navin Ipe < >> [email protected]> wrote: >> >>> Hi, >>> >>> I was initially surprised that Storm couldn't run more than 4 topologies >>> by default. On increasing the number of supervisor slots, I was able to run >>> more topologies. But there are two things I don't understand: >>> >>> 1. Was Storm designed to support only 4 default slots because it has to >>> allocate memory for each slot and supporting 10 slots by default would have >>> eaten up too much memory? >>> 2. Since I have to go to each supervisor and alter each storm.yaml file >>> to support more than 4 slots, then if I assign 5 slots to 5 supervisors, >>> will I be able to run 5*5=25 topologies? (I've tried it only on my local >>> system until now). >>> >>> -- >>> Regards, >>> Navin >>> >> >> >> >> -- >> Regards, >> Navin >> > > > > -- > Regards, > Navin > -- 是故勝兵先勝而後求戰，敗兵先戰而後求勝。
