I've figured out the answer to this. A slot is used by a worker. A worker is a JVM. So each JVM would require a clump of heap memory of its own. So a default of 4 slots would use 4*x amount of memory, where x is the memory used by a worker JVM. Now obviously if you declare more than 4 ports, it'll take up that much more memory. The problem with taking up too much memory, is that your topologies will suddenly crash with a GC overhead limit exceeded exception and the spout or bolt will get re-started constantly. As I understand, you'd be better off with increasing the number of servers or RAM on the existing server, if you want to have many workers/topologies.
On Mon, Jul 18, 2016 at 3:43 PM, Navin Ipe <[email protected]> wrote: > Ok, if there's an answer to the first question, then anyone who knows > details about Storm's design, please help in this thread. For the second > question, I'll be starting a separate thread, since there would be people > who'd have experience with running multiple topologies. > > On Mon, Jul 18, 2016 at 12:12 PM, Navin Ipe < > [email protected]> wrote: > >> Hi, >> >> I was initially surprised that Storm couldn't run more than 4 topologies >> by default. On increasing the number of supervisor slots, I was able to run >> more topologies. But there are two things I don't understand: >> >> 1. Was Storm designed to support only 4 default slots because it has to >> allocate memory for each slot and supporting 10 slots by default would have >> eaten up too much memory? >> 2. Since I have to go to each supervisor and alter each storm.yaml file >> to support more than 4 slots, then if I assign 5 slots to 5 supervisors, >> will I be able to run 5*5=25 topologies? (I've tried it only on my local >> system until now). >> >> -- >> Regards, >> Navin >> > > > > -- > Regards, > Navin > -- Regards, Navin
