Yes, with this approach only two containers are required: one for stram and another for all operators. You can easily fit around 10 operators in less than 1GB. On 27 Sep 2015 00:32, "Timothy Farkas" <[email protected]> wrote:
> Hi Ram, > > You could make all the operators thread local. This cuts down on the > overhead of separate containers and maximizes the memory available to each > operator. > > Tim > > On Sat, Sep 26, 2015 at 10:07 AM, Munagala Ramanath <[email protected]> > wrote: > > > Hi, > > > > I was running into memory issues when deploying my app on the sandbox > > where all the operators were stuck forever in the PENDING state because > > they were being continually aborted and restarted because of the limited > > memory on the sandbox. After some experimentation, I found that the > > following config values seem to work: > > ------------------------------------------ > > <https://datatorrent.slack.com/archives/engineering/p1443263607000010> > > > > > > > > *<property> <name>dt.attr.MASTER_MEMORY_MB</name> > <value>500</value> > > </property> <property> <name>dt.application..operator.* > > > > > > > > > > > > *.attr.MEMORY_MB</name> <value>200</value> </property> <property> > > > > > <name>dt.application.TopNWordsWithQueries.operator.fileWordCount.attr.MEMORY_MB</name> > > <value>512</value> </property>* > > ------------------------------------------------ > > Are these reasonable values ? Is there a more systematic way of coming up > > with these values than trial-and-error ? Most of my operators -- with the > > exception of fileWordCount -- need very little memory; is there a way to > > cut all values down to the bare minimum and maximize available memory for > > this one operator ? > > > > Thanks. > > > > Ram > > >
