Hi Ram, You could make all the operators thread local. This cuts down on the overhead of separate containers and maximizes the memory available to each operator.
Tim On Sat, Sep 26, 2015 at 10:07 AM, Munagala Ramanath <[email protected]> wrote: > Hi, > > I was running into memory issues when deploying my app on the sandbox > where all the operators were stuck forever in the PENDING state because > they were being continually aborted and restarted because of the limited > memory on the sandbox. After some experimentation, I found that the > following config values seem to work: > ------------------------------------------ > <https://datatorrent.slack.com/archives/engineering/p1443263607000010> > > > > *<property> <name>dt.attr.MASTER_MEMORY_MB</name> <value>500</value> > </property> <property> <name>dt.application..operator.* > > > > > > *.attr.MEMORY_MB</name> <value>200</value> </property> <property> > > > <name>dt.application.TopNWordsWithQueries.operator.fileWordCount.attr.MEMORY_MB</name> > <value>512</value> </property>* > ------------------------------------------------ > Are these reasonable values ? Is there a more systematic way of coming up > with these values than trial-and-error ? Most of my operators -- with the > exception of fileWordCount -- need very little memory; is there a way to > cut all values down to the bare minimum and maximize available memory for > this one operator ? > > Thanks. > > Ram >
