[
https://issues.apache.org/jira/browse/SAMZA-76?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13829113#comment-13829113
]
Chris Riccomini commented on SAMZA-76:
--------------------------------------
Nice. Few things:
1. The yarn.container.memory.mb and yarn.am.container.memory.mb are both used
in code, and need to be updated. The places where they're used are:
{noformat}
./samza-yarn/src/main/scala/org/apache/samza/job/yarn/YarnJob.scala:
config.getAMContainerMaxMemoryMb.getOrElse(512),
./samza-yarn/src/main/scala/org/apache/samza/job/yarn/SamzaAppMasterTaskManager.scala:
val DEFAULT_CONTAINER_MEM = 256
{noformat}
2. As part of (1), could you add an object YarnJob {} to YarnJob.scala, and put
a DEFAULT_AM_CONTAINER_MEM variable in there (set to 768). This is the same
pattern as we have for DEFAULT_CONTAINER_MEM.
3. Could you post an RB?
4. Did you do any testing on this? It's kind of tricky to test the bash
scripts, so I usually just try and run hello-samza with my changes released to
~/.m2 (./gradlew -PscalaVersion=2.8.1 clean publishToMavenLocal). If you can do
that, and verify that the containers are coming up with -Xmx7468 (in the stdout
logs) and container sizes of 1024M (in the YARN UI), then I think we're set.
> Improve default memory settings for samza tasks
> -----------------------------------------------
>
> Key: SAMZA-76
> URL: https://issues.apache.org/jira/browse/SAMZA-76
> Project: Samza
> Issue Type: Improvement
> Reporter: Jakob Homan
> Assignee: Rekha Joshi
> Priority: Minor
> Attachments: SAMZA_76.patch, SAMZA_76_1.patch
>
>
> The {{run-class.sh}} script hardcodes the xmx setting for the launched class
> at 160M. This is generally too low, and should be bumped up, but should also
> be specified in a more overridable-friendly way via an environmental variable.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
