[
https://issues.apache.org/jira/browse/SAMZA-76?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13827965#comment-13827965
]
Chris Riccomini commented on SAMZA-76:
--------------------------------------
Thinking on this a bit more. Actually, I think we should just replace
SAMZA_OPTS completely with JAVA_OPTS, since that's a more standard name.
In this case, we should just do a search/replace in run-class.sh from
SAMZA_OPTS to JAVA_OPTS. Then, in the -z block where we set up defaults (e.g.
the -XX:+PrintGCDateStamps stuff ) we should just default to -Xmx768M there.
We'll need to then update all mentions of SAMZA_OPTS in the code to be
JAVA_OPTS instead:
{noformat}
./samza-core/src/main/scala/org/apache/samza/config/ShellCommandConfig.scala:
val ENV_SAMZA_OPTS = "SAMZA_OPTS"
./samza-core/src/main/scala/org/apache/samza/job/ShellCommandBuilder.scala:
ShellCommandConfig.ENV_SAMZA_OPTS -> config.getTaskOpts.getOrElse(""))
./samza-shell/src/main/bash/run-class.sh:if [ -z "$SAMZA_OPTS" ]; then
./samza-shell/src/main/bash/run-class.sh: SAMZA_OPTS="-Xmx160M
-XX:+PrintGCDateStamps -Xloggc:$SAMZA_LOG_DIR/gc.log
-Dsamza.log.dir=$SAMZA_LOG_DIR -Dsamza.container.name=$SAMZA_CONTAINER_NAME"
./samza-shell/src/main/bash/run-class.sh: SAMZA_OPTS="$SAMZA_OPTS
-Dlog4j.configuration=file:$base_dir/lib/log4j.xml"
./samza-shell/src/main/bash/run-class.sh:echo $JAVA $SAMZA_OPTS -cp $CLASSPATH
$@
./samza-shell/src/main/bash/run-class.sh:exec $JAVA $SAMZA_OPTS -cp $CLASSPATH
$@
./samza-yarn/src/main/scala/org/apache/samza/job/yarn/YarnJob.scala:
ShellCommandConfig.ENV_SAMZA_OPTS ->
Util.envVarEscape(config.getAmOpts.getOrElse("")))),
{noformat}
Lastly, we'll still need to update the yarn.container.memory.mb, and
yarn.am.container.memory.mb default values to 1024, as described above.
> Improve default memory settings for samza tasks
> -----------------------------------------------
>
> Key: SAMZA-76
> URL: https://issues.apache.org/jira/browse/SAMZA-76
> Project: Samza
> Issue Type: Improvement
> Reporter: Jakob Homan
> Assignee: Rekha Joshi
> Priority: Minor
> Attachments: SAMZA_76.patch, run_class.sh
>
>
> The {{run-class.sh}} script hardcodes the xmx setting for the launched class
> at 160M. This is generally too low, and should be bumped up, but should also
> be specified in a more overridable-friendly way via an environmental variable.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
