Chris Riccomini created SAMZA-37:
------------------------------------
Summary: Allow run-am.sh override for YARN AM or disable
run-task.sh override
Key: SAMZA-37
URL: https://issues.apache.org/jira/browse/SAMZA-37
Project: Samza
Issue Type: Bug
Components: container, yarn
Affects Versions: 0.6.0
Reporter: Chris Riccomini
Samza allows developers to over-ride the path for the run-task.sh script, but
does not allow developers to override the path for the run-am.sh script. The
run-am.sh script is hard coded to __package/bin/run-am.sh.
The two relevant configurations that allow run-task.sh override are:
* task.command.class=org.apache.samza.job.ShellCommandBuilder
* task.execute=bin/run-task.sh
The motivation for allowing this override is to support Samza job packages with
different package layouts, and also to allow for supporting other languages
(e.g. running a python virtual environment as a Samza job).
If we believe that the run-task.sh override feature is a good thing (one might
argue it's kind of complex, and we should just force a convention), then we
should allow run-am.sh override using the same style. If we go this route, we
should add the following configuration to YarnConfig:
* yarn.am.command.class=org.apache.samza.job.YarnAmShellCommandBuilder
* yarm.am.execute=bin/run-task.sh
The YarnAmShellCommandBuilder should just extend ShellCommandBuilder, but
override buildCommand to call config.getAmCommand (yarn.am.execute), which will
be defined in YarnConfig.
I think we should just force the bin/run-task.sh convention, and disable
run-task.sh override. We can still keep the ShellCommand* classes around, since
they help build the environment variables for the SamzaContainer.
I also think we should rename run-task.sh to run-container.sh, but I'll open a
separate ticket for that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
