Repository: incubator-samza Updated Branches: refs/heads/master c79033454 -> d382cb6f3
SAMZA-491; support non-default queues in YARN Project: http://git-wip-us.apache.org/repos/asf/incubator-samza/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-samza/commit/d382cb6f Tree: http://git-wip-us.apache.org/repos/asf/incubator-samza/tree/d382cb6f Diff: http://git-wip-us.apache.org/repos/asf/incubator-samza/diff/d382cb6f Branch: refs/heads/master Commit: d382cb6f38ee5d6186f5077080bf45fdb33775c9 Parents: c790334 Author: Jay Bae <metac...@gmail.com> Authored: Tue Dec 9 10:12:09 2014 -0800 Committer: Chris Riccomini <cricc...@criccomi-mn.linkedin.biz> Committed: Tue Dec 9 10:12:09 2014 -0800 ---------------------------------------------------------------------- .../documentation/versioned/jobs/configuration-table.html | 10 +++++++++- docs/learn/documentation/versioned/jobs/yarn-jobs.md | 10 +++++++++- 2 files changed, 18 insertions(+), 2 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-samza/blob/d382cb6f/docs/learn/documentation/versioned/jobs/configuration-table.html ---------------------------------------------------------------------- diff --git a/docs/learn/documentation/versioned/jobs/configuration-table.html b/docs/learn/documentation/versioned/jobs/configuration-table.html index 12955ad..6a5d209 100644 --- a/docs/learn/documentation/versioned/jobs/configuration-table.html +++ b/docs/learn/documentation/versioned/jobs/configuration-table.html @@ -1244,7 +1244,7 @@ <td class="property" id="yarn-am-poll-interval-ms">yarn.am.poll.interval.ms</td> <td class="default">1000</td> <td class="description"> - THe Samza ApplicationMaster sends regular heartbeats to the YARN ResourceManager + The Samza ApplicationMaster sends regular heartbeats to the YARN ResourceManager to confirm that it is alive. This property determines the time (in milliseconds) between heartbeats. </td> @@ -1260,6 +1260,14 @@ </tr> <tr> + <td class="property" id="yarn-queue">yarn.queue</td> + <td class="default"></td> + <td class="description"> + Determines which YARN queue will be used for Samza job. + </td> + </tr> + + <tr> <th colspan="3" class="section" id="metrics"><a href="../container/metrics.html">Metrics</a></th> </tr> http://git-wip-us.apache.org/repos/asf/incubator-samza/blob/d382cb6f/docs/learn/documentation/versioned/jobs/yarn-jobs.md ---------------------------------------------------------------------- diff --git a/docs/learn/documentation/versioned/jobs/yarn-jobs.md b/docs/learn/documentation/versioned/jobs/yarn-jobs.md index 58ca50d..827cc14 100644 --- a/docs/learn/documentation/versioned/jobs/yarn-jobs.md +++ b/docs/learn/documentation/versioned/jobs/yarn-jobs.md @@ -19,7 +19,7 @@ title: YARN Jobs limitations under the License. --> -When you define `job.factory.class=org.apache.samza.job.yarn.YarnJobFactory` in your job's configuration, Samza will use YARN to execute your job. The YarnJobFactory will use the YARN_HOME environment variable on the machine that run-job.sh is executed on to get the appropriate YARN configuration, which will define where the YARN resource manager is. The YarnJob will work with the resource manager to get your job started on the YARN cluster. +When you define `job.factory.class=org.apache.samza.job.yarn.YarnJobFactory` in your job's configuration, Samza will use YARN to execute your job. The YarnJobFactory will use the HADOOP_YARN_HOME environment variable on the machine that run-job.sh is executed on to get the appropriate YARN configuration, which will define where the YARN resource manager is. The YarnJob will work with the resource manager to get your job started on the YARN cluster. If you want to use YARN to run your Samza job, you'll also need to define the location of your Samza job's package. For example, you might say: @@ -29,6 +29,14 @@ yarn.package.path=http://my.http.server/jobs/ingraphs-package-0.0.55.tgz This .tgz file follows the conventions outlined on the [Packaging](packaging.html) page (it has bin/run-am.sh and bin/run-container.sh). YARN NodeManagers will take responsibility for downloading this .tgz file on the appropriate machines, and untar'ing them. From there, YARN will execute run-am.sh or run-container.sh for the Samza Application Master, and SamzaContainer, respectively. +If you want to run Samza job in the specific YARN queue not the default one, you can set `yarn.queue` property in your job's configuration. For example, with the following property setting, + +{% highlight jproperties %} +yarn.queue=root.adhoc +{% endhighlight %} + +Samza job will run in the root.adhoc queue. + <!-- TODO document yarn.container.count and other key configs --> ## [Logging »](logging.html)
