[
https://issues.apache.org/jira/browse/WHIRR-391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tom White updated WHIRR-391:
----------------------------
Attachment: WHIRR-391.patch
Here's a patch to add a YARN service. You can start it using
{noformat}
bin/whirr launch-cluster --config recipes/hadoop-yarn-ec2.properties
{noformat}
Once it has started, log in to the head node and run a MR job:
{noformat}
# Currently need to run as hadoop
sudo su - hadoop
hadoop fs -put $HADOOP_CONF_DIR input
hadoop jar $HADOOP_HOME/hadoop-mapreduce-examples-0.24.0-SNAPSHOT.jar wordcount
\
-Dmapreduce.job.user.name=$USER \
-Dmapreduce.clientfactory.class.name=org.apache.hadoop.mapred.YarnClientFactory
\
-libjars
$YARN_HOME/modules/hadoop-mapreduce-client-jobclient-0.24.0-SNAPSHOT.jar \
input output
{noformat}
Current limitations:
* It changes the Hadoop service so that the MR daemons are not started. This
needs fixing before committing.
* There is no proxy, so you have to log into the head node to use the cluster.
It's still tricky to install Hadoop with YARN (see HADOOP-7642,
http://svn.apache.org/repos/asf/hadoop/common/trunk/hadoop-mapreduce-project/INSTALL),
so this limitation is OK for now.
> Write a YARN (Hadoop MR2) service for Whirr
> -------------------------------------------
>
> Key: WHIRR-391
> URL: https://issues.apache.org/jira/browse/WHIRR-391
> Project: Whirr
> Issue Type: New Feature
> Components: new service, service/hadoop
> Reporter: Tom White
> Assignee: Tom White
> Attachments: WHIRR-391.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
