[
https://issues.apache.org/jira/browse/HADOOP-5123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12667406#action_12667406
]
Steve Loughran commented on HADOOP-5123:
----------------------------------------
The use case for the {{<submit>}} ant task is to submit a job as part of a
build; print
out enough information for you to track it's progress. Upload the JAR file.
{code}
<hadoop:submit tracker="http://jobtracker:50030";
in="hdfs://host:port/tmp/in/something"
out="hdfs://host:port/tmp/out/something"
jobProperty="myJob"
jar="dist/myapp.jar"
>
<property name="dfs.replication.factor" value="4" />
<mapper classname="org.example.identity" />
<reducer classname="org.example.count" />
</hadoop:submit>
{code}
# No attempt to do a block for the job submission. The task will print out
the jobID.
# jobProperty names a property to set for the job ID
# list zero or more JAR files. No attempt to do sanity checks like loading
classes -the far end can do that.
# No separate configuration files for the map/reduce/combine
# Maybe, a configuration file attribute {{conf}}; defines a conf file to use.
If set, no other properties can be set (would force the ant task to parse the
XML, edit it, save it etc.
# JAR file is optional, but if listed, it had better be there
Tests without cluster
* fail to submit if the JAR is missing
* fail to submit if there is no tracker
* error if the mapper or reducer is not defined
Tests with MiniMR up
* submit a job
> Ant tasks for job submission
> ----------------------------
>
> Key: HADOOP-5123
> URL: https://issues.apache.org/jira/browse/HADOOP-5123
> Project: Hadoop Core
> Issue Type: New Feature
> Affects Versions: 0.21.0
> Environment: Both platforms, Linux and Windows
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Minor
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Ant tasks to make it easy to work with hadoop filesystem and submit jobs.
> <submit> : uploads JAR, submits job as user, with various settings
> filesystem operations: mkdir, copyin, copyout, delete
> -We could maybe use Ant1.7 "resources" here, and so use hdfs as a source or
> dest in Ant's own tasks
> # security. Need to specify user; pick up user.name from JVM as default?
> # cluster binding: namenode/job tracker (hostname,port) or url are all that
> is needed?
> #job conf: how to configure the job that is submitted? support a list of
> <property name="name" value="something"> children
> # testing. AntUnit to generate <junitreport> compatible XML files
> # Documentation. With an example using Ivy to fetch the JARs for the tasks
> and hadoop client.
> # Polling: ant task to block for a job finished?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
