Andrey Pankov wrote:
Hi all,
Currently I'm able to run map-reduce jobs from box where NameNode and
JobTracker are running. But I'd like to run my jobs from separate box,
from which I have access to HDFS. I have updated params
fs.default.name and mapred.job.tracker in local hadoop dir to point
to the clusters master. Now Hadoop returns me following error:
[EMAIL PROTECTED]:/usr/local/hadoop-0.16.0$ bin/hadoop jar
hadoop-0.16.0-examples.jar wordcount /user/username/gutenberg
/user/username/gutenberg-output
08/03/31 10:21:46 INFO mapred.FileInputFormat: Total input paths to
process : 3
org.apache.hadoop.ipc.RemoteException: java.io.IOException:
/mnt/hadoop/mapred/system/job_200803210640_0852/job.xml: No such file
or directory
I tried running this with trunk and it works. Started a cluster and
fired a job from my local machine with the local hadoop-site.xml
pointing to the JobTracker and Namenode. Can you provide us with more
details as to how to reproduce the problem?
Amar
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:159)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:133)
at
org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1083)
...
Here account 'username' has passwordless access to master box. Cluster
runs over EC2.
As a variant I can run tasks via ssh, i.e.
ssh master /usr/local/hadoop-0.16.0bin/hadoop jar
/home/username/jobs/hadoop-0.16.0-examples.jar wordcount
/user/username/gutenberg /user/username/gutenberg-output
But you need to put your jar file to the NameNode box before you run it.
Thanks in advance.
