Yes I hav mark it as final. Now one more exception arises, my map reduce
program for word count is throwing exception.
09/05/21 11:31:37 WARN mapred.JobClient: Use GenericOptionsParser for
parsing the arguments. Applications should implement Tool for the same.
09/05/21 11:31:37 INFO hdfs.DFSClient:
org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
/hadoop/mapred/system/job_200905211117_0003/job.jar could only be
replicated to 0 nodes, instead of 1
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(F
SNamesystem.java:1280)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:3
51)
at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
Impl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:481)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:894)
at org.apache.hadoop.ipc.Client.call(Client.java:697)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:216)
at $Proxy0.addBlock(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvo
cationHandler.java:82)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocation
Handler.java:59)
at $Proxy0.addBlock(Unknown Source)
at
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DF
SClient.java:2814)
at
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(D
FSClient.java:2696)
at
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2000(DFSClient.j
ava:1996)
at
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSCli
ent.java:2183)
09/05/21 11:31:37 WARN hdfs.DFSClient: NotReplicatedYetException
sleeping /hadoop/mapred/system/job_200905211117_0003/job.jar retries
left 4
-----Original Message-----
From: Michael C. Toren [mailto:[email protected]]
Sent: Thursday, May 21, 2009 11:27 AM
To: [email protected]
Subject: Re: More replication of map reduce output
On Thu, May 21, 2009 at 11:16:59AM +0530, Puri, Aseem wrote:
> I mean when my reduce tasks is set 1 part-00000 filw shows replication
facor as 3. But I set replication factor as 1 in hadoop-site.xml
Did you mark the replication factor configuration option as "final"?
e.g.:
<property>
<name>dfs.replication</name>
<value>1</value>
<final>true</final>
</property>
-mct
