u r most welcome :)
On Fri, Nov 15, 2013 at 12:46 PM, chandu banavaram < [email protected]> wrote: > thanks > > > On Thu, Nov 14, 2013 at 10:18 PM, unmesha sreeveni > <[email protected]>wrote: > >> @chandu banavaram: >> This exception usually happens if hdfs is trying to write into a file >> which is no more in hdfs.. >> >> I think in my case certain files are not created in my hdfs.it failed to >> create due to some permissions. >> >> I am trying out. >> >> >> On Wed, Nov 13, 2013 at 9:25 AM, unmesha sreeveni >> <[email protected]>wrote: >> >>> :) Ok >>> Why u also experienced the same? >>> >>> >>> On Tue, Nov 12, 2013 at 5:14 PM, chandu banavaram < >>> [email protected]> wrote: >>> >>>> plz send the answer to me for this query >>>> >>>> >>>> On Tue, Nov 12, 2013 at 2:52 AM, unmesha sreeveni < >>>> [email protected]> wrote: >>>> >>>>> While running job with 90 Mb file i am getting LeaseExpiredException >>>>> >>>>> 13/11/12 15:46:41 WARN mapred.JobClient: Use GenericOptionsParser for >>>>> parsing the arguments. Applications should implement Tool for the same. >>>>> 13/11/12 15:46:42 INFO input.FileInputFormat: Total input paths to >>>>> process : 1 >>>>> 13/11/12 15:46:43 INFO mapred.JobClient: Running job: >>>>> job_201310301645_25033 >>>>> 13/11/12 15:46:44 INFO mapred.JobClient: map 0% reduce 0% >>>>> 13/11/12 15:46:56 INFO mapred.JobClient: Task Id : >>>>> attempt_201310301645_25033_m_000000_0, Status : FAILED >>>>> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException): >>>>> Lease mismatch on /user/hdfs/in/map owned by >>>>> DFSClient_NONMAPREDUCE_-1622335545_1 but is accessed by >>>>> DFSClient_NONMAPREDUCE_-1561990512_1 >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2459) >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2437) >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFileInternal(FSNamesystem.java:2503) >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFile(FSNamesystem.java:2480) >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.complete(NameNodeRpcServer.java:556) >>>>> at >>>>> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.complete(ClientNamenodeProtocolServerSideTranslatorPB.java:337) >>>>> at >>>>> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:44958) >>>>> at org. >>>>> attempt_201310301645_25033_m_000000_0: SLF4J: Class path contains >>>>> multiple SLF4J bindings. >>>>> attempt_201310301645_25033_m_000000_0: SLF4J: Found binding in >>>>> [jar:file:/usr/lib/zookeeper/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] >>>>> attempt_201310301645_25033_m_000000_0: SLF4J: Found binding in >>>>> [jar:file:/tmp/hadoop-mapred/mapred/local/taskTracker/hdfs/jobcache/job_201310301645_25033/jars/job.jar!/org/slf4j/impl/StaticLoggerBinder.class] >>>>> attempt_201310301645_25033_m_000000_0: SLF4J: See >>>>> http://www.slf4j.org/codes.html#multiple_bindings for an explanation. >>>>> 13/11/12 15:47:02 INFO mapred.JobClient: Task Id : >>>>> attempt_201310301645_25033_m_000000_1, Status : FAILED >>>>> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException): >>>>> Lease mismatch on /user/hdfs/in/map owned by >>>>> DFSClient_NONMAPREDUCE_-1622335545_1 but is accessed by >>>>> DFSClient_NONMAPREDUCE_-1662926329_1 >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2459) >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:2262) >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2175) >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:501) >>>>> at >>>>> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:299) >>>>> at >>>>> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:44954) >>>>> at >>>>> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java >>>>> attempt_201310301645_25033_m_000000_1: SLF4J: Class path contains >>>>> multiple SLF4J bindings. >>>>> attempt_201310301645_25033_m_000000_1: SLF4J: Found binding in >>>>> [jar:file:/usr/lib/zookeeper/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] >>>>> attempt_201310301645_25033_m_000000_1: SLF4J: Found binding in >>>>> [jar:file:/tmp/hadoop-mapred/mapred/local/taskTracker/hdfs/jobcache/job_201310301645_25033/jars/job.jar!/org/slf4j/impl/StaticLoggerBinder.class] >>>>> attempt_201310301645_25033_m_000000_1: SLF4J: See >>>>> http://www.slf4j.org/codes.html#multiple_bindings for an explanation. >>>>> attempt_201310301645_25033_m_000000_1: log4j:WARN No appenders could >>>>> be found for logger (org.apache.hadoop.hdfs.DFSClient). >>>>> attempt_201310301645_25033_m_000000_1: log4j:WARN Please initialize >>>>> the log4j system properly. >>>>> attempt_201310301645_25033_m_000000_1: log4j:WARN See >>>>> http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. >>>>> 13/11/12 15:47:10 INFO mapred.JobClient: Task Id : >>>>> attempt_201310301645_25033_m_000001_0, Status : FAILED >>>>> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException): >>>>> No lease on /user/hdfs/in/map: File is not open for writing. Holder >>>>> DFSClient_NONMAPREDUCE_-1622335545_1 does not have any open files. >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2452) >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2437) >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFileInternal(FSNamesystem.java:2503) >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFile(FSNamesystem.java:2480) >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.complete(NameNodeRpcServer.java:556) >>>>> at >>>>> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.complete(ClientNamenodeProtocolServerSideTranslatorPB.java:337) >>>>> at >>>>> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:44958) >>>>> >>>>> Why is it so? >>>>> My mapper code is >>>>> >>>>> public void map(Object key, Text value, Context context) >>>>> throws IOException, InterruptedException { >>>>> Configuration conf = new Configuration(); >>>>> FileSystem fs = FileSystem.get(conf); >>>>> >>>>> Path inputfile = new Path("in/map"); >>>>> BufferedWriter getdatabuffer = new BufferedWriter(new >>>>> OutputStreamWriter(fs.create(inputfile))); >>>>> getdatabuffer.write(value.toString()); >>>>> getdatabuffer.close(); >>>>> Path Attribute = new Path("in/Attribute"); >>>>> int row =0; >>>>> BufferedReader read = new BufferedReader(new >>>>> InputStreamReader(fs.open(inputfile))); >>>>> String str = null; >>>>> while((str = read.readLine())!=null){ >>>>> >>>>> row++; //total row count >>>>> StringTokenizer st =new StringTokenizer(str," "); >>>>> col = st.countTokens(); >>>>> } >>>>> read.close(); >>>>> ........... >>>>> ........... >>>>> ............. >>>>> ............ >>>>> Further computation is based on the above "map" file. >>>>> >>>>> Why this happens? >>>>> I think it is unable to write into in/map for several times. >>>>> How to get rid of this? >>>>> *Any Suggestions?* >>>>> >>>>> -- >>>>> *Thanks & Regards* >>>>> >>>>> Unmesha Sreeveni U.B >>>>> >>>>> *Junior Developer* >>>>> >>>>> >>>>> >>>> >>> >>> >>> -- >>> *Thanks & Regards* >>> >>> Unmesha Sreeveni U.B >>> >>> *Junior Developer* >>> >>> >>> >> >> >> -- >> *Thanks & Regards* >> >> Unmesha Sreeveni U.B >> >> *Junior Developer* >> >> >> > -- *Thanks & Regards* Unmesha Sreeveni U.B *Junior Developer*
