what are those libraries and how are they reading data from HDFS? you were trying with MR jobs if i'm not wrong? in order to perform read/write on HDFS we need HDFS API with a Configuration object. how are you doing it here?
Best Regards, Tariq +91-9741563634 https://mtariq.jux.com/ On Fri, Dec 28, 2012 at 2:38 AM, jamal sasha <[email protected]> wrote: > Hi, > Thanks for throwing insight. > > So the code snippet looks like this > > String interout = final_output + "/intermediate"; > > try { > new CreateInterOutput().main(new String[] { input, interout }); > } catch (Exception e) { > e.printStackTrace(); > return; > } > > try { > new CreateFinalOutput().main(new String[] { interout, final_output }); > > } catch (Exception e) { > e.printStackTrace(); > return; > } > > Any suggestions where it might be faltering? > > > On Thu, Dec 27, 2012 at 12:49 PM, Mohammad Tariq <[email protected]>wrote: > >> Hello Jamal, >> >> Please find my commands embedded below : >> >> Q1) How did putting those two lines solved the issue?? >> >> By adding those two resources you make sure that your code looks for >> the input path inside HDFS, which would otherwise look for it in the local >> FS by default. The files core-site.xml and hdfs-site.xml tell your code >> where to go for NN and DN. >> >> Q2) >> I am now using third party libraries which are taking input from hdfs >> and writing output to hdfs... >> But in an intermediatory step, it creates a raw output.. I am again >> getting the error: >> ERROR security.UserGroupInformation: PriviledgedActionException >> as:mhduser cause:org.apache.hadoop.mapred.InvalidInputException: Input path >> does not exist: hdfs://localhost:54310/user/hduser/wiki-inter-output >> How do I resolve this? >> >>If you are able to compile your code properly then there is no problem >> with the third party libraries which you are using. It looks like to me >> that your code doesn't have the proper info about the intermediate path. >> Please make sure you have told your code the exact location of intermediate >> output. >> >> >> Best Regards, >> Tariq >> +91-9741563634 >> https://mtariq.jux.com/ >> >> >> On Fri, Dec 28, 2012 at 1:33 AM, jamal sasha <[email protected]>wrote: >> >>> Hi, >>> So I am still in process of learning hadoop. >>> I tried to run wordcount.java (by writing my own mapper reducer.. >>> creating jar and then running it in a pseudo distributed mode). >>> >>> At that time I got an error, something like >>> ERROR security.UserGroupInformation: PriviledgedActionException >>> as:mhduser cause:org.apache.hadoop.mapred.InvalidInputException: Input path >>> does not exist: hdfs://localhost:54310/user/hduser/wiki >>> So I googled around.. and found that I should put the following two >>> lines on my driver code: >>> >>> conf.addResource(new Path("/usr/local/hadoop/conf/core-site.xml")); >>> conf.addResource(new Path("/usr/local/hadoop/conf/hdfs-site.xml >>> file")); >>> >>> (path of where my core-site and hdfs-site are) and after that it ran >>> just fine. >>> >>> Q1) How did putting those two lines solved the issue?? >>> Q2) >>> I am now using third party libraries which are taking input from hdfs >>> and writing output to hdfs... >>> But in an intermediatory step, it creates a raw output.. I am again >>> getting the error: >>> ERROR security.UserGroupInformation: PriviledgedActionException >>> as:mhduser cause:org.apache.hadoop.mapred.InvalidInputException: Input path >>> does not exist: hdfs://localhost:54310/user/hduser/wiki-inter-output >>> How do I resolve this? >>> >>> Any suggestions. >>> THanks >>> Jamal. >>> >> >> >
