Hi Jay, Are you able to look at the logs or the web interface? Can you find out why it's getting killed?
Also, can you verify that these ports are open and a process is connected to them (maybe with netstat)? http://www.cloudera.com/blog/2009/08/hadoop-default-ports-quick-reference/ Thanks, Tom On Fri, Oct 28, 2011 at 7:57 PM, Jay Vyas <[email protected]> wrote: > Thanks tom : Thats interesting.... > > First, I tried, and it complained that the input directory didnt exist, so I > ran > $> hadoop fs -mkdir /user/cloudera/input > > Then, I tried to do this : > > $> hadoop jar /usr/lib/hadoop-0.20/hadoop-examples.jar grep input output2 > 'dfs[a-z.]+' > > And it seemed to start working ...... But then it abruptly printed "killed" > somehow at the end of the job [scroll down] ? > > Maybe this is related to why i cant connect ..... ?! > > 1) the hadoop jar 11/10/14 21:34:43 WARN util.NativeCodeLoader: Unable to > load native-hadoop library for your platform... using builtin-java classes > where applicable > 11/10/14 21:34:43 WARN snappy.LoadSnappy: Snappy native library not loaded > 11/10/14 21:34:43 INFO mapred.FileInputFormat: Total input paths to process > : 0 > 11/10/14 21:34:44 INFO mapred.JobClient: Running job: job_201110142010_0009 > 11/10/14 21:34:45 INFO mapred.JobClient: map 0% reduce 0% > 11/10/14 21:34:55 INFO mapred.JobClient: map 0% reduce 100% > 11/10/14 21:34:57 INFO mapred.JobClient: Job complete: job_201110142010_0009 > 11/10/14 21:34:57 INFO mapred.JobClient: Counters: 14 > 11/10/14 21:34:57 INFO mapred.JobClient: Job Counters > 11/10/14 21:34:57 INFO mapred.JobClient: Launched reduce tasks=1 > 11/10/14 21:34:57 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=5627 > 11/10/14 21:34:57 INFO mapred.JobClient: Total time spent by all reduces > waiting after reserving slots (ms)=0 > 11/10/14 21:34:57 INFO mapred.JobClient: Total time spent by all maps > waiting after reserving slots (ms)=0 > 11/10/14 21:34:57 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=5050 > 11/10/14 21:34:57 INFO mapred.JobClient: FileSystemCounters > 11/10/14 21:34:57 INFO mapred.JobClient: FILE_BYTES_WRITTEN=53452 > 11/10/14 21:34:57 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=86 > 11/10/14 21:34:57 INFO mapred.JobClient: Map-Reduce Framework > 11/10/14 21:34:57 INFO mapred.JobClient: Reduce input groups=0 > 11/10/14 21:34:57 INFO mapred.JobClient: Combine output records=0 > 11/10/14 21:34:57 INFO mapred.JobClient: Reduce shuffle bytes=0 > 11/10/14 21:34:57 INFO mapred.JobClient: Reduce output records=0 > 11/10/14 21:34:57 INFO mapred.JobClient: Spilled Records=0 > 11/10/14 21:34:57 INFO mapred.JobClient: Combine input records=0 > 11/10/14 21:34:57 INFO mapred.JobClient: Reduce input records=0 > 11/10/14 21:34:57 WARN mapred.JobClient: Use GenericOptionsParser for > parsing the arguments. Applications should implement Tool for the same. > 11/10/14 21:34:58 INFO mapred.FileInputFormat: Total input paths to process > : 1 > 11/10/14 21:34:58 INFO mapred.JobClient: Running job: job_201110142010_0010 > 11/10/14 21:34:59 INFO mapred.JobClient: map 0% reduce 0% > Killed > > > On Fri, Oct 28, 2011 at 8:24 PM, Tom Melendez <[email protected]> wrote: > >> Hi Jay, >> >> Some questions for you: >> >> - Does the hadoop client itself work from that same machine? >> - Are you actually able to run the hadoop example jar (in other words, >> your setup is valid otherwise)? >> - Is port 8020 actually available? (you can telnet or nc to it?) >> - What does jps show on the namenode? >> >> Thanks, >> >> Tom >> >> On Fri, Oct 28, 2011 at 4:04 PM, Jay Vyas <[email protected]> wrote: >> > Hi guys : Made more progress debugging my hadoop connection, but still >> > haven't got it working...... It looks like my VM (cloudera hadoop) won't >> > let me in. I find that there is no issue connecting to the name node - >> that >> > is , using hftp and 50070...... >> > >> > via standard HFTP as in here : >> > >> > //This method works fine - connecting directly to hadoop's namenode and >> > querying the filesystem >> > public static void main1(String[] args) throws Exception >> > { >> > String uri = "hftp://155.37.101.76:50070/";; >> > >> > System.out.println( "uri: " + uri ); >> > Configuration conf = new Configuration(); >> > >> > FileSystem fs = FileSystem.get( URI.create( uri ), conf ); >> > fs.printStatistics(); >> > } >> > >> > >> > But unfortunately, I can't get into hdfs ..... Any thoughts on this ? I >> am >> > modifying the uri to access port 8020 >> > which is what is in my core-site.xml . >> > >> > // This fails, resulting (trys to connect over and over again, >> eventually >> > gives up printing "already tried to connect 20 times"....) >> > public static void main(String[] args) >> > { >> > try { >> > String uri = "hdfs://155.37.101.76:8020/"; >> > >> > System.out.println( "uri: " + uri ); >> > Configuration conf = new Configuration(); >> > >> > FileSystem fs = FileSystem.get( URI.create( uri ), conf ); >> > fs.printStatistics(); >> > } catch (Exception e) { >> > // TODO Auto-generated catch block >> > e.printStackTrace(); >> > } >> > } >> > >> > The error message is : >> > >> > 11/10/28 19:03:38 INFO ipc.Client: Retrying connect to server: / >> > 155.37.101.76:8020. Already tried 0 time(s). >> > 11/10/28 19:03:39 INFO ipc.Client: Retrying connect to server: / >> > 155.37.101.76:8020. Already tried 1 time(s). >> > 11/10/28 19:03:40 INFO ipc.Client: Retrying connect to server: / >> > 155.37.101.76:8020. Already tried 2 time(s). >> > 11/10/28 19:03:41 INFO ipc.Client: Retrying connect to server: / >> > 155.37.101.76:8020. Already tried 3 time(s). >> > >> > Any thoughts on this would be *really* be appreciated ... Thanks guys. >> > >> > > > > -- > Jay Vyas > MMSB/UCHC >
