On Fri, Oct 28, 2011 at 8:41 PM, Tom Melendez <[email protected]> wrote:
> Hi Jay, > > Are you able to look at the logs or the web interface? Can you find > out why it's getting killed? > > Also, can you verify that these ports are open and a process is > connected to them (maybe with netstat)? > > http://www.cloudera.com/blog/2009/08/hadoop-default-ports-quick-reference/ > > Thanks, > > Tom > > On Fri, Oct 28, 2011 at 7:57 PM, Jay Vyas <[email protected]> wrote: > > Thanks tom : Thats interesting.... > > > > First, I tried, and it complained that the input directory didnt exist, > so I > > ran > > $> hadoop fs -mkdir /user/cloudera/input > > > > Then, I tried to do this : > > > > $> hadoop jar /usr/lib/hadoop-0.20/hadoop-examples.jar grep input output2 > > 'dfs[a-z.]+' > > > > And it seemed to start working ...... But then it abruptly printed > "killed" > > somehow at the end of the job [scroll down] ? > > > > Maybe this is related to why i cant connect ..... ?! > > > > 1) the hadoop jar 11/10/14 21:34:43 WARN util.NativeCodeLoader: Unable to > > load native-hadoop library for your platform... using builtin-java > classes > > where applicable > > 11/10/14 21:34:43 WARN snappy.LoadSnappy: Snappy native library not > loaded > > 11/10/14 21:34:43 INFO mapred.FileInputFormat: Total input paths to > process > > : 0 > > 11/10/14 21:34:44 INFO mapred.JobClient: Running job: > job_201110142010_0009 > > 11/10/14 21:34:45 INFO mapred.JobClient: map 0% reduce 0% > > 11/10/14 21:34:55 INFO mapred.JobClient: map 0% reduce 100% > > 11/10/14 21:34:57 INFO mapred.JobClient: Job complete: > job_201110142010_0009 > > 11/10/14 21:34:57 INFO mapred.JobClient: Counters: 14 > > 11/10/14 21:34:57 INFO mapred.JobClient: Job Counters > > 11/10/14 21:34:57 INFO mapred.JobClient: Launched reduce tasks=1 > > 11/10/14 21:34:57 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=5627 > > 11/10/14 21:34:57 INFO mapred.JobClient: Total time spent by all > reduces > > waiting after reserving slots (ms)=0 > > 11/10/14 21:34:57 INFO mapred.JobClient: Total time spent by all maps > > waiting after reserving slots (ms)=0 > > 11/10/14 21:34:57 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=5050 > > 11/10/14 21:34:57 INFO mapred.JobClient: FileSystemCounters > > 11/10/14 21:34:57 INFO mapred.JobClient: FILE_BYTES_WRITTEN=53452 > > 11/10/14 21:34:57 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=86 > > 11/10/14 21:34:57 INFO mapred.JobClient: Map-Reduce Framework > > 11/10/14 21:34:57 INFO mapred.JobClient: Reduce input groups=0 > > 11/10/14 21:34:57 INFO mapred.JobClient: Combine output records=0 > > 11/10/14 21:34:57 INFO mapred.JobClient: Reduce shuffle bytes=0 > > 11/10/14 21:34:57 INFO mapred.JobClient: Reduce output records=0 > > 11/10/14 21:34:57 INFO mapred.JobClient: Spilled Records=0 > > 11/10/14 21:34:57 INFO mapred.JobClient: Combine input records=0 > > 11/10/14 21:34:57 INFO mapred.JobClient: Reduce input records=0 > > 11/10/14 21:34:57 WARN mapred.JobClient: Use GenericOptionsParser for > > parsing the arguments. Applications should implement Tool for the same. > > 11/10/14 21:34:58 INFO mapred.FileInputFormat: Total input paths to > process > > : 1 > > 11/10/14 21:34:58 INFO mapred.JobClient: Running job: > job_201110142010_0010 > > 11/10/14 21:34:59 INFO mapred.JobClient: map 0% reduce 0% > > Killed > > > > > > On Fri, Oct 28, 2011 at 8:24 PM, Tom Melendez <[email protected]> wrote: > > > >> Hi Jay, > >> > >> Some questions for you: > >> > >> - Does the hadoop client itself work from that same machine? > >> - Are you actually able to run the hadoop example jar (in other words, > >> your setup is valid otherwise)? > >> - Is port 8020 actually available? (you can telnet or nc to it?) > >> - What does jps show on the namenode? > >> > >> Thanks, > >> > >> Tom > >> > >> On Fri, Oct 28, 2011 at 4:04 PM, Jay Vyas <[email protected]> wrote: > >> > Hi guys : Made more progress debugging my hadoop connection, but still > >> > haven't got it working...... It looks like my VM (cloudera hadoop) > won't > >> > let me in. I find that there is no issue connecting to the name node > - > >> that > >> > is , using hftp and 50070...... > >> > > >> > via standard HFTP as in here : > >> > > >> > //This method works fine - connecting directly to hadoop's namenode > and > >> > querying the filesystem > >> > public static void main1(String[] args) throws Exception > >> > { > >> > String uri = "hftp://155.37.101.76:50070/";; > >> > > >> > System.out.println( "uri: " + uri ); > >> > Configuration conf = new Configuration(); > >> > > >> > FileSystem fs = FileSystem.get( URI.create( uri ), conf ); > >> > fs.printStatistics(); > >> > } > >> > > >> > > >> > But unfortunately, I can't get into hdfs ..... Any thoughts on this ? > I > >> am > >> > modifying the uri to access port 8020 > >> > which is what is in my core-site.xml . > >> > > >> > // This fails, resulting (trys to connect over and over again, > >> eventually > >> > gives up printing "already tried to connect 20 times"....) > >> > public static void main(String[] args) > >> > { > >> > try { > >> > String uri = "hdfs://155.37.101.76:8020/"; > >> > > >> > System.out.println( "uri: " + uri ); > >> > Configuration conf = new Configuration(); > >> > > >> > FileSystem fs = FileSystem.get( URI.create( uri ), conf ); > >> > fs.printStatistics(); > >> > } catch (Exception e) { > >> > // TODO Auto-generated catch block > >> > e.printStackTrace(); > >> > } > >> > } > >> > > >> > The error message is : > >> > > >> > 11/10/28 19:03:38 INFO ipc.Client: Retrying connect to server: / > >> > 155.37.101.76:8020. Already tried 0 time(s). > >> > 11/10/28 19:03:39 INFO ipc.Client: Retrying connect to server: / > >> > 155.37.101.76:8020. Already tried 1 time(s). > >> > 11/10/28 19:03:40 INFO ipc.Client: Retrying connect to server: / > >> > 155.37.101.76:8020. Already tried 2 time(s). > >> > 11/10/28 19:03:41 INFO ipc.Client: Retrying connect to server: / > >> > 155.37.101.76:8020. Already tried 3 time(s). > >> > > >> > Any thoughts on this would be *really* be appreciated ... Thanks > guys. > >> > > >> > > > > > > > > -- > > Jay Vyas > > MMSB/UCHC > > >
