See inline at **. More questions and many Thanks :D
>________________________________ >From: Uma Maheswara Rao G 72686 <[email protected]> >To: [email protected]; A Df <[email protected]> >Cc: "[email protected]" <[email protected]> >Sent: Wednesday, 27 July 2011, 17:31 >Subject: Re: cygwin not connecting to Hadoop server > > >Hi A Df, > >Did you format the NameNode first? > >** I had formatted it already but then I had reinstalled Java and upgraded the >plugins in cygwin so I reformatted it again. :D yes it worked!! I am not sure >all the steps that got it to finally work but I will have to document it to >prevent this headache in the future. Although I typed ssh localhost too , so >question is, do I need to type ssh localhost each time I need to run hadoop?? >Also, since I need to work with Eclipse maybe you can have a look at my post >about the plugin cause I can get the patch to work. The subject is "Re: Cygwin >not working with Hadoop and Eclipse Plugin". I plan to read up on how to write >programs for Hadoop. I am using the tutorial at Yahoo but if you know of any >really good about coding with Hadoop or just about understanding Hadoop then >please let me know. > >Can you check the NN logs whether NN is started or not? >** I checked and the previous runs had some logs missing but now the last one >have all 5 logs and I got two conf files in xml. I also copied out the other >output files which I plan to examine. Where do I specify the output extension >format that I want for my output file? I was hoping for an txt file it shows >the output in a file with no extension even though I can read it in Notepad++. >I also got to view the web interface at: > NameNode - http://localhost:50070/ > JobTracker - http://localhost:50030/ > >** See below for the working version, finally!! Thanks ><CMD> >Williams@TWilliams-LTPC ~/hadoop-0.20.2 >$ bin/hadoop jar hadoop-0.20.2-examples.jar grep input >11/07/27 17:42:20 INFO mapred.FileInputFormat: Total in > >11/07/27 17:42:20 INFO mapred.JobClient: Running job: j >11/07/27 17:42:21 INFO mapred.JobClient: map 0% reduce >11/07/27 17:42:33 INFO mapred.JobClient: map 15% reduc >11/07/27 17:42:36 INFO mapred.JobClient: map 23% reduc >11/07/27 17:42:39 INFO mapred.JobClient: map 38% reduc >11/07/27 17:42:42 INFO mapred.JobClient: map 38% reduc >11/07/27 17:42:45 INFO mapred.JobClient: map 53% reduc >11/07/27 17:42:48 INFO mapred.JobClient: map 69% reduc >11/07/27 17:42:51 INFO mapred.JobClient: map 76% reduc >11/07/27 17:42:54 INFO mapred.JobClient: map 92% reduc >11/07/27 17:42:57 INFO mapred.JobClient: map 100% redu >11/07/27 17:43:06 INFO mapred.JobClient: map 100% redu >11/07/27 17:43:09 INFO mapred.JobClient: Job complete: >11/07/27 17:43:09 INFO mapred.JobClient: Counters: 18 >11/07/27 17:43:09 INFO mapred.JobClient: Job Counters >11/07/27 17:43:09 INFO mapred.JobClient: Launched r >11/07/27 17:43:09 INFO mapred.JobClient: Launched m >11/07/27 17:43:09 INFO mapred.JobClient: Data-local >11/07/27 17:43:09 INFO mapred.JobClient: FileSystemCo >11/07/27 17:43:09 INFO mapred.JobClient: FILE_BYTES >11/07/27 17:43:09 INFO mapred.JobClient: HDFS_BYTES >11/07/27 17:43:09 INFO mapred.JobClient: FILE_BYTES >11/07/27 17:43:09 INFO mapred.JobClient: HDFS_BYTES >11/07/27 17:43:09 INFO mapred.JobClient: Map-Reduce F >11/07/27 17:43:09 INFO mapred.JobClient: Reduce inp >11/07/27 17:43:09 INFO mapred.JobClient: Combine ou >11/07/27 17:43:09 INFO mapred.JobClient: Map input >11/07/27 17:43:09 INFO mapred.JobClient: Reduce shu >11/07/27 17:43:09 INFO mapred.JobClient: Reduce out >11/07/27 17:43:09 INFO mapred.JobClient: Spilled Re >11/07/27 17:43:09 INFO mapred.JobClient: Map output >11/07/27 17:43:09 INFO mapred.JobClient: Map input >11/07/27 17:43:09 INFO mapred.JobClient: Combine in >11/07/27 17:43:09 INFO mapred.JobClient: Map output >11/07/27 17:43:09 INFO mapred.JobClient: Reduce inp >11/07/27 17:43:09 WARN mapred.JobClient: Use GenericOpt >e arguments. Applications should implement Tool for the >11/07/27 17:43:09 INFO mapred.FileInputFormat: Total in >11/07/27 17:43:09 INFO mapred.JobClient: Running job: j >11/07/27 17:43:10 INFO mapred.JobClient: map 0% reduce >11/07/27 17:43:22 INFO mapred.JobClient: map 100% redu >11/07/27 17:43:31 INFO mapred.JobClient: map 100% redu >11/07/27 17:43:36 INFO mapred.JobClient: map 100% redu >11/07/27 17:43:38 INFO mapred.JobClient: Job complete: >11/07/27 17:43:39 INFO mapred.JobClient: Counters: 18 >11/07/27 17:43:39 INFO mapred.JobClient: Job Counters >11/07/27 17:43:39 INFO mapred.JobClient: Launched r >11/07/27 17:43:39 INFO mapred.JobClient: Launched m >11/07/27 17:43:39 INFO mapred.JobClient: Data-local >11/07/27 17:43:39 INFO mapred.JobClient: FileSystemCo >11/07/27 17:43:39 INFO mapred.JobClient: FILE_BYTES >11/07/27 17:43:39 INFO mapred.JobClient: HDFS_BYTES >11/07/27 17:43:39 INFO mapred.JobClient: FILE_BYTES >11/07/27 17:43:39 INFO mapred.JobClient: HDFS_BYTES >11/07/27 17:43:39 INFO mapred.JobClient: Map-Reduce F >11/07/27 17:43:39 INFO mapred.JobClient: Reduce inp >11/07/27 17:43:39 INFO mapred.JobClient: Combine ou >11/07/27 17:43:39 INFO mapred.JobClient: Map input >11/07/27 17:43:39 INFO mapred.JobClient: Reduce shu >11/07/27 17:43:39 INFO mapred.JobClient: Reduce out >11/07/27 17:43:39 INFO mapred.JobClient: Spilled Re >11/07/27 17:43:39 INFO mapred.JobClient: Map output >11/07/27 17:43:39 INFO mapred.JobClient: Map input >11/07/27 17:43:39 INFO mapred.JobClient: Combine in >11/07/27 17:43:39 INFO mapred.JobClient: Map output >11/07/27 17:43:39 INFO mapred.JobClient: Reduce inp > >Williams@TWilliams-LTPC ~/hadoop-0.20.2 >$ bin/hadoop fs -get output output > >Williams@TWilliams-LTPC ~/hadoop-0.20.2 >$ cat output/* >cat: output/_logs: Is a directory >3 dfs.class >2 dfs.period >1 dfs.file >1 dfs.replication >1 dfs.servers >1 dfsadmin >1 dfsmetrics.log ></CMD> > >Regards, >Uma >****************************************************************************************** >This email and its attachments contain confidential information from HUAWEI, >which is intended only for the person or entity whose address is listed above. >Any use of the information contained here in any way (including, but not >limited to, total or partial disclosure, reproduction, or dissemination) by >persons other than the intended recipient(s) is prohibited. If you receive >this email in error, please notify the sender by phone or email immediately >and delete it! >***************************************************************************************** > >----- Original Message ----- >From: A Df <[email protected]> >Date: Wednesday, July 27, 2011 9:55 pm >Subject: cygwin not connecting to Hadoop server >To: "[email protected]" <[email protected]> > >> Hi All: >> >> I am have Hadoop 0.20.2 and I am using cygwin on Windows 7. I >> modified the files as shown below for the Hadoop configuration. >> >> conf/core-site.xml: >> >> <configuration> >> <property> >> <name>fs.default.name</name> >> <value>hdfs://localhost:9100</value> >> </property> >> </configuration> >> >> >> conf/hdfs-site.xml: >> >> <configuration> >> <property> >> <name>dfs.replication</name> >> <value>1</value> >> </property> >> </configuration> >> >> >> conf/mapred-site.xml: >> >> <configuration> >> <property> >> <name>mapred.job.tracker</name> >> <value>localhost:9101</value> >> </property> >> </configuration> >> >> Then I have the PATH variable with >> $PATH:/cygdrive/c/cygwin/bin:/cygdrive/c/cygwin/usr/bin >> >> I added JAVA_HOME to the file in cygwin\home\Williams\hadoop- >> 0.20.2\conf\hadoop-env.sh. >> My Java home is now at C:\Java\jdk1.6.0_26 so there is not space. I >> also turned off my firewall. >> However, I get the error from the command line: >> >> <CODE> >> Williams@TWilliams-LTPC ~ >> $ pwd >> /home/Williams >> >> Williams@TWilliams-LTPC ~ >> $ cd hadoop-0.20.2 >> >> Williams@TWilliams-LTPC ~/hadoop-0.20.2 >> $ bin/start-all.sh >> starting namenode, logging to /home/Williams/hadoop- >> 0.20.2/bin/../logs/hadoop-Wi >> lliams-namenode-TWilliams-LTPC.out >> localhost: starting datanode, logging to /home/Williams/hadoop- >> 0.20.2/bin/../logs/hadoop-Williams-datanode-TWilliams-LTPC.out >> localhost: starting secondarynamenode, logging to >> /home/Williams/hadoop-0.20.2/b >> in/../logs/hadoop-Williams-secondarynamenode-TWilliams-LTPC.out >> starting jobtracker, logging to /home/Williams/hadoop- >> 0.20.2/bin/../logs/hadoop- >> Williams-jobtracker-TWilliams-LTPC.out >> localhost: starting tasktracker, logging to /home/Williams/hadoop- >> 0.20.2/bin/../logs/hadoop-Williams-tasktracker-TWilliams-LTPC.out >> >> Williams@TWilliams-LTPC ~/hadoop-0.20.2 >> $ bin/hadoop fs -put conf input >> 11/07/27 17:11:28 INFO ipc.Client: Retrying connect to server: >> localhost/127.0.0.1:9100. Already tried 0 time(s). >> 11/07/27 17:11:30 INFO ipc.Client: Retrying connect to server: >> localhost/127.0.0.1:9100. Already tried 1 time(s). >> 11/07/27 17:11:32 INFO ipc.Client: Retrying connect to server: >> localhost/127.0.0.1:9100. Already tried 2 time(s). >> 11/07/27 17:11:34 INFO ipc.Client: Retrying connect to server: >> localhost/127.0.0.1:9100. Already tried 3 time(s). >> 11/07/27 17:11:36 INFO ipc.Client: Retrying connect to server: >> localhost/127.0.0.1:9100. Already tried 4 time(s). >> 11/07/27 17:11:38 INFO ipc.Client: Retrying connect to server: >> localhost/127.0.0.1:9100. Already tried 5 time(s). >> 11/07/27 17:11:40 INFO ipc.Client: Retrying connect to server: >> localhost/127.0.0.1:9100. Already tried 6 time(s). >> 11/07/27 17:11:43 INFO ipc.Client: Retrying connect to server: >> localhost/127.0.0.1:9100. Already tried 7 time(s). >> 11/07/27 17:11:45 INFO ipc.Client: Retrying connect to server: >> localhost/127.0.0.1:9100. Already tried 8 time(s). >> 11/07/27 17:11:47 INFO ipc.Client: Retrying connect to server: >> localhost/127.0.0.1:9100. Already tried 9 time(s). >> Bad connection to FS. command aborted. >> >> Williams@TWilliams-LTPC ~/hadoop-0.20.2 >> $ bin/hadoop fs -put conf input >> 11/07/27 17:17:29 INFO ipc.Client: Retrying connect to server: >> localhost/127.0.0.1:9100. Already tried 0 time(s). >> 11/07/27 17:17:31 INFO ipc.Client: Retrying connect to server: >> localhost/127.0.0.1:9100. Already tried 1 time(s). >> 11/07/27 17:17:33 INFO ipc.Client: Retrying connect to server: >> localhost/127.0.0.1:9100. Already tried 2 time(s). >> 11/07/27 17:17:35 INFO ipc.Client: Retrying connect to server: >> localhost/127.0.0.1:9100. Already tried 3 time(s). >> 11/07/27 17:17:37 INFO ipc.Client: Retrying connect to server: >> localhost/127.0.0.1:9100. Already tried 4 time(s). >> 11/07/27 17:17:39 INFO ipc.Client: Retrying connect to server: >> localhost/127.0.0.1:9100. Already tried 5 time(s). >> 11/07/27 17:17:41 INFO ipc.Client: Retrying connect to server: >> localhost/127.0.0.1:9100. Already tried 6 time(s). >> 11/07/27 17:17:44 INFO ipc.Client: Retrying connect to server: >> localhost/127.0.0.1:9100. Already tried 7 time(s). >> 11/07/27 17:17:46 INFO ipc.Client: Retrying connect to server: >> localhost/127.0.0.1:9100. Already tried 8 time(s). >> 11/07/27 17:17:48 INFO ipc.Client: Retrying connect to server: >> localhost/127.0.0.1:9100. Already tried 9 time(s). >> Bad connection to FS. command aborted. >> >> Williams@TWilliams-LTPC ~/hadoop-0.20.2 >> $ ping 127.0.0.1:9100 >> Ping request could not find host 127.0.0.1:9100. Please check the >> name and try a >> gain. >> </CODE> >> >> I am not sure why the ip address seem to have localhost/127.0.0.1 >> which seems to be repeating itself. The conf files are fine. I also >> know that when Hadoop is running there is a web interface to check >> but do the default ones work from cygwin which are: >> * NameNode - http://localhost:50070/ >> * JobTracker - http://localhost:50030/ >> >> I wanted to give the cygwin a try once more before just switching >> to a cloudera hadoop vmware. I was hoping that it would not have so >> many problems just to get it working on Windows! Thanks again. >> >> Cheers, >> A Df > > >
