Hi Ravi, Yes , it Running. Here is the output:- rn13067@WSUSJXLHRN13067 /home/hadoop-1.0.3 $ jps 5068 NameNode 5836 Jps 3516 JobTracker
Here are the logs from JOBTRACKER:- 2012-05-17 21:41:31,772 INFO org.apache.hadoop.mapred.TaskTracker: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting TaskTracker STARTUP_MSG: host = WSUSJXLHRN13067/192.168.0.16 STARTUP_MSG: args = [] STARTUP_MSG: version = 1.0.3 STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r 1335192; compiled by 'hortonfo' on Tue May 8 20:31:25 UTC 2012 ************************************************************/ 2012-05-17 21:41:31,944 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties 2012-05-17 21:41:31,990 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source MetricsSystem,sub=Stats registered. 2012-05-17 21:41:31,990 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s). 2012-05-17 21:41:31,990 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: TaskTracker metrics system started 2012-05-17 21:41:32,256 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ugi registered. 2012-05-17 21:41:32,256 WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already exists! 2012-05-17 21:41:32,365 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2012-05-17 21:41:32,412 INFO org.apache.hadoop.http.HttpServer: Added global filtersafety (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter) 2012-05-17 21:41:32,428 INFO org.apache.hadoop.mapred.TaskLogsTruncater: Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1 2012-05-17 21:41:32,444 INFO org.apache.hadoop.mapred.TaskTracker: Starting tasktracker with owner as SYSTEM 2012-05-17 21:41:32,444 INFO org.apache.hadoop.mapred.TaskTracker: Good mapred local directories are: /tmp/hadoop-SYSTEM/mapred/local 2012-05-17 21:41:32,459 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2012-05-17 21:41:32,459 ERROR org.apache.hadoop.mapred.TaskTracker: Can not start task tracker because java.io.IOException: Failed to set permissions of path: \tmp\hadoop-SYSTEM\mapred\local\ttprivate to 0700 at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:689) at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:662) at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509) at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:344) at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189) at org.apache.hadoop.mapred.TaskTracker.initialize(TaskTracker.java:728) at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:1459) at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:3742) 2012-05-17 21:41:32,459 INFO org.apache.hadoop.mapred.TaskTracker: SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down TaskTracker at WSUSJXLHRN13067/192.168.0.16 ************************************************************/ Any clue? Thanks Regards, ravi On Fri, May 18, 2012 at 12:01 AM, Ravi Prakash <ravihad...@gmail.com> wrote: > Ravishankar, > > If you run $ jps, do you see a TaskTracker process running? Can you please > post the tasktracker logs as well? > > On Thu, May 17, 2012 at 8:49 PM, Ravishankar Nair < > ravishankar.n...@gmail.com> wrote: > > > Dear experts, > > > > Today is my tenth day working with Hadoop on installing on my windows > > machine. I am trying again and again because , some where someone has > > written that it works on Windows with CYGWIN.(And noone has written that > > Hadoop wont work on Windows). I am attaching my config files. > > > > Kindly help me, if anything can make this work.A feeble and humble > request > > to all experts out there. > > > > Here is the error, if you search , you can see thousands have reported > > this and there is no solution I got yet, though I tried all ways > possible. > > I am using Windows XP SP3, Hadoop (tried with five versions so far > > including 1.0.3). I am running on a single node.(machine WSUSJXLHRN13067 > > IP:- 192.168.0.16) > > When I start Hadoop, no issues in any of the versions > > > > rn13067@WSUSJXLHRN13067 /home/hadoop-1.0.3 > > $ bin/start-all.sh > > starting namenode, logging to > > > /home/hadoop-1.0.3/libexec/../logs/hadoop-SUNDOOP-namenode-WSUSJXLHRN13067.out > > localhost: starting datanode, logging to > > > /home/hadoop-1.0.3/libexec/../logs/hadoop-SUNDOOP-datanode-WSUSJXLHRN13067.out > > localhost: starting secondarynamenode, logging to > > > /home/hadoop-1.0.3/libexec/../logs/hadoop-SUNDOOP-secondarynamenode-WSUSJXLHRN13067.out > > starting jobtracker, logging to > > > /home/hadoop-1.0.3/libexec/../logs/hadoop-SUNDOOP-jobtracker-WSUSJXLHRN13067.out > > localhost: starting tasktracker, logging to > > > /home/hadoop-1.0.3/libexec/../logs/hadoop-SUNDOOP-tasktracker-WSUSJXLHRN13067.out > > > > > > > > When I run the example program, this is what is printed on my console:- > > $ bin/hadoop jar hadoop-examples-1.0.3.jar grep input output > 'dfs[a-z.]+' > > 12/05/17 21:44:46 WARN util.NativeCodeLoader: Unable to load > native-hadoop > > library for your platform... using builtin-java classes where applicable > > 12/05/17 21:44:46 WARN snappy.LoadSnappy: Snappy native library not > loaded > > 12/05/17 21:44:46 INFO mapred.FileInputFormat: Total input paths to > > process : 16 > > 12/05/17 21:44:47 INFO mapred.JobClient: Running job: > job_201205172141_0001 > > 12/05/17 21:44:48 INFO mapred.JobClient: map 0% reduce 0% > > > > > > Now it is HUNG!!. IN most of the versions this is the behaviour. > > > > Here is the log from JOBTRACKER:- > > > > > > 2012-05-17 21:41:28,037 INFO org.apache.hadoop.mapred.JobTracker: > > STARTUP_MSG: > > /************************************************************ > > STARTUP_MSG: Starting JobTracker > > STARTUP_MSG: host = WSUSJXLHRN13067/192.168.0.16 > > STARTUP_MSG: args = [] > > STARTUP_MSG: version = 1.0.3 > > STARTUP_MSG: build = > > https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r > > 1335192; compiled by 'hortonfo' on Tue May 8 20:31:25 UTC 2012 > > ************************************************************/ > > 2012-05-17 21:41:28,147 INFO > > org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from > > hadoop-metrics2.properties > > 2012-05-17 21:41:28,147 INFO > > org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source > > MetricsSystem,sub=Stats registered. > > 2012-05-17 21:41:28,162 INFO > > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot > > period at 10 second(s). > > 2012-05-17 21:41:28,162 INFO > > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: JobTracker metrics > > system started > > 2012-05-17 21:41:28,209 INFO > > org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source > > QueueMetrics,q=default registered. > > 2012-05-17 21:41:28,428 INFO > > org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source > ugi > > registered. > > 2012-05-17 21:41:28,428 WARN > > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi > already > > exists! > > 2012-05-17 21:41:28,428 INFO > > > org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: > > Updating the current master key for generating delegation tokens > > 2012-05-17 21:41:28,428 INFO > > > org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: > > Starting expired delegation token remover thread, > > tokenRemoverScanInterval=60 min(s) > > 2012-05-17 21:41:28,428 INFO > > > org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: > > Updating the current master key for generating delegation tokens > > 2012-05-17 21:41:28,428 INFO org.apache.hadoop.mapred.JobTracker: > > Scheduler configured with (memSizeForMapSlotOnJT, > memSizeForReduceSlotOnJT, > > limitMaxMemForMapTasks, limitMaxMemForReduceTasks) (-1, -1, -1, -1) > > 2012-05-17 21:41:28,428 INFO org.apache.hadoop.util.HostsFileReader: > > Refreshing hosts (include/exclude) list > > 2012-05-17 21:41:28,444 INFO org.apache.hadoop.mapred.JobTracker: > Starting > > jobtracker with owner as rn13067 > > 2012-05-17 21:41:28,475 INFO org.apache.hadoop.ipc.Server: Starting > > SocketReader > > 2012-05-17 21:41:28,475 INFO > > org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source > > RpcDetailedActivityForPort47111 registered. > > 2012-05-17 21:41:28,475 INFO > > org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source > > RpcActivityForPort47111 registered. > > 2012-05-17 21:41:28,522 INFO org.mortbay.log: Logging to > > org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via > > org.mortbay.log.Slf4jLog > > 2012-05-17 21:41:28,584 INFO org.apache.hadoop.http.HttpServer: Added > > global filtersafety > > (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter) > > 2012-05-17 21:41:28,615 WARN org.apache.hadoop.util.NativeCodeLoader: > > Unable to load native-hadoop library for your platform... using > > builtin-java classes where applicable > > 2012-05-17 21:41:28,615 INFO org.apache.hadoop.http.HttpServer: Port > > returned by webServer.getConnectors()[0].getLocalPort() before open() is > > -1. Opening the listener on 50030 > > 2012-05-17 21:41:28,615 INFO org.apache.hadoop.http.HttpServer: > > listener.getLocalPort() returned 50030 > > webServer.getConnectors()[0].getLocalPort() returned 50030 > > 2012-05-17 21:41:28,615 INFO org.apache.hadoop.http.HttpServer: Jetty > > bound to port 50030 > > 2012-05-17 21:41:28,615 INFO org.mortbay.log: jetty-6.1.26 > > 2012-05-17 21:41:28,834 INFO org.mortbay.log: Started > > SelectChannelConnector@0.0.0.0:50030 > > 2012-05-17 21:41:28,834 INFO > > org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source > jvm > > registered. > > 2012-05-17 21:41:28,834 INFO > > org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source > > JobTrackerMetrics registered. > > 2012-05-17 21:41:28,850 INFO org.apache.hadoop.mapred.JobTracker: > > JobTracker up at: 47111 > > 2012-05-17 21:41:28,850 INFO org.apache.hadoop.mapred.JobTracker: > > JobTracker webserver: 50030 > > 2012-05-17 21:41:29,225 INFO org.apache.hadoop.mapred.JobTracker: > Cleaning > > up the system directory > > 2012-05-17 21:41:29,772 INFO org.apache.hadoop.mapred.JobHistory: > Creating > > DONE folder at file:/C:/cygwin/home/hadoop-1.0.3/logs/history/done > > 2012-05-17 21:41:29,787 INFO org.apache.hadoop.mapred.JobTracker: History > > server being initialized in embedded mode > > 2012-05-17 21:41:29,787 INFO org.apache.hadoop.mapred.JobHistoryServer: > > Started job history server at: localhost:50030 > > 2012-05-17 21:41:29,787 INFO org.apache.hadoop.mapred.JobTracker: Job > > History Server web address: localhost:50030 > > 2012-05-17 21:41:29,787 INFO > > org.apache.hadoop.mapred.CompletedJobStatusStore: Completed job store is > > inactive > > 2012-05-17 21:41:29,990 INFO org.apache.hadoop.mapred.JobTracker: > > Refreshing hosts information > > 2012-05-17 21:41:30,006 INFO org.apache.hadoop.util.HostsFileReader: > > Setting the includes file to > > 2012-05-17 21:41:30,006 INFO org.apache.hadoop.util.HostsFileReader: > > Setting the excludes file to > > 2012-05-17 21:41:30,006 INFO org.apache.hadoop.util.HostsFileReader: > > Refreshing hosts (include/exclude) list > > 2012-05-17 21:41:30,006 INFO org.apache.hadoop.mapred.JobTracker: > > Decommissioning 0 nodes > > 2012-05-17 21:41:30,006 INFO org.apache.hadoop.ipc.Server: IPC Server > > Responder: starting > > 2012-05-17 21:41:30,006 INFO org.apache.hadoop.ipc.Server: IPC Server > > listener on 47111: starting > > 2012-05-17 21:41:30,006 INFO org.apache.hadoop.ipc.Server: IPC Server > > handler 0 on 47111: starting > > 2012-05-17 21:41:30,006 INFO org.apache.hadoop.ipc.Server: IPC Server > > handler 1 on 47111: starting > > 2012-05-17 21:41:30,006 INFO org.apache.hadoop.ipc.Server: IPC Server > > handler 2 on 47111: starting > > 2012-05-17 21:41:30,006 INFO org.apache.hadoop.ipc.Server: IPC Server > > handler 3 on 47111: starting > > 2012-05-17 21:41:30,006 INFO org.apache.hadoop.ipc.Server: IPC Server > > handler 4 on 47111: starting > > 2012-05-17 21:41:30,006 INFO org.apache.hadoop.ipc.Server: IPC Server > > handler 5 on 47111: starting > > 2012-05-17 21:41:30,006 INFO org.apache.hadoop.ipc.Server: IPC Server > > handler 6 on 47111: starting > > 2012-05-17 21:41:30,006 INFO org.apache.hadoop.ipc.Server: IPC Server > > handler 7 on 47111: starting > > 2012-05-17 21:41:30,006 INFO org.apache.hadoop.mapred.JobTracker: > Starting > > RUNNING > > 2012-05-17 21:41:30,006 INFO org.apache.hadoop.ipc.Server: IPC Server > > handler 8 on 47111: starting > > 2012-05-17 21:41:30,006 INFO org.apache.hadoop.ipc.Server: IPC Server > > handler 9 on 47111: starting > > 2012-05-17 21:44:47,069 INFO org.apache.hadoop.mapred.JobInProgress: > > job_201205172141_0001: nMaps=16 nReduces=1 max=-1 > > 2012-05-17 21:44:47,069 INFO org.apache.hadoop.mapred.JobTracker: Job > > job_201205172141_0001 added successfully for user 'rn13067' to queue > > 'default' > > 2012-05-17 21:44:47,069 INFO org.apache.hadoop.mapred.JobTracker: > > Initializing job_201205172141_0001 > > 2012-05-17 21:44:47,069 INFO org.apache.hadoop.mapred.JobInProgress: > > Initializing job_201205172141_0001 > > 2012-05-17 21:44:47,069 INFO org.apache.hadoop.mapred.AuditLogger: > > USER=rn13067 IP=192.168.0.16 OPERATION=SUBMIT_JOB > > TARGET=job_201205172141_0001 RESULT=SUCCESS > > *2012-05-17 21:44:47,084 ERROR org.apache.hadoop.mapred.JobHistory: > > Failed creating job history log file for job job_201205172141_0001 > > java.io.IOException: Failed to set permissions of path: > > > C:\cygwin\home\hadoop-1.0.3\logs\history\job_201205172141_0001_1337305487022_rn13067_grep-search > > to 0744* > > at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:689) > > at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:678) > > at > > > org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509) > > at > > > org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:286) > > at > > > org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:385) > > at > > > org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:364) > > at > > > org.apache.hadoop.mapred.JobHistory$JobInfo.logSubmitted(JobHistory.java:1696) > > at > org.apache.hadoop.mapred.JobInProgress$3.run(JobInProgress.java:681) > > at java.security.AccessController.doPrivileged(Native Method) > > at javax.security.auth.Subject.doAs(Subject.java:396) > > at > > > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121) > > at > > org.apache.hadoop.mapred.JobInProgress.initTasks(JobInProgress.java:678) > > at org.apache.hadoop.mapred.JobTracker.initJob(JobTracker.java:4207) > > at > > > org.apache.hadoop.mapred.EagerTaskInitializationListener$InitJob.run(EagerTaskInitializationListener.java:79) > > at > > > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > > at > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > > at java.lang.Thread.run(Thread.java:662) > > > > Kindly help me. CYGWIN is in the path. (As some one suggested in some > > thread). > > > > > > > > > > -- > > Warmest Regards, > > > > Ravi > > > -- Warmest Regards, Ravi Shankar
