Le 3/23/12 8:50 AM, Manish Bhoge a écrit : > I have Hadoop running on Standalone box. When I am starting deamon for > namenode, secondarynamenode, job tracker, task tracker and data node, it is > starting gracefully. But soon after it start job tracker it doesn't > show up job tracker service. when i run 'jps' it is showing me all the > services including task tracker except Job Tracker. > > Is there any time limit that need to set up or is it going into the safe > mode. Because when i saw job tracker log this what it is showing, looks > like it is starting the namenode but soon after it shutdown: > > 2012-03-22 23:26:04,061 INFO org.apache.hadoop.mapred.JobTracker: > STARTUP_MSG: > /************************************************************ > STARTUP_MSG: Starting JobTracker > STARTUP_MSG: host = manish/10.131.18.119 > STARTUP_MSG: args = [] > STARTUP_MSG: version = 0.20.2-cdh3u3 > STARTUP_MSG: build = > file:///data/1/tmp/nightly_2012-02-16_09-46-24_3/hadoop-0.20-0.20.2+923.195-1~maverick > -r 217a3767c48ad11d4632e19a22897677268c40c4; compiled by 'root' on Thu Feb > 16 10:22:53 PST 2012 > ************************************************************/ > 2012-03-22 23:26:04,140 INFO > org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: > Updating the current master key for generating delegation tokens > 2012-03-22 23:26:04,141 INFO > org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: > Starting expired delegation token remover thread, > tokenRemoverScanInterval=60 min(s) > 2012-03-22 23:26:04,141 INFO > org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: > Updating the current master key for generating delegation tokens > 2012-03-22 23:26:04,142 INFO org.apache.hadoop.mapred.JobTracker: Scheduler > configured with (memSizeForMapSlotOnJT, memSizeForReduceSlotOnJT, > limitMaxMemForMapTasks, limitMaxMemForReduceTasks) (-1, -1, -1, -1) > 2012-03-22 23:26:04,143 INFO org.apache.hadoop.util.HostsFileReader: > Refreshing hosts (include/exclude) list > 2012-03-22 23:26:04,186 INFO org.apache.hadoop.mapred.JobTracker: Starting > jobtracker with owner as mapred > 2012-03-22 23:26:04,201 INFO org.apache.hadoop.ipc.Server: Starting Socket > Reader #1 for port 54311 > 2012-03-22 23:26:04,203 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: > Initializing RPC Metrics with hostName=JobTracker, port=54311 > 2012-03-22 23:26:04,206 INFO > org.apache.hadoop.ipc.metrics.RpcDetailedMetrics: Initializing RPC Metrics > with hostName=JobTracker, port=54311 > 2012-03-22 23:26:09,250 INFO org.mortbay.log: Logging to > org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via > org.mortbay.log.Slf4jLog > 2012-03-22 23:26:09,298 INFO org.apache.hadoop.http.HttpServer: Added global > filtersafety (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter) > 2012-03-22 23:26:09,318 INFO org.apache.hadoop.http.HttpServer: Port returned > by webServer.getConnectors()[0].getLocalPort() before open() is -1. Opening > the listener on 50030 > 2012-03-22 23:26:09,318 INFO org.apache.hadoop.http.HttpServer: > listener.getLocalPort() returned 50030 > webServer.getConnectors()[0].getLocalPort() returned 50030 > 2012-03-22 23:26:09,318 INFO org.apache.hadoop.http.HttpServer: Jetty bound > to port 50030 > 2012-03-22 23:26:09,319 INFO org.mortbay.log: jetty-6.1.26.cloudera.1 > 2012-03-22 23:26:09,517 INFO org.mortbay.log: Started > SelectChannelConnector@0.0.0.0:50030 > 2012-03-22 23:26:09,519 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: > Initializing JVM Metrics with processName=JobTracker, sessionId= > 2012-03-22 23:26:09,519 INFO org.apache.hadoop.mapred.JobTracker: JobTracker > up at: 54311 > 2012-03-22 23:26:09,519 INFO org.apache.hadoop.mapred.JobTracker: JobTracker > webserver: 50030 > 2012-03-22 23:26:09,648 WARN org.apache.hadoop.mapred.JobTracker: Failed to > operate on mapred.system.dir > (hdfs://localhost:54310/app/hadoop/tmp/mapred/system) because of permissions. > 2012-03-22 23:26:09,648 WARN org.apache.hadoop.mapred.JobTracker: This > directory should be owned by the user 'mapred (auth:SIMPLE)' > 2012-03-22 23:26:09,650 WARN org.apache.hadoop.mapred.JobTracker: Bailing out > ... > org.apache.hadoop.security.AccessControlException: The systemdir > hdfs://localhost:54310/app/hadoop/tmp/mapred/system is not owned by mapred at > org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:2241) at > org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:2050) at > org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:296) at > org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:288) at > org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:4792) > 2012-03-22 23:26:09,652 FATAL org.apache.hadoop.mapred.JobTracker: > org.apache.hadoop.security.AccessControlException: The systemdir > hdfs://localhost:54310/app/hadoop/tmp/mapred/system is not owned by mapred at > org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:2241) at > org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:2050) at > org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:296) at > org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:288) at > org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:4792) 2012-03-22 > 23:26:09,654 INFO org.apache.hadoop.mapred.JobTracker: SHUTDOWN_MSG: > /************************************************************ you seem to have dfs access right issues. Debian/Ubuntu packages create 2 users mapred and hdfs, you certainly have the same. A hdfs chmod in you hdfs dir for mapred should solve your issue : chown to mapred or add write access to hadoop group as mapred and hdfs user are both in hadoop group
Olivier > SHUTDOWN_MSG: Shutting down JobTracker at manish/10.131.18.119 > ************************************************************/ -- gpg key id: 4096R/326D8438 (keyring.debian.org) Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438
