Hi All,
I was trying to run one of the bundled oozie examples on MapR Sandbox but
getting the following error:
JA009 JA009: Cannot initialize Cluster. Please check your configuration for
mapreduce.framework.name and the correspond server addresses.
The /opt/mapr/oozie/oozie-<version>/examples/apps/map-reduce/job.properties is:
nameNode=maprfs:///
jobTracker=maprfs:///
queueName=default
examplesRoot=examples
oozie.wf.application.path=maprfs:/user/${user.name}/${examplesRoot}/apps/map-reduce
outputDir=map-reduce
Referred: http://doc.mapr.com/display/MapR/Setup+and+Run+Oozie+Examples
Later on I changed the following in the
/opt/mapr/oozie/oozie-<version>/examples/apps/map-reduce/job.properties
JobTracker=<ResourceManager_address>:8032
But now getting this error:
2016-08-14 13:56:17,935 INFO [main]
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for
application appattempt_1471207888052_0001_000001
2016-08-14 13:56:21,191 WARN [main] org.apache.hadoop.conf.Configuration:
job.xml:an attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
2016-08-14 13:56:21,204 WARN [main] org.apache.hadoop.conf.Configuration:
job.xml:an attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
2016-08-14 13:56:21,802 INFO [main]
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Executing with tokens:
2016-08-14 13:56:21,802 INFO [main]
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Kind: YARN_AM_RM_TOKEN,
Service: , Ident: (org.apache.hadoop.yarn.security.AMRMTokenIdentifier@5e476e34)
2016-08-14 13:56:22,099 INFO [main]
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Kind: RM_DELEGATION_TOKEN,
Service: 192.168.60.128:8032, Ident: (owner=mapr, renewer=oozie mr token,
realUser=mapr, issueDate=1471208157557, maxDate=1471812957557,
sequenceNumber=2, masterKeyId=14)
2016-08-14 13:56:22,196 INFO [main]
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: The specific max attempts: 2
for application: 1. Attempt num: 1 is last retry: false
2016-08-14 13:56:22,533 WARN [main] org.apache.hadoop.conf.Configuration:
job.xml:an attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
2016-08-14 13:56:22,534 WARN [main] org.apache.hadoop.conf.Configuration:
job.xml:an attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
2016-08-14 13:56:22,641 INFO [main]
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter set in config
null
2016-08-14 13:56:22,645 INFO [main]
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter is
org.apache.hadoop.mapred.FileOutputCommitter
2016-08-14 13:56:22,874 INFO [main]
org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
org.apache.hadoop.mapreduce.jobhistory.EventType for class
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler
2016-08-14 13:56:22,876 INFO [main]
org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
org.apache.hadoop.mapreduce.v2.app.job.event.JobEventType for class
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher
2016-08-14 13:56:22,878 INFO [main]
org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
org.apache.hadoop.mapreduce.v2.app.job.event.TaskEventType for class
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskEventDispatcher
2016-08-14 13:56:22,880 INFO [main]
org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
org.apache.hadoop.mapreduce.v2.app.job.event.TaskAttemptEventType for class
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher
2016-08-14 13:56:22,881 INFO [main]
org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventType for class
org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler
2016-08-14 13:56:22,883 INFO [main]
org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
org.apache.hadoop.mapreduce.v2.app.speculate.Speculator$EventType for class
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$SpeculatorEventDispatcher
2016-08-14 13:56:22,884 INFO [main]
org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
org.apache.hadoop.mapreduce.v2.app.rm.ContainerAllocator$EventType for class
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter
2016-08-14 13:56:22,886 INFO [main]
org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncher$EventType for
class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerLauncherRouter
2016-08-14 13:56:23,060 INFO [main]
org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class
org.apache.hadoop.mapreduce.v2.app.job.event.JobFinishEvent$Type for class
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler
2016-08-14 13:56:24,266 INFO [main]
org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from
hadoop-metrics2.properties
2016-08-14 13:56:24,513 INFO [main]
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at
10 second(s).
2016-08-14 13:56:24,513 INFO [main]
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MRAppMaster metrics system
started
2016-08-14 13:56:24,556 INFO [main]
org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Adding job token for
job_1471207888052_0001 to jobTokenSecretManager
2016-08-14 13:56:24,619 INFO [main]
org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Not uberizing
job_1471207888052_0001 because: not enabled;
2016-08-14 13:56:24,724 INFO [main]
org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Input size for job
job_1471207888052_0001 = 5. Number of splits = 1
2016-08-14 13:56:24,724 INFO [main]
org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Number of reduces for job
job_1471207888052_0001 = 0
2016-08-14 13:56:24,725 INFO [main]
org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1471207888052_0001Job
Transitioned from NEW to INITED
2016-08-14 13:56:24,729 INFO [main]
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: MRAppMaster launching normal,
non-uberized, multi-container job job_1471207888052_0001.
2016-08-14 13:56:24,865 INFO [main] org.apache.hadoop.ipc.CallQueueManager:
Using callQueue class java.util.concurrent.LinkedBlockingQueue
2016-08-14 13:56:24,919 INFO [Socket Reader #1 for port 45515]
org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 45515
2016-08-14 13:56:24,986 INFO [main]
org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl: Adding
protocol org.apache.hadoop.mapreduce.v2.api.MRClientProtocolPB to the server
2016-08-14 13:56:24,987 INFO [IPC Server Responder]
org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2016-08-14 13:56:24,992 INFO [IPC Server listener on 45515]
org.apache.hadoop.ipc.Server: IPC Server listener on 45515: starting
2016-08-14 13:56:25,044 INFO [main]
org.apache.hadoop.mapreduce.v2.app.client.MRClientService: Instantiated
MRClientService at maprdemo/192.168.60.128:45515
2016-08-14 13:56:25,184 INFO [main] org.mortbay.log: Logging to
org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2016-08-14 13:56:25,194 INFO [main] org.apache.hadoop.http.HttpRequestLog: Http
request log for http.requests.mapreduce is not defined
2016-08-14 13:56:25,221 INFO [main] org.apache.hadoop.http.HttpServer2: Added
global filter 'safety'
(class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
2016-08-14 13:56:25,242 INFO [main] org.apache.hadoop.http.HttpServer2: Added
filter AM_PROXY_FILTER
(class=org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter) to context
mapreduce
2016-08-14 13:56:25,243 INFO [main] org.apache.hadoop.http.HttpServer2: Added
filter AM_PROXY_FILTER
(class=org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter) to context
static
2016-08-14 13:56:25,265 INFO [main] org.apache.hadoop.http.HttpServer2: adding
path spec: /mapreduce/*
2016-08-14 13:56:25,265 INFO [main] org.apache.hadoop.http.HttpServer2: adding
path spec: /ws/*
2016-08-14 13:56:25,376 INFO [main] org.apache.hadoop.http.HttpServer2: Jetty
bound to port 43314
2016-08-14 13:56:25,376 INFO [main] org.mortbay.log: jetty-6.1.26
2016-08-14 13:56:25,480 INFO [main] org.mortbay.log: Extract
jar:file:/opt/mapr/hadoop/hadoop-2.5.1/share/hadoop/yarn/hadoop-yarn-common-2.5.1-mapr-1501.jar!/webapps/mapreduce
to /tmp/Jetty_0_0_0_0_43314_mapreduce____9q4g9u/webapp
2016-08-14 13:56:27,161 INFO [main] org.mortbay.log: Started
[email protected]:43314
2016-08-14 13:56:27,161 INFO [main] org.apache.hadoop.yarn.webapp.WebApps: Web
app /mapreduce started at 43314
2016-08-14 13:56:28,659 INFO [main] org.apache.hadoop.yarn.webapp.WebApps:
Registered webapp guice modules
2016-08-14 13:56:28,677 INFO [main] org.apache.hadoop.ipc.CallQueueManager:
Using callQueue class java.util.concurrent.LinkedBlockingQueue
2016-08-14 13:56:28,680 INFO [Socket Reader #1 for port 51062]
org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 51062
2016-08-14 13:56:28,710 INFO [IPC Server Responder]
org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2016-08-14 13:56:28,710 INFO [IPC Server listener on 51062]
org.apache.hadoop.ipc.Server: IPC Server listener on 51062: starting
2016-08-14 13:56:28,783 INFO [main]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor:
nodeBlacklistingEnabled:true
2016-08-14 13:56:28,783 INFO [main]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor:
maxTaskFailuresPerNode is 3
2016-08-14 13:56:28,783 INFO [main]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor:
blacklistDisablePercent is 33
2016-08-14 13:56:28,995 WARN [main] org.apache.hadoop.conf.Configuration:
job.xml:an attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
2016-08-14 13:56:28,996 WARN [main] org.apache.hadoop.conf.Configuration:
job.xml:an attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
2016-08-14 13:56:29,029 INFO [main] org.apache.hadoop.service.AbstractService:
Service RMCommunicator failed in state STARTED; cause:
java.lang.ArrayIndexOutOfBoundsException: 0
java.lang.ArrayIndexOutOfBoundsException: 0
at
org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider.init(ConfiguredRMFailoverProxyProvider.java:62)
at
org.apache.hadoop.yarn.client.RMProxy.createRMFailoverProxyProvider(RMProxy.java:157)
at
org.apache.hadoop.yarn.client.RMProxy.createRMProxy(RMProxy.java:87)
at
org.apache.hadoop.yarn.client.ClientRMProxy.createRMProxy(ClientRMProxy.java:70)
at
org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.createSchedulerProxy(RMCommunicator.java:297)
at
org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.serviceStart(RMCommunicator.java:112)
at
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.serviceStart(RMContainerAllocator.java:229)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter.serviceStart(MRAppMaster.java:818)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1081)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1489)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1566)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1485)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1418)
2016-08-14 13:56:29,037 INFO [main]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Final Stats:
PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:0 AssignedReds:0
CompletedMaps:0 CompletedReds:0 ContAlloc:0 ContRel:0 HostLocal:0 RackLocal:0
2016-08-14 13:56:29,038 INFO [main] org.apache.hadoop.service.AbstractService:
Service org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter
failed in state STARTED; cause: java.lang.ArrayIndexOutOfBoundsException: 0
java.lang.ArrayIndexOutOfBoundsException: 0
at
org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider.init(ConfiguredRMFailoverProxyProvider.java:62)
at
org.apache.hadoop.yarn.client.RMProxy.createRMFailoverProxyProvider(RMProxy.java:157)
at
org.apache.hadoop.yarn.client.RMProxy.createRMProxy(RMProxy.java:87)
at
org.apache.hadoop.yarn.client.ClientRMProxy.createRMProxy(ClientRMProxy.java:70)
at
org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.createSchedulerProxy(RMCommunicator.java:297)
at
org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.serviceStart(RMCommunicator.java:112)
at
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.serviceStart(RMContainerAllocator.java:229)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter.serviceStart(MRAppMaster.java:818)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1081)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1489)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1566)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1485)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1418)
2016-08-14 13:56:29,039 INFO [main] org.apache.hadoop.service.AbstractService:
Service org.apache.hadoop.mapreduce.v2.app.MRAppMaster failed in state STARTED;
cause: java.lang.ArrayIndexOutOfBoundsException: 0
java.lang.ArrayIndexOutOfBoundsException: 0
at
org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider.init(ConfiguredRMFailoverProxyProvider.java:62)
at
org.apache.hadoop.yarn.client.RMProxy.createRMFailoverProxyProvider(RMProxy.java:157)
at
org.apache.hadoop.yarn.client.RMProxy.createRMProxy(RMProxy.java:87)
at
org.apache.hadoop.yarn.client.ClientRMProxy.createRMProxy(ClientRMProxy.java:70)
at
org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.createSchedulerProxy(RMCommunicator.java:297)
at
org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.serviceStart(RMCommunicator.java:112)
at
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.serviceStart(RMContainerAllocator.java:229)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter.serviceStart(MRAppMaster.java:818)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1081)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1489)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1566)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1485)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1418)
My question is what should be the value in
/opt/mapr/oozie/oozie-<version>/examples/apps/map-reduce/job.properties
nameNode=maprfs:///
jobTracker=maprfs:///
Should I use the default ones or need to changes this as I think there in MapR
there is by default HA in NameNode(CLDB) and RM?
Any pointers here will be of great help.
Thanks,
-Nirmal
________________________________
NOTE: This message may contain information that is confidential, proprietary,
privileged or otherwise protected by law. The message is intended solely for
the named addressee. If received in error, please destroy and notify the
sender. Any use of this email is prohibited when received in error. Impetus
does not represent, warrant and/or guarantee, that the integrity of this
communication has been maintained nor that the communication is free of errors,
virus, interception or interference.
