Hey all,
I'm having trouble starting Zeppelin in yarn mode. I am getting the
following error:
INFO [2015-06-08 16:30:09,404] ({pool-2-thread-4}
Server.java[doStart]:272) - jetty-8.y.z-SNAPSHOT
INFO [2015-06-08 16:30:09,410] ({pool-2-thread-4}
AbstractConnector.java[doStart]:338) - Started
SelectChannelConnector@0.0.0.0:4048
INFO [2015-06-08 16:30:09,410] ({pool-2-thread-4}
Logging.scala[logInfo]:59) - Successfully started service 'SparkUI' on port
4048.
INFO [2015-06-08 16:30:09,411] ({pool-2-thread-4}
Logging.scala[logInfo]:59) - Started SparkUI at
http://CLUSTER.COMPANY.com:4048
INFO [2015-06-08 16:30:09,511] ({pool-2-thread-4}
Logging.scala[logInfo]:59) - Created default pool default, schedulingMode:
FIFO, minShare: 0, weight: 1
INFO [2015-06-08 16:30:09,664] ({pool-2-thread-4}
RMProxy.java[createRMProxy]:56) - Connecting to ResourceManager at
CLUSTER.COMPANY.com/9.1.70.134:18040
INFO [2015-06-08 16:30:09,666] ({pool-2-thread-4}
Logging.scala[logInfo]:59) - Requesting a new application from cluster with
6 NodeManagers
INFO [2015-06-08 16:30:09,667] ({pool-2-thread-4}
Logging.scala[logInfo]:59) - Verifying our application has not requested
more than the maximum memory capability of the cluster (42124 MB per
container)
INFO [2015-06-08 16:30:09,668] ({pool-2-thread-4}
Logging.scala[logInfo]:59) - Will allocate AM container, with 896 MB memory
including 384 MB overhead
INFO [2015-06-08 16:30:09,668] ({pool-2-thread-4}
Logging.scala[logInfo]:59) - Setting up container launch context for our AM
INFO [2015-06-08 16:30:09,668] ({pool-2-thread-4}
Logging.scala[logInfo]:59) - Preparing resources for our AM container
INFO [2015-06-08 16:30:09,758] ({pool-2-thread-4}
Logging.scala[logInfo]:59) - Uploading resource
file:/home/biadmin/incubator-zeppelin/interpreter/spark/zeppelin-spark-0.5.0-incubating-SNAPSHOT.jar
-> hdfs://
CLUSTER.COMPANY.com:9000/user/biadmin/.sparkStaging/application_1427331872947_9732/zeppelin-spark-0.5.0-incubating-SNAPSHOT.jar
INFO [2015-06-08 16:30:23,876] ({pool-2-thread-4}
Logging.scala[logInfo]:59) - Setting up the launch environment for our AM
container
INFO [2015-06-08 16:30:23,881] ({pool-2-thread-4}
Logging.scala[logInfo]:59) - Changing view acls to: biadmin
INFO [2015-06-08 16:30:23,881] ({pool-2-thread-4}
Logging.scala[logInfo]:59) - Changing modify acls to: biadmin
INFO [2015-06-08 16:30:23,882] ({pool-2-thread-4}
Logging.scala[logInfo]:59) - SecurityManager: authentication disabled; ui
acls disabled; users with view permissions: Set(biadmin); users with modify
permissions: Set(biadmin)
INFO [2015-06-08 16:30:23,882] ({pool-2-thread-4}
Logging.scala[logInfo]:59) - Submitting application 9732 to ResourceManager
INFO [2015-06-08 16:30:23,920] ({pool-2-thread-4}
YarnClientImpl.java[submitApplication]:174) - Submitted application
application_1427331872947_9732 to ResourceManager at
CLUSTER.COMPANY.com/9.1.70.134:18040
INFO [2015-06-08 16:30:24,922] ({pool-2-thread-4}
Logging.scala[logInfo]:59) - Application report for
application_1427331872947_9732 (state: ACCEPTED)
INFO [2015-06-08 16:30:24,923] ({pool-2-thread-4}
Logging.scala[logInfo]:59) -
client token: N/A
diagnostics: N/A
ApplicationMaster host: N/A
ApplicationMaster RPC port: -1
queue: default
start time: 1433806223885
final status: UNDEFINED
tracking URL:
http://CLUSTER.COMPANY.com:18088/proxy/application_1427331872947_9732/
user: biadmin
INFO [2015-06-08 16:30:25,925] ({pool-2-thread-4}
Logging.scala[logInfo]:59) - Application report for
application_1427331872947_9732 (state: ACCEPTED)
INFO [2015-06-08 16:30:26,927] ({pool-2-thread-4}
Logging.scala[logInfo]:59) - Application report for
application_1427331872947_9732 (state: ACCEPTED)
INFO [2015-06-08 16:30:27,929] ({pool-2-thread-4}
Logging.scala[logInfo]:59) - Application report for
application_1427331872947_9732 (state: ACCEPTED)
INFO [2015-06-08 16:30:28,931] ({pool-2-thread-4}
Logging.scala[logInfo]:59) - Application report for
application_1427331872947_9732 (state: ACCEPTED)
INFO [2015-06-08 16:30:29,933] ({pool-2-thread-4}
Logging.scala[logInfo]:59) - Application report for
application_1427331872947_9732 (state: ACCEPTED)
INFO [2015-06-08 16:30:30,935] ({pool-2-thread-4}
Logging.scala[logInfo]:59) - Application report for
application_1427331872947_9732 (state: ACCEPTED)
INFO [2015-06-08 16:30:31,937] ({pool-2-thread-4}
Logging.scala[logInfo]:59) - Application report for
application_1427331872947_9732 (state: ACCEPTED)
INFO [2015-06-08 16:30:32,939] ({pool-2-thread-4}
Logging.scala[logInfo]:59) - Application report for
application_1427331872947_9732 (state: ACCEPTED)
INFO [2015-06-08 16:30:33,941] ({pool-2-thread-4}
Logging.scala[logInfo]:59) - Application report for
application_1427331872947_9732 (state: ACCEPTED)
INFO [2015-06-08 16:30:34,944] ({pool-2-thread-4}
Logging.scala[logInfo]:59) - Application report for
application_1427331872947_9732 (state: ACCEPTED)
INFO [2015-06-08 16:30:35,946] ({pool-2-thread-4}
Logging.scala[logInfo]:59) - Application report for
application_1427331872947_9732 (state: ACCEPTED)
INFO [2015-06-08 16:30:36,948] ({pool-2-thread-4}
Logging.scala[logInfo]:59) - Application report for
application_1427331872947_9732 (state: ACCEPTED)
INFO [2015-06-08 16:30:37,951] ({pool-2-thread-4}
Logging.scala[logInfo]:59) - Application report for
application_1427331872947_9732 (state: ACCEPTED)
INFO [2015-06-08 16:30:38,648]
({sparkDriver-akka.actor.default-dispatcher-2} Logging.scala[logInfo]:59) -
ApplicationMaster registered as Actor[akka.tcp://
sparkyar...@xray.cluster.company.com:39372/user/YarnAM#946706347]
WARN [2015-06-08 16:30:38,679]
({sparkDriver-akka.actor.default-dispatcher-2}
Slf4jLogger.scala[apply$mcV$sp]:71) - Association with remote system
[akka.tcp://sparkyar...@xray.cluster.company.com:39372] has failed, address
is now gated for [5000] ms. Reason is: [Disassociated].
INFO [2015-06-08 16:30:38,953] ({pool-2-thread-4}
Logging.scala[logInfo]:59) - Application report for
application_1427331872947_9732 (state: ACCEPTED)
INFO [2015-06-08 16:30:39,955] ({pool-2-thread-4}
Logging.scala[logInfo]:59) - Application report for
application_1427331872947_9732 (state: FAILED)
INFO [2015-06-08 16:30:39,956] ({pool-2-thread-4}
Logging.scala[logInfo]:59) -
client token: N/A
diagnostics: Application application_1427331872947_9732 failed 2
times due to AM Container for appattempt_1427331872947_9732_000002 exited
with exitCode: 1 due to: Exception from container-launch:
org.apache.hadoop.util.Shell$ExitCodeException:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:485)
at org.apache.hadoop.util.Shell.run(Shell.java:398)
at
org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:630)
at
org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:279)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
at java.util.concurrent.FutureTask.run(FutureTask.java:273)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1176)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at java.lang.Thread.run(Thread.java:853)
main : command provided 1
main : user is yarn
main : requested yarn user is biadmin
Container exited with a non-zero exit code 1
.Failing this attempt.. Failing the application.
ApplicationMaster host: N/A
ApplicationMaster RPC port: -1
queue: default
start time: 1433806223885
final status: FAILED
tracking URL:
http://CLUSTER.COMPANY.com:18088/cluster/app/application_1427331872947_9732
user: biadmin
how can I make it, so that the programm is run as the user "biadmin" ?
