Hi,
I am extending MRQL to support Hama v0.7 on Yarn (see
https://issues.apache.org/jira/browse/MRQL-75 ).
Currently, MRQL on Hama works fine on Mesos but I have problems running
it on Yarn.
1) Without using the PartitioningRunner, the Yarn Application master
crashes.
It seems that the reason is that I have 1 input block (1 split) but
I use 4 tasks.
This may be caused by my input format.
But the Application master shouldn't crash; it should have used 1
task instead.
The log is attached below.
2) If I use the PartitioningRunner using:
job.setPartitioner(org.apache.hama.bsp.HashPartitioner.class);
job.setNumBspTask(4);
job.set("bsp.min.split.size","102");
it fails because it expects a Long key. Here is the log:
15/07/15 09:31:40 INFO bsp.BSPJobClient: Running job: job_localrunner_0001
15/07/15 09:31:42 INFO bsp.LocalBSPRunner: Setting up a new barrier for
4 tasks!
15/07/15 09:31:42 ERROR bsp.LocalBSPRunner: Exception during BSP execution!
java.io.IOException: wrong key class: org.apache.mrql.MRContainer is not
class org.apache.hadoop.io.LongWritable
at
org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:1306)
at
org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:1298)
at
org.apache.hama.bsp.SequenceFileRecordWriter.write(SequenceFileRecordWriter.java:47)
at
org.apache.hama.bsp.SequenceFileRecordWriter.write(SequenceFileRecordWriter.java:31)
at org.apache.hama.bsp.BSPPeerImpl$1.collect(BSPPeerImpl.java:335)
at org.apache.hama.bsp.BSPPeerImpl.write(BSPPeerImpl.java:628)
at
org.apache.hama.bsp.PartitioningRunner.bsp(PartitioningRunner.java:156)
Thanks,
Leonidas Fegaras
15/07/15 09:24:20 INFO bsp.ApplicationMaster: Initializing ApplicationMaster
15/07/15 09:24:23 WARN conf.Configuration:
org.apache.hadoop.hdfs.client.HdfsDataInputStream@f107c50:an attempt to
override final parameter: mapreduce.job.end-notification.max.retry.interval;
Ignoring.
15/07/15 09:24:24 WARN conf.Configuration:
org.apache.hadoop.hdfs.client.HdfsDataInputStream@f107c50:an attempt to
override final parameter: mapreduce.job.end-notification.max.attempts;
Ignoring.
15/07/15 09:24:24 INFO ipc.Server: Starting Socket Reader #1 for port 12000
15/07/15 09:24:24 INFO ipc.Server: IPC Server Responder: starting
15/07/15 09:24:24 INFO ipc.Server: IPC Server listener on 12000: starting
15/07/15 09:24:24 INFO ipc.Server: Starting Socket Reader #1 for port 10000
15/07/15 09:24:24 INFO ipc.Server: IPC Server Responder: starting
15/07/15 09:24:24 INFO ipc.Server: IPC Server handler 0 on 10000: starting
15/07/15 09:24:24 INFO ipc.Server: IPC Server listener on 10000: starting
15/07/15 09:24:24 INFO ipc.Server: IPC Server handler 0 on 12000: starting
15/07/15 09:24:24 INFO bsp.ApplicationMaster: Written new configuration back to
hdfs://localhost:9000/tmp/hadoop-fegaras/bsp/local/submit_lj3a8l/job.xml
15/07/15 09:24:25 INFO bsp.ApplicationMaster: Starting ApplicationMaster
15/07/15 09:24:25 INFO bsp.ApplicationMaster: Executing with tokens:
15/07/15 09:24:25 INFO bsp.ApplicationMaster: Kind: YARN_AM_RM_TOKEN, Service:
, Ident: (appAttemptId { application_id { id: 54 cluster_timestamp:
1436796131530 } attemptId: 1 } keyId: -311344489)
15/07/15 09:24:25 INFO client.RMProxy: Connecting to ResourceManager at
/0.0.0.0:8030
15/07/15 09:24:25 INFO impl.NMClientAsyncImpl: Upper bound of the thread pool
size is 500
15/07/15 09:24:25 INFO impl.ContainerManagementProtocolProxy:
yarn.client.max-cached-nodemanagers-proxies : 0
15/07/15 09:24:25 INFO bsp.ApplicationMaster: Max mem capability of resources
in this cluster 2048
15/07/15 09:24:25 INFO bsp.ApplicationMaster: Max vcores capability of
resources in this cluster 2
15/07/15 09:24:25 INFO bsp.ApplicationMaster:
appattempt_1436796131530_0054_000001 received 0 previous attempts' running
containers on AM registration.
15/07/15 09:24:25 INFO bsp.ApplicationMaster: Requested container ask:
Capability[<memory:100, vCores:1>]Priority[0]
15/07/15 09:24:25 INFO bsp.ApplicationMaster: Requested container ask:
Capability[<memory:100, vCores:1>]Priority[0]
15/07/15 09:24:25 INFO bsp.ApplicationMaster: Requested container ask:
Capability[<memory:100, vCores:1>]Priority[0]
15/07/15 09:24:25 INFO bsp.ApplicationMaster: Requested container ask:
Capability[<memory:100, vCores:1>]Priority[0]
15/07/15 09:24:27 INFO impl.AMRMClientImpl: Received new token for : crete:37056
15/07/15 09:24:27 INFO bsp.ApplicationMaster: Got response from RM for
container ask, allocatedCnt=1
15/07/15 09:24:27 INFO bsp.ApplicationMaster: Launching shell command on a new
container., containerId=container_1436796131530_0054_01_000002,
containerNode=crete:37056, containerNodeURI=crete:8042,
containerResourceMemory128, containerResourceVirtualCores1
15/07/15 09:24:27 INFO bsp.ApplicationMaster: Setting up container launch
container for containerid=container_1436796131530_0054_01_000002
15/07/15 09:24:27 INFO bsp.ApplicationMaster: PackageURL has been composed to
scheme: "hdfs" host: "localhost" port: 9000 file:
"/tmp/hadoop-fegaras/bsp/local/submit_lj3a8l/job.jar"
15/07/15 09:24:27 INFO bsp.ApplicationMaster: Reverting packageURL to path:
hdfs://localhost:9000/tmp/hadoop-fegaras/bsp/local/submit_lj3a8l/job.jar
15/07/15 09:24:27 INFO bsp.ApplicationMaster: Hama release URL has been
composed to scheme: "hdfs" host: "localhost" port: 9000 file:
"/tmp/hadoop-fegaras/bsp/local/hama"
15/07/15 09:24:27 INFO bsp.ApplicationMaster: Starting commands:
[${JAVA_HOME}/bin/java -cp
$CLASSPATH:./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*
org.apache.hama.bsp.BSPRunner appattempt_1436796131530_0054_000001 2
hdfs://localhost:9000/tmp/hadoop-fegaras/bsp/local/submit_lj3a8l/job.xml
1><LOG_DIR>/hama-worker.stdout 2><LOG_DIR>/hama-worker.stderr ]
15/07/15 09:24:27 INFO impl.NMClientAsyncImpl: Processing Event EventType:
START_CONTAINER for Container container_1436796131530_0054_01_000002
15/07/15 09:24:27 INFO impl.ContainerManagementProtocolProxy: Opening proxy :
crete:37056
15/07/15 09:24:27 INFO impl.NMClientAsyncImpl: Processing Event EventType:
QUERY_CONTAINER for Container container_1436796131530_0054_01_000002
15/07/15 09:24:27 INFO impl.ContainerManagementProtocolProxy: Opening proxy :
crete:37056
15/07/15 09:24:28 INFO bsp.ApplicationMaster: Got response from RM for
container ask, allocatedCnt=1
15/07/15 09:24:28 INFO bsp.ApplicationMaster: Launching shell command on a new
container., containerId=container_1436796131530_0054_01_000003,
containerNode=crete:37056, containerNodeURI=crete:8042,
containerResourceMemory128, containerResourceVirtualCores1
15/07/15 09:24:28 INFO bsp.ApplicationMaster: Setting up container launch
container for containerid=container_1436796131530_0054_01_000003
15/07/15 09:24:28 INFO bsp.ApplicationMaster: PackageURL has been composed to
scheme: "hdfs" host: "localhost" port: 9000 file:
"/tmp/hadoop-fegaras/bsp/local/submit_lj3a8l/job.jar"
15/07/15 09:24:28 INFO bsp.ApplicationMaster: Reverting packageURL to path:
hdfs://localhost:9000/tmp/hadoop-fegaras/bsp/local/submit_lj3a8l/job.jar
15/07/15 09:24:28 INFO bsp.ApplicationMaster: Hama release URL has been
composed to scheme: "hdfs" host: "localhost" port: 9000 file:
"/tmp/hadoop-fegaras/bsp/local/hama"
15/07/15 09:24:28 INFO bsp.ApplicationMaster: Starting commands:
[${JAVA_HOME}/bin/java -cp
$CLASSPATH:./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*
org.apache.hama.bsp.BSPRunner appattempt_1436796131530_0054_000001 3
hdfs://localhost:9000/tmp/hadoop-fegaras/bsp/local/submit_lj3a8l/job.xml
1><LOG_DIR>/hama-worker.stdout 2><LOG_DIR>/hama-worker.stderr ]
15/07/15 09:24:28 INFO impl.NMClientAsyncImpl: Processing Event EventType:
START_CONTAINER for Container container_1436796131530_0054_01_000003
15/07/15 09:24:28 INFO impl.ContainerManagementProtocolProxy: Opening proxy :
crete:37056
15/07/15 09:24:28 INFO impl.NMClientAsyncImpl: Processing Event EventType:
QUERY_CONTAINER for Container container_1436796131530_0054_01_000003
15/07/15 09:24:28 INFO impl.ContainerManagementProtocolProxy: Opening proxy :
crete:37056
15/07/15 09:24:29 INFO bsp.ApplicationMaster: Got response from RM for
container ask, allocatedCnt=1
15/07/15 09:24:29 INFO bsp.ApplicationMaster: Launching shell command on a new
container., containerId=container_1436796131530_0054_01_000004,
containerNode=crete:37056, containerNodeURI=crete:8042,
containerResourceMemory128, containerResourceVirtualCores1
15/07/15 09:24:29 INFO bsp.ApplicationMaster: Setting up container launch
container for containerid=container_1436796131530_0054_01_000004
15/07/15 09:24:29 INFO bsp.ApplicationMaster: PackageURL has been composed to
scheme: "hdfs" host: "localhost" port: 9000 file:
"/tmp/hadoop-fegaras/bsp/local/submit_lj3a8l/job.jar"
15/07/15 09:24:29 INFO bsp.ApplicationMaster: Reverting packageURL to path:
hdfs://localhost:9000/tmp/hadoop-fegaras/bsp/local/submit_lj3a8l/job.jar
15/07/15 09:24:29 INFO bsp.ApplicationMaster: Hama release URL has been
composed to scheme: "hdfs" host: "localhost" port: 9000 file:
"/tmp/hadoop-fegaras/bsp/local/hama"
15/07/15 09:24:29 INFO bsp.ApplicationMaster: Starting commands:
[${JAVA_HOME}/bin/java -cp
$CLASSPATH:./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*
org.apache.hama.bsp.BSPRunner appattempt_1436796131530_0054_000001 4
hdfs://localhost:9000/tmp/hadoop-fegaras/bsp/local/submit_lj3a8l/job.xml
1><LOG_DIR>/hama-worker.stdout 2><LOG_DIR>/hama-worker.stderr ]
15/07/15 09:24:29 INFO impl.NMClientAsyncImpl: Processing Event EventType:
START_CONTAINER for Container container_1436796131530_0054_01_000004
15/07/15 09:24:29 INFO impl.ContainerManagementProtocolProxy: Opening proxy :
crete:37056
15/07/15 09:24:29 INFO impl.NMClientAsyncImpl: Processing Event EventType:
QUERY_CONTAINER for Container container_1436796131530_0054_01_000004
15/07/15 09:24:29 INFO impl.ContainerManagementProtocolProxy: Opening proxy :
crete:37056
15/07/15 09:24:29 ERROR quorum.QuorumPeerConfig: Invalid configuration, only
one server specified (ignoring)
15/07/15 09:24:29 INFO server.ZooKeeperServerMain: Starting server
15/07/15 09:24:29 INFO server.ZooKeeperServer: Server
environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT
15/07/15 09:24:29 INFO server.ZooKeeperServer: Server
environment:host.name=crete
15/07/15 09:24:29 INFO server.ZooKeeperServer: Server
environment:java.version=1.8.0_45
15/07/15 09:24:29 INFO server.ZooKeeperServer: Server
environment:java.vendor=Oracle Corporation
15/07/15 09:24:29 INFO server.ZooKeeperServer: Server
environment:java.home=/usr/lib/jvm/java-8-oracle/jre
15/07/15 09:24:29 INFO server.ZooKeeperServer: Server
environment:java.class.path=$CLASSPATH:./hadoop-yarn-api-2.6.0.jar:./commons-math3-3.0.jar:.
...
15/07/15 09:24:29 INFO server.ZooKeeperServer: Server
environment:java.library.path=:/home/fegaras/hadoop-2.6.0/lib/native:/home/fegaras/hadoop-2.6.0/lib/native:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
15/07/15 09:24:29 INFO server.ZooKeeperServer: Server
environment:java.io.tmpdir=/tmp
15/07/15 09:24:29 INFO server.ZooKeeperServer: Server
environment:java.compiler=<NA>
15/07/15 09:24:29 INFO server.ZooKeeperServer: Server environment:os.name=Linux
15/07/15 09:24:29 INFO server.ZooKeeperServer: Server environment:os.arch=amd64
15/07/15 09:24:29 INFO server.ZooKeeperServer: Server
environment:os.version=3.2.0-83-generic
15/07/15 09:24:29 INFO server.ZooKeeperServer: Server
environment:user.name=fegaras
15/07/15 09:24:29 INFO server.ZooKeeperServer: Server
environment:user.home=/home/fegaras
15/07/15 09:24:29 INFO server.ZooKeeperServer: Server
environment:user.dir=/home/hadoop/tmp/nm-local-dir/usercache/fegaras/appcache/application_1436796131530_0054/container_1436796131530_0054_01_000001
15/07/15 09:24:29 INFO server.ZooKeeperServer: tickTime set to 3000
15/07/15 09:24:29 INFO server.ZooKeeperServer: minSessionTimeout set to -1
15/07/15 09:24:29 INFO server.ZooKeeperServer: maxSessionTimeout set to 1200000
15/07/15 09:24:29 INFO server.NIOServerCnxnFactory: binding to port
0.0.0.0/0.0.0.0:21810
15/07/15 09:24:29 INFO persistence.FileSnap: Reading snapshot
/tmp/hama-fegaras/zookeeper/version-2/snapshot.183
15/07/15 09:24:29 INFO persistence.FileTxnSnapLog: Snapshotting: 0x183 to
/tmp/hama-fegaras/zookeeper/version-2/snapshot.183
15/07/15 09:24:30 INFO bsp.ApplicationMaster: Got response from RM for
container ask, allocatedCnt=1
15/07/15 09:24:30 INFO bsp.ApplicationMaster: Launching shell command on a new
container., containerId=container_1436796131530_0054_01_000005,
containerNode=crete:37056, containerNodeURI=crete:8042,
containerResourceMemory128, containerResourceVirtualCores1
15/07/15 09:24:30 INFO bsp.ApplicationMaster: Setting up container launch
container for containerid=container_1436796131530_0054_01_000005
15/07/15 09:24:30 INFO bsp.ApplicationMaster: PackageURL has been composed to
scheme: "hdfs" host: "localhost" port: 9000 file:
"/tmp/hadoop-fegaras/bsp/local/submit_lj3a8l/job.jar"
15/07/15 09:24:30 INFO bsp.ApplicationMaster: Reverting packageURL to path:
hdfs://localhost:9000/tmp/hadoop-fegaras/bsp/local/submit_lj3a8l/job.jar
15/07/15 09:24:30 INFO bsp.ApplicationMaster: Hama release URL has been
composed to scheme: "hdfs" host: "localhost" port: 9000 file:
"/tmp/hadoop-fegaras/bsp/local/hama"
15/07/15 09:24:30 INFO bsp.ApplicationMaster: Starting commands:
[${JAVA_HOME}/bin/java -cp
$CLASSPATH:./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*
org.apache.hama.bsp.BSPRunner appattempt_1436796131530_0054_000001 5
hdfs://localhost:9000/tmp/hadoop-fegaras/bsp/local/submit_lj3a8l/job.xml
1><LOG_DIR>/hama-worker.stdout 2><LOG_DIR>/hama-worker.stderr ]
15/07/15 09:24:30 INFO impl.NMClientAsyncImpl: Processing Event EventType:
START_CONTAINER for Container container_1436796131530_0054_01_000005
15/07/15 09:24:30 INFO impl.ContainerManagementProtocolProxy: Opening proxy :
crete:37056
15/07/15 09:24:30 INFO impl.NMClientAsyncImpl: Processing Event EventType:
QUERY_CONTAINER for Container container_1436796131530_0054_01_000005
15/07/15 09:24:30 INFO impl.ContainerManagementProtocolProxy: Opening proxy :
crete:37056
15/07/15 09:24:31 INFO ipc.Server: IPC Server handler 0 on 10000, call
getTask(attempt_appattempt_1436796131530_0054_000001_0000_000004_4) from
127.0.0.1:52326: error: java.io.IOException:
java.lang.ArrayIndexOutOfBoundsException: 4
java.io.IOException: java.lang.ArrayIndexOutOfBoundsException: 4
at
org.apache.hama.bsp.ApplicationMaster.getTask(ApplicationMaster.java:950)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.hama.ipc.RPC$Server.call(RPC.java:615)
at org.apache.hama.ipc.Server$Handler$1.run(Server.java:1211)
at org.apache.hama.ipc.Server$Handler$1.run(Server.java:1207)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hama.ipc.Server$Handler.run(Server.java:1207)
15/07/15 09:24:32 INFO ipc.Server: IPC Server handler 0 on 10000, call
getTask(attempt_appattempt_1436796131530_0054_000001_0000_000005_5) from
127.0.0.1:52329: error: java.io.IOException:
java.lang.ArrayIndexOutOfBoundsException: 5
java.io.IOException: java.lang.ArrayIndexOutOfBoundsException: 5
at
org.apache.hama.bsp.ApplicationMaster.getTask(ApplicationMaster.java:950)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.hama.ipc.RPC$Server.call(RPC.java:615)
at org.apache.hama.ipc.Server$Handler$1.run(Server.java:1211)
at org.apache.hama.ipc.Server$Handler$1.run(Server.java:1207)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hama.ipc.Server$Handler.run(Server.java:1207)
15/07/15 09:24:33 INFO bsp.ApplicationMaster: Got response from RM for
container ask, completedCnt=1
15/07/15 09:24:33 INFO bsp.ApplicationMaster:
appattempt_1436796131530_0054_000001 got container status for
containerID=container_1436796131530_0054_01_000004, state=COMPLETE,
exitStatus=1, diagnostics=Exception from container-launch.
Container id: container_1436796131530_0054_01_000004
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
at org.apache.hadoop.util.Shell.run(Shell.java:455)
at
org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
at
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Container exited with a non-zero exit code 1
15/07/15 09:24:34 INFO bsp.ApplicationMaster: Got response from RM for
container ask, completedCnt=1
15/07/15 09:24:34 INFO bsp.ApplicationMaster:
appattempt_1436796131530_0054_000001 got container status for
containerID=container_1436796131530_0054_01_000005, state=COMPLETE,
exitStatus=1, diagnostics=Exception from container-launch.
Container id: container_1436796131530_0054_01_000005
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
at org.apache.hadoop.util.Shell.run(Shell.java:455)
at
org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
at
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Container exited with a non-zero exit code 1
