marcoabreu opened a new issue #11249: Flaky Scala test: 
IllegalArgumentException: requirement failed: Failed to start ps scheduler
URL: https://github.com/apache/incubator-mxnet/issues/11249
 
 
   
http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/incubator-mxnet/detail/PR-11246/1/pipeline/
   ``Exception in thread "Thread-113" java.lang.IllegalArgumentException: 
requirement failed: Failed to start ps scheduler process with exit code 134``
   
   ```
   18/06/12 17:16:53 INFO Utils: 
/work/mxnet/scala-package/assembly/linux-x86_64-cpu/target/mxnet-full_2.11-linux-x86_64-cpu-1.3.0-SNAPSHOT.jar
 has been previously copied to 
/tmp/spark-202cac2e-62d9-4ac4-a99a-27bca2aaf87f/userFiles-2f1680a1-aea4-4de0-8784-3300ad265be7/mxnet-full_2.11-linux-x86_64-cpu-1.3.0-SNAPSHOT.jar
   
   18/06/12 17:16:53 INFO Executor: Fetching 
file:/work/mxnet/scala-package/spark/target/mxnet-spark_2.11-1.3.0-SNAPSHOT.jar 
with timestamp 1528823813012
   
   18/06/12 17:16:53 INFO Utils: 
/work/mxnet/scala-package/spark/target/mxnet-spark_2.11-1.3.0-SNAPSHOT.jar has 
been previously copied to 
/tmp/spark-202cac2e-62d9-4ac4-a99a-27bca2aaf87f/userFiles-2f1680a1-aea4-4de0-8784-3300ad265be7/mxnet-spark_2.11-1.3.0-SNAPSHOT.jar
   
   18/06/12 17:16:53 INFO MXNet: Starting server ...
   
   18/06/12 17:16:53 INFO HadoopRDD: Input split: 
file:/tmp/mxnet-spark-test-15288237675593208593691818310660/train.txt:234881024+8528911
   
   18/06/12 17:16:53 INFO HadoopRDD: Input split: 
file:/tmp/mxnet-spark-test-15288237675593208593691818310660/train.txt:67108864+33554432
   
   18/06/12 17:16:53 INFO HadoopRDD: Input split: 
file:/tmp/mxnet-spark-test-15288237675593208593691818310660/train.txt:167772160+33554432
   
   18/06/12 17:16:53 INFO HadoopRDD: Input split: 
file:/tmp/mxnet-spark-test-15288237675593208593691818310660/train.txt:201326592+33554432
   
   18/06/12 17:16:53 INFO HadoopRDD: Input split: 
file:/tmp/mxnet-spark-test-15288237675593208593691818310660/train.txt:134217728+33554432
   
   18/06/12 17:16:53 INFO HadoopRDD: Input split: 
file:/tmp/mxnet-spark-test-15288237675593208593691818310660/train.txt:33554432+33554432
   
   18/06/12 17:16:53 INFO HadoopRDD: Input split: 
file:/tmp/mxnet-spark-test-15288237675593208593691818310660/train.txt:0+33554432
   
   18/06/12 17:16:53 INFO HadoopRDD: Input split: 
file:/tmp/mxnet-spark-test-15288237675593208593691818310660/train.txt:100663296+33554432
   
   18/06/12 17:16:53 INFO ParameterServer: Started process: java  -cp 
/tmp/spark-202cac2e-62d9-4ac4-a99a-27bca2aaf87f/userFiles-2f1680a1-aea4-4de0-8784-3300ad265be7/mxnet-full_2.11-linux-x86_64-cpu-1.3.0-SNAPSHOT.jar:/tmp/spark-202cac2e-62d9-4ac4-a99a-27bca2aaf87f/userFiles-2f1680a1-aea4-4de0-8784-3300ad265be7/mxnet-spark_2.11-1.3.0-SNAPSHOT.jar
 org.apache.mxnet.spark.ParameterServer --role=server --root-uri=172.17.0.4 
--root-port=45669 --num-server=1 --num-worker=2 --timeout=300 at 
172.17.0.4:45669
   
   18/06/12 17:16:53 INFO ParameterServer: Starting InputStream-Redirecter 
Thread for 172.17.0.4:45669
   
   18/06/12 17:16:53 INFO ParameterServer: Starting ErrorStream-Redirecter 
Thread for 172.17.0.4:45669
   
   SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
   
   SLF4J: Defaulting to no-operation (NOP) logger implementation
   
   SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further 
details.
   
   18/06/12 17:16:53 INFO Executor: Finished task 7.0 in stage 1.0 (TID 8). 
2254 bytes result sent to driver
   
   18/06/12 17:16:53 INFO TaskSetManager: Finished task 7.0 in stage 1.0 (TID 
8) in 530 ms on localhost (1/8)
   
   18/06/12 17:16:54 INFO Executor: Finished task 0.0 in stage 1.0 (TID 1). 
2254 bytes result sent to driver
   
   18/06/12 17:16:54 INFO TaskSetManager: Finished task 0.0 in stage 1.0 (TID 
1) in 1083 ms on localhost (2/8)
   
   18/06/12 17:16:54 INFO Executor: Finished task 5.0 in stage 1.0 (TID 6). 
2254 bytes result sent to driver
   
   18/06/12 17:16:54 INFO TaskSetManager: Finished task 5.0 in stage 1.0 (TID 
6) in 1092 ms on localhost (3/8)
   
   18/06/12 17:16:54 INFO Executor: Finished task 2.0 in stage 1.0 (TID 3). 
2254 bytes result sent to driver
   
   18/06/12 17:16:54 INFO Executor: Finished task 1.0 in stage 1.0 (TID 2). 
2254 bytes result sent to driver
   
   18/06/12 17:16:54 INFO TaskSetManager: Finished task 2.0 in stage 1.0 (TID 
3) in 1093 ms on localhost (4/8)
   
   18/06/12 17:16:54 INFO TaskSetManager: Finished task 1.0 in stage 1.0 (TID 
2) in 1094 ms on localhost (5/8)
   
   18/06/12 17:16:54 INFO Executor: Finished task 6.0 in stage 1.0 (TID 7). 
2254 bytes result sent to driver
   
   18/06/12 17:16:54 INFO TaskSetManager: Finished task 6.0 in stage 1.0 (TID 
7) in 1097 ms on localhost (6/8)
   
   18/06/12 17:16:54 INFO Executor: Finished task 3.0 in stage 1.0 (TID 4). 
2254 bytes result sent to driver
   
   18/06/12 17:16:54 INFO TaskSetManager: Finished task 3.0 in stage 1.0 (TID 
4) in 1100 ms on localhost (7/8)
   
   18/06/12 17:16:54 INFO Executor: Finished task 4.0 in stage 1.0 (TID 5). 
2254 bytes result sent to driver
   
   18/06/12 17:16:54 INFO TaskSetManager: Finished task 4.0 in stage 1.0 (TID 
5) in 1125 ms on localhost (8/8)
   
   18/06/12 17:16:54 INFO TaskSchedulerImpl: Removed TaskSet 1.0, whose tasks 
have all completed, from pool 
   
   18/06/12 17:16:54 INFO DAGScheduler: ShuffleMapStage 1 (repartition at 
MXNet.scala:251) finished in 1.126 s
   
   18/06/12 17:16:54 INFO DAGScheduler: looking for newly runnable stages
   
   18/06/12 17:16:54 INFO DAGScheduler: running: Set(ResultStage 0)
   
   18/06/12 17:16:54 INFO DAGScheduler: waiting: Set(ResultStage 2)
   
   18/06/12 17:16:54 INFO DAGScheduler: failed: Set()
   
   18/06/12 17:16:54 INFO DAGScheduler: Submitting ResultStage 2 
(MapPartitionsRDD[8] at mapPartitions at MXNet.scala:209), which has no missing 
parents
   
   18/06/12 17:16:54 INFO MemoryStore: Block broadcast_3 stored as values in 
memory (estimated size 8.6 KB, free 19.8 GB)
   
   18/06/12 17:16:54 INFO MemoryStore: Block broadcast_3_piece0 stored as bytes 
in memory (estimated size 3.8 KB, free 19.8 GB)
   
   18/06/12 17:16:54 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory 
on localhost:38790 (size: 3.8 KB, free: 19.8 GB)
   
   18/06/12 17:16:54 INFO SparkContext: Created broadcast 3 from broadcast at 
DAGScheduler.scala:1006
   
   18/06/12 17:16:54 INFO DAGScheduler: Submitting 2 missing tasks from 
ResultStage 2 (MapPartitionsRDD[8] at mapPartitions at MXNet.scala:209)
   
   18/06/12 17:16:54 INFO TaskSchedulerImpl: Adding task set 2.0 with 2 tasks
   
   18/06/12 17:16:54 INFO TaskSetManager: Starting task 0.0 in stage 2.0 (TID 
9, localhost, partition 0,NODE_LOCAL, 2296 bytes)
   
   18/06/12 17:16:54 INFO TaskSetManager: Starting task 1.0 in stage 2.0 (TID 
10, localhost, partition 1,NODE_LOCAL, 2296 bytes)
   
   18/06/12 17:16:54 INFO Executor: Running task 0.0 in stage 2.0 (TID 9)
   
   18/06/12 17:16:54 INFO Executor: Running task 1.0 in stage 2.0 (TID 10)
   
   18/06/12 17:16:54 INFO CacheManager: Partition rdd_8_0 not found, computing 
it
   
   18/06/12 17:16:54 INFO CacheManager: Partition rdd_8_1 not found, computing 
it
   
   18/06/12 17:16:54 INFO ShuffleBlockFetcherIterator: Getting 8 non-empty 
blocks out of 8 blocks
   
   18/06/12 17:16:54 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches 
in 0 ms
   
   18/06/12 17:16:54 INFO ShuffleBlockFetcherIterator: Getting 8 non-empty 
blocks out of 8 blocks
   
   18/06/12 17:16:54 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches 
in 0 ms
   
   18/06/12 17:16:56 INFO MXNet: Launching worker ...
   
   18/06/12 17:16:56 INFO MXNet: Batch 128
   
   18/06/12 17:16:56 INFO MXNet: Launching worker ...
   
   18/06/12 17:16:56 INFO MXNet: Batch 128
   
   18/06/12 17:17:16 INFO MXNet: Start training ...
   
   18/06/12 17:17:16 INFO MXNet: Start training ...
   
   18/06/12 17:17:16 INFO DataParallelExecutorManager: Start training with 
[cpu(0),cpu(1)]
   
   18/06/12 17:17:16 INFO DataParallelExecutorManager: Start training with 
[cpu(0),cpu(1)]
   
   18/06/12 17:18:20 INFO Model: Epoch[0] Train-accuracy=0.85657054
   
   18/06/12 17:18:20 INFO Model: Epoch[0] Time cost=63440
   
   18/06/12 17:18:20 INFO Model: Epoch[0] Train-accuracy=0.8499599
   
   18/06/12 17:18:20 INFO Model: Epoch[0] Time cost=63442
   
   18/06/12 17:18:59 INFO Model: Epoch[1] Train-accuracy=0.9565972
   
   18/06/12 17:18:59 INFO Model: Epoch[1] Time cost=38775
   
   18/06/12 17:18:59 INFO Model: Epoch[1] Train-accuracy=0.95492786
   
   18/06/12 17:18:59 INFO Model: Epoch[1] Time cost=38779
   
   18/06/12 17:19:32 INFO Model: Epoch[2] Train-accuracy=0.9720219
   
   18/06/12 17:19:32 INFO Model: Epoch[2] Time cost=33500
   
   18/06/12 17:19:32 INFO Model: Epoch[2] Train-accuracy=0.97215545
   
   18/06/12 17:19:32 INFO Model: Epoch[2] Time cost=33507
   
   18/06/12 17:19:58 INFO Model: Epoch[3] Train-accuracy=0.9793002
   
   18/06/12 17:19:58 INFO Model: Epoch[3] Time cost=25923
   
   18/06/12 17:19:58 INFO Model: Epoch[3] Train-accuracy=0.9788996
   
   18/06/12 17:19:58 INFO Model: Epoch[3] Time cost=26168
   
   18/06/12 17:20:29 INFO Model: Epoch[4] Train-accuracy=0.9821715
   
   18/06/12 17:20:29 INFO Model: Epoch[4] Time cost=30778
   
   18/06/12 17:20:29 INFO Model: Epoch[4] Train-accuracy=0.9831063
   
   18/06/12 17:20:29 INFO Model: Epoch[4] Time cost=30799
   
   18/06/12 17:20:59 INFO Model: Epoch[5] Train-accuracy=0.9841079
   
   18/06/12 17:20:59 INFO Model: Epoch[5] Time cost=29636
   
   18/06/12 17:20:59 INFO Model: Epoch[5] Train-accuracy=0.9863782
   
   18/06/12 17:20:59 INFO Model: Epoch[5] Time cost=29593
   
   18/06/12 17:21:24 INFO Model: Epoch[6] Train-accuracy=0.9855769
   
   18/06/12 17:21:24 INFO Model: Epoch[6] Time cost=25173
   
   18/06/12 17:21:24 INFO Model: Epoch[6] Train-accuracy=0.9882479
   
   18/06/12 17:21:24 INFO Model: Epoch[6] Time cost=25212
   
   18/06/12 17:21:51 INFO Model: Epoch[7] Train-accuracy=0.9881143
   
   18/06/12 17:21:51 INFO Model: Epoch[7] Time cost=26722
   
   18/06/12 17:21:51 INFO Model: Epoch[7] Train-accuracy=0.9898504
   
   18/06/12 17:21:51 INFO Model: Epoch[7] Time cost=26627
   
   terminate called without an active exception
   
   terminate called without an active exception
   
   Exception in thread "Thread-113" java.lang.IllegalArgumentException: 
requirement failed: Failed to start ps scheduler process with exit code 134
   
        at scala.Predef$.require(Predef.scala:224)
   
        at 
org.apache.mxnet.spark.MXNet.org$apache$mxnet$spark$MXNet$$startPSSchedulerInner$1(MXNet.scala:159)
   
        at 
org.apache.mxnet.spark.MXNet$$anonfun$startPSScheduler$1.apply(MXNet.scala:162)
   
        at 
org.apache.mxnet.spark.MXNet$$anonfun$startPSScheduler$1.apply(MXNet.scala:162)
   
        at 
org.apache.mxnet.spark.MXNet$MXNetControllingThread.run(MXNet.scala:38)
   
   18/06/12 17:21:57 ERROR Executor: Exception in task 0.0 in stage 0.0 (TID 0)
   
   java.lang.IllegalArgumentException: requirement failed: ps server process 
quit with exit code 134
   
        at scala.Predef$.require(Predef.scala:224)
   
        at 
org.apache.mxnet.spark.MXNet$$anonfun$org$apache$mxnet$spark$MXNet$$startPSServersInner$1$1.apply(MXNet.scala:137)
   
        at 
org.apache.mxnet.spark.MXNet$$anonfun$org$apache$mxnet$spark$MXNet$$startPSServersInner$1$1.apply(MXNet.scala:126)
   
        at 
org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
   
        at 
org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
   
        at 
org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
   
        at 
org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
   
        at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
   
        at org.apache.spark.scheduler.Task.run(Task.scala:89)
   
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:227)
   
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   
        at java.lang.Thread.run(Thread.java:748)
   
   18/06/12 17:21:57 WARN TaskSetManager: Lost task 0.0 in stage 0.0 (TID 0, 
localhost): java.lang.IllegalArgumentException: requirement failed: ps server 
process quit with exit code 134
   
        at scala.Predef$.require(Predef.scala:224)
   
        at 
org.apache.mxnet.spark.MXNet$$anonfun$org$apache$mxnet$spark$MXNet$$startPSServersInner$1$1.apply(MXNet.scala:137)
   
        at 
org.apache.mxnet.spark.MXNet$$anonfun$org$apache$mxnet$spark$MXNet$$startPSServersInner$1$1.apply(MXNet.scala:126)
   
        at 
org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
   
        at 
org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
   
        at 
org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
   
        at 
org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
   
        at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
   
        at org.apache.spark.scheduler.Task.run(Task.scala:89)
   
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:227)
   
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   
        at java.lang.Thread.run(Thread.java:748)
   
   
   
   18/06/12 17:21:57 ERROR TaskSetManager: Task 0 in stage 0.0 failed 1 times; 
aborting job
   
   18/06/12 17:21:57 INFO TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks 
have all completed, from pool 
   
   18/06/12 17:21:57 INFO TaskSchedulerImpl: Cancelling stage 0
   
   18/06/12 17:21:57 INFO DAGScheduler: ResultStage 0 (foreachPartition at 
MXNet.scala:126) failed in 304.794 s
   
   18/06/12 17:21:57 INFO DAGScheduler: Job 0 failed: foreachPartition at 
MXNet.scala:126, took 304.801371 s
   
   Exception in thread "Thread-114" org.apache.spark.SparkException: Job 
aborted due to stage failure: Task 0 in stage 0.0 failed 1 times, most recent 
failure: Lost task 0.0 in stage 0.0 (TID 0, localhost): 
java.lang.IllegalArgumentException: requirement failed: ps server process quit 
with exit code 134
   
        at scala.Predef$.require(Predef.scala:224)
   
        at 
org.apache.mxnet.spark.MXNet$$anonfun$org$apache$mxnet$spark$MXNet$$startPSServersInner$1$1.apply(MXNet.scala:137)
   
        at 
org.apache.mxnet.spark.MXNet$$anonfun$org$apache$mxnet$spark$MXNet$$startPSServersInner$1$1.apply(MXNet.scala:126)
   
        at 
org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
   
        at 
org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
   
        at 
org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
   
        at 
org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
   
        at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
   
        at org.apache.spark.scheduler.Task.run(Task.scala:89)
   
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:227)
   
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   
        at java.lang.Thread.run(Thread.java:748)
   
   
   
   Driver stacktrace:
   
        at 
org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1431)
   
        at 
org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1419)
   
        at 
org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1418)
   
        at 
scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
   
        at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
   
        at 
org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1418)
   
        at 
org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:799)
   
        at 
org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:799)
   
        at scala.Option.foreach(Option.scala:257)
   
        at 
org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:799)
   
        at 
org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1640)
   
        at 
org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1599)
   
        at 
org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1588)
   
        at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
   
        at 
org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:620)
   
        at org.apache.spark.SparkContext.runJob(SparkContext.scala:1832)
   
        at org.apache.spark.SparkContext.runJob(SparkContext.scala:1845)
   
        at org.apache.spark.SparkContext.runJob(SparkContext.scala:1858)
   
        at org.apache.spark.SparkContext.runJob(SparkContext.scala:1929)
   
        at 
org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1.apply(RDD.scala:920)
   
        at 
org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1.apply(RDD.scala:918)
   
        at 
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150)
   
        at 
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:111)
   
        at org.apache.spark.rdd.RDD.withScope(RDD.scala:316)
   
        at org.apache.spark.rdd.RDD.foreachPartition(RDD.scala:918)
   
        at 
org.apache.mxnet.spark.MXNet.org$apache$mxnet$spark$MXNet$$startPSServersInner$1(MXNet.scala:126)
   
        at 
org.apache.mxnet.spark.MXNet$$anonfun$startPSServers$1.apply(MXNet.scala:140)
   
        at 
org.apache.mxnet.spark.MXNet$$anonfun$startPSServers$1.apply(MXNet.scala:140)
   
        at 
org.apache.mxnet.spark.MXNet$MXNetControllingThread.run(MXNet.scala:38)
   
   Caused by: java.lang.IllegalArgumentException: requirement failed: ps server 
process quit with exit code 134
   
        at scala.Predef$.require(Predef.scala:224)
   
        at 
org.apache.mxnet.spark.MXNet$$anonfun$org$apache$mxnet$spark$MXNet$$startPSServersInner$1$1.apply(MXNet.scala:137)
   
        at 
org.apache.mxnet.spark.MXNet$$anonfun$org$apache$mxnet$spark$MXNet$$startPSServersInner$1$1.apply(MXNet.scala:126)
   
        at 
org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
   
        at 
org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
   
        at 
org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
   
        at 
org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
   
        at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
   
        at org.apache.spark.scheduler.Task.run(Task.scala:89)
   
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:227)
   
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   
        at java.lang.Thread.run(Thread.java:748)
   
   18/06/12 17:46:53 INFO BlockManagerInfo: Removed broadcast_2_piece0 on 
localhost:38790 in memory (size: 2.4 KB, free: 19.8 GB)
   
   18/06/12 17:46:53 INFO BlockManagerInfo: Removed broadcast_1_piece0 on 
localhost:38790 in memory (size: 2.7 KB, free: 19.8 GB)
   
   18/06/12 17:46:53 INFO ContextCleaner: Cleaned accumulator 5
   
   Sending interrupt signal to process
   
   build.py: 2018-06-12 19:08:38,620 Running of command in container failed 
(-15): docker run --rm -t --shm-size=500m -v 
/home/jenkins_slave/workspace/ut-scala-cpu:/work/mxnet -v 
/home/jenkins_slave/workspace/ut-scala-cpu/build:/work/build -v 
/tmp/ci_ccache:/work/ccache -u 1001:1001 -e CCACHE_MAXSIZE=10G -e 
CCACHE_DIR=/work/ccache mxnetci/build.ubuntu_cpu /work/runtime_functions.sh 
unittest_ubuntu_cpu_scala
   
   build.py: 2018-06-12 19:08:38,620 You can try to get into the container by 
using the following command: docker run --rm -t --shm-size=500m -v 
/home/jenkins_slave/workspace/ut-scala-cpu:/work/mxnet -v 
/home/jenkins_slave/workspace/ut-scala-cpu/build:/work/build -v 
/tmp/ci_ccache:/work/ccache -u 1001:1001 -ti --entrypoint /bin/bash -e 
CCACHE_MAXSIZE=10G -e CCACHE_DIR=/work/ccache mxnetci/build.ubuntu_cpu 
/work/runtime_functions.sh unittest_ubuntu_cpu_scala
   
   Terminated
   
   script returned exit code 143
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

Reply via email to