[
https://issues.apache.org/jira/browse/SPARK-16798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15400089#comment-15400089
]
Charles Allen commented on SPARK-16798:
---------------------------------------
Adding some more flavor, this is running in Mesos coarse mode against 0.28.2.
If I take a subset of the data that failed and run it locally (local[4] or
local[1]), it succeeds, which is annoying.
here are the info logs from the failing tasks:
{code}
16/07/29 18:19:20 INFO HadoopRDD: Input split: REDACTED1.gz:0+163064
16/07/29 18:19:20 INFO TorrentBroadcast: Started reading broadcast variable 0
16/07/29 18:19:20 INFO MemoryStore: Block broadcast_0_piece0 stored as bytes in
memory (estimated size 18.2 KB, free 3.6 GB)
16/07/29 18:19:20 INFO TorrentBroadcast: Reading broadcast variable 0 took 10 ms
16/07/29 18:19:20 INFO MemoryStore: Block broadcast_0 stored as values in
memory (estimated size 209.2 KB, free 3.6 GB)
16/07/29 18:19:20 INFO deprecation: mapred.tip.id is deprecated. Instead, use
mapreduce.task.id
16/07/29 18:19:20 INFO deprecation: mapred.task.id is deprecated. Instead, use
mapreduce.task.attempt.id
16/07/29 18:19:20 INFO deprecation: mapred.task.is.map is deprecated. Instead,
use mapreduce.task.ismap
16/07/29 18:19:20 INFO deprecation: mapred.task.partition is deprecated.
Instead, use mapreduce.task.partition
16/07/29 18:19:20 INFO deprecation: mapred.job.id is deprecated. Instead, use
mapreduce.job.id
16/07/29 18:19:21 INFO NativeS3FileSystem: Opening 'REDACTED1' for reading
16/07/29 18:19:21 INFO CodecPool: Got brand-new decompressor [.gz]
16/07/29 18:19:21 ERROR Executor: Exception in task 9.0 in stage 0.0 (TID 9)
java.lang.IllegalArgumentException: bound must be positive
at java.util.Random.nextInt(Random.java:388)
at
org.apache.spark.rdd.RDD$$anonfun$coalesce$1$$anonfun$9.apply(RDD.scala:445)
at
org.apache.spark.rdd.RDD$$anonfun$coalesce$1$$anonfun$9.apply(RDD.scala:444)
at
org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndex$1$$anonfun$apply$25.apply(RDD.scala:807)
at
org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndex$1$$anonfun$apply$25.apply(RDD.scala:807)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:319)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:283)
at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:79)
at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:47)
at org.apache.spark.scheduler.Task.run(Task.scala:85)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
16/07/29 18:19:21 INFO CoarseGrainedExecutorBackend: Got assigned task 14
16/07/29 18:19:21 INFO Executor: Running task 14.0 in stage 0.0 (TID 14)
16/07/29 18:19:21 INFO HadoopRDD: Input split: REDACTED2:0+157816
16/07/29 18:19:21 INFO NativeS3FileSystem: Opening 'REDACTED2' for reading
16/07/29 18:19:21 INFO CodecPool: Got brand-new decompressor [.gz]
16/07/29 18:19:21 ERROR Executor: Exception in task 14.0 in stage 0.0 (TID 14)
java.lang.IllegalArgumentException: bound must be positive
at java.util.Random.nextInt(Random.java:388)
at
org.apache.spark.rdd.RDD$$anonfun$coalesce$1$$anonfun$9.apply(RDD.scala:445)
at
org.apache.spark.rdd.RDD$$anonfun$coalesce$1$$anonfun$9.apply(RDD.scala:444)
at
org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndex$1$$anonfun$apply$25.apply(RDD.scala:807)
at
org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndex$1$$anonfun$apply$25.apply(RDD.scala:807)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:319)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:283)
at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:79)
at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:47)
at org.apache.spark.scheduler.Task.run(Task.scala:85)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
16/07/29 18:19:21 INFO CoarseGrainedExecutorBackend: Got assigned task 15
16/07/29 18:19:21 INFO Executor: Running task 9.1 in stage 0.0 (TID 15)
{code}
> java.lang.IllegalArgumentException: bound must be positive : Worked in 1.5.2
> ----------------------------------------------------------------------------
>
> Key: SPARK-16798
> URL: https://issues.apache.org/jira/browse/SPARK-16798
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 2.0.0
> Reporter: Charles Allen
>
> Code at https://github.com/metamx/druid-spark-batch which was working under
> 1.5.2 has ceased to function under 2.0.0 with the below stacktrace.
> {code}
> java.lang.IllegalArgumentException: bound must be positive
> at java.util.Random.nextInt(Random.java:388)
> at
> org.apache.spark.rdd.RDD$$anonfun$coalesce$1$$anonfun$9.apply(RDD.scala:445)
> at
> org.apache.spark.rdd.RDD$$anonfun$coalesce$1$$anonfun$9.apply(RDD.scala:444)
> at
> org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndex$1$$anonfun$apply$25.apply(RDD.scala:807)
> at
> org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndex$1$$anonfun$apply$25.apply(RDD.scala:807)
> at
> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:319)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:283)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:79)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:47)
> at org.apache.spark.scheduler.Task.run(Task.scala:85)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
