Michael Kaufmann created CRAIL-37:
-------------------------------------
Summary: Map failed
Key: CRAIL-37
URL: https://issues.apache.org/jira/browse/CRAIL-37
Project: Apache Crail
Issue Type: Bug
Reporter: Michael Kaufmann
Crail fails to clean up the cache directory (I assume) if applications crash.
This needs to be fixed somehow since this happens every now and then (and very
often for me because I'm debugging some other, unrelated issues). I think this
makes it impossible to use Crail as it is right now in a production environment.
As a side node - map failed is not a very useful error message and without your
own personal [~pepperjo] you might really get stuck there. As a first step, I
would expect an understandable error message that indicates what the problem is
and how it can be resolved.
{{Exception in thread "main" java.util.concurrent.ExecutionException:
java.io.IOException: Map failed}}
{{ at
org.apache.crail.core.CoreMetaDataOperation.get(CoreMetaDataOperation.java:97)}}
{{ at
org.apache.spark.storage.CrailDispatcher.org$apache$spark$storage$CrailDispatcher$$init(CrailDispatcher.scala:127)}}
{{ at
org.apache.spark.storage.CrailDispatcher$.get(CrailDispatcher.scala:613)}}
{{ at
org.apache.spark.shuffle.crail.CrailShuffleManager.registerShuffle(CrailShuffleManager.scala:52)}}
{{ at org.apache.spark.ShuffleDependency.<init>(Dependency.scala:90)}}
{{ at
org.apache.spark.rdd.CoGroupedRDD$$anonfun$getDependencies$1.apply(CoGroupedRDD.scala:106)}}
{{ at
org.apache.spark.rdd.CoGroupedRDD$$anonfun$getDependencies$1.apply(CoGroupedRDD.scala:100)}}
{{ at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)}}
{{ at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)}}
{{ at scala.collection.immutable.List.foreach(List.scala:381)}}
{{ at
scala.collection.TraversableLike$class.map(TraversableLike.scala:234)}}
{{ at scala.collection.immutable.List.map(List.scala:285)}}
{{ at
org.apache.spark.rdd.CoGroupedRDD.getDependencies(CoGroupedRDD.scala:100)}}
{{ at
org.apache.spark.rdd.RDD$$anonfun$dependencies$2.apply(RDD.scala:239)}}
{{ at
org.apache.spark.rdd.RDD$$anonfun$dependencies$2.apply(RDD.scala:237)}}
{{ at scala.Option.getOrElse(Option.scala:121)}}
{{ at org.apache.spark.rdd.RDD.dependencies(RDD.scala:237)}}
{{ at
org.apache.spark.rdd.CoGroupedRDD$$anonfun$getPartitions$1$$anonfun$apply$mcVI$sp$1.apply(CoGroupedRDD.scala:118)}}
{{ at
org.apache.spark.rdd.CoGroupedRDD$$anonfun$getPartitions$1$$anonfun$apply$mcVI$sp$1.apply(CoGroupedRDD.scala:116)}}
{{ at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)}}
{{ at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)}}
{{ at scala.collection.immutable.List.foreach(List.scala:381)}}
{{ at
scala.collection.TraversableLike$class.map(TraversableLike.scala:234)}}
{{ at scala.collection.immutable.List.map(List.scala:285)}}
{{ at
org.apache.spark.rdd.CoGroupedRDD$$anonfun$getPartitions$1.apply$mcVI$sp(CoGroupedRDD.scala:116)}}
{{ at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:160)}}
{{ at
org.apache.spark.rdd.CoGroupedRDD.getPartitions(CoGroupedRDD.scala:114)}}
{{ at
org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:252)}}
{{ at
org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:250)}}
{{ at scala.Option.getOrElse(Option.scala:121)}}
{{ at org.apache.spark.rdd.RDD.partitions(RDD.scala:250)}}
{{ at
org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:35)}}
{{ at
org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:252)}}
{{ at
org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:250)}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
