Good catch, we should probably just load it from the checkpoint in the latter case.
Please send pull requests to apache/incubator-spark instead -- there are some new docs on it here: https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark Matei On Sep 18, 2013, at 1:01 PM, Vadim Chekan <[email protected]> wrote: > Hi all, > > I'm trying to load streaming context from a checkpoint: "val ctx = new > StreamingContext("\\Temp\\spark-checkpoint")" and I'm getting: > "Spark Streaming cannot be used without setting spark.cleaner.ttl; set this > property before creating a SparkContext (use SPARK_JAVA_OPTS for the shell)" > > Apparently MetadataCleaner delay is set to default 3600 when streaming > context is created as "new StreamingContext("local", "cpc-stub", > Seconds(10))" but not when "new > StreamingContext("\\Temp\\spark-checkpoint")" > > https://github.com/mesos/spark/blob/master/streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala#L546 > > Is there a reason for spark.cleaner.ttl to be set to a default value in one > constructor but not in another? If it is just an omission, is github's > mesos/spark still preferred way to send pull request? > > Vadim. > > -- > From RFC 2631: In ASN.1, EXPLICIT tagging is implicit unless IMPLICIT is > explicitly specified
