Github user lw-lin commented on the pull request:
https://github.com/apache/spark/pull/12035#issuecomment-202881741
As reported in
[\[Spark-13211\]](https://issues.apache.org/jira/browse/SPARK-13211),
`StreamingContext` would throw `NoSuchElementException` when created from
non-existent checkpoint directory.
Actually, the `SparkContext.getOrCreate(path, ...)` was added in
https://github.com/apache/spark/commit/a1b8dd53e3474dae2c49b30bc9719c7f6b98c7cc
as the major API for users to create a `StreamingContext` from some checkpoint;
see
http://spark.apache.org/docs/latest/streaming-programming-guide.html#checkpointing.
`SparkContext.getOrCreate(path, ...)` checks if the path exists, checks if any
valid checkpoint file exists in the given path, and does or does not re-throw
the exception when encounter one.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
