GitHub user brkyvz opened a pull request:

    [SPARK-17921] failfast on checkpointLocation specified for memory streams

    ## What changes were proposed in this pull request?
    The checkpointLocation option in memory streams in StructuredStreaming is 
not used during recovery. However, it can use this location if it is being set. 
However, during recovery, if this location was set, we get an exception saying 
that we will not use this location for recovery, please delete it.
    It's better to just fail before you start the stream in the first place
    ## How was this patch tested?
    Unit tests

You can merge this pull request into a Git repository by running:

    $ git pull ff-cpl-memory

Alternatively you can review and apply these changes as the patch at:

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #15470
commit 31ac2306e991a6ee2f26f8d6b05b33be84a7052e
Author: Burak Yavuz <>
Date:   2016-10-13T20:56:18Z

    failfast on checkpointLocation specified for memory streams


If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at or file a JIRA ticket
with INFRA.

To unsubscribe, e-mail:
For additional commands, e-mail:

Reply via email to