Chuanlei Ni created SPARK-17632:

             Summary: make console sink and other sinks work with 
'recoverFromCheckpointLocation' option enabled
                 Key: SPARK-17632
             Project: Spark
          Issue Type: Bug
          Components: SQL, Streaming
    Affects Versions: 2.0.0
            Reporter: Chuanlei Ni
            Priority: Trivial

val (useTempCheckpointLocation, recoverFromCheckpointLocation) =
        if (source == "console") {
          (true, false)
        } else {
          (false, true)
I think 'source' sink should work with 'recoverFromCheckpoint' option enabled. 
If user specified checkpointLocation, we may lost the processed data, however 
it is acceptable since this sink used for test mainly.

And other sink can be work with 'useTempCheckpointLocation' enabled, which 
means  replaying all the data in kafka. If user mean it, it is ok.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

Reply via email to