Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/19407#discussion_r150314230
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamWriter.scala
---
@@ -267,11 +267,12 @@ final class DataStreamWriter[T] private[sql](ds:
Dataset[T]) {
useTempCheckpointLocation = true,
trigger = trigger)
} else {
- val (useTempCheckpointLocation, recoverFromCheckpointLocation) =
+ val recoverFromCheckpointLocation = true
+ val useTempCheckpointLocation =
--- End diff --
nit: `val useTempCheckpointLocation = source == "console"`
you can just also update the below statement to
```
df.sparkSession.sessionState.streamingQueryManager.startQuery(
extraOptions.get("queryName"),
extraOptions.get("checkpointLocation"),
df,
dataSource.createSink(outputMode),
outputMode,
useTempCheckpointLocation = source == "console",
recoverFromCheckpointLocation = true,
trigger = trigger)
```
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
