Github user arunmahadevan commented on the issue:
https://github.com/apache/spark/pull/21490
If "checkpointLocation" is set and the SS application is restarted, it
throws the below exception because the currentOffset and lastOffsetCommitted
are initialized to -1.
```
18/06/01 22:47:04 ERROR MicroBatchExecution: Query [id =
0bdc4428-5d21-4237-9d64-898ae65f28f3, runId =
f6822423-2bd2-47c1-8ed6-799d1c481195] terminated with error
java.lang.RuntimeException: Offsets committed out of order: 2 followed by -1
at scala.sys.package$.error(package.scala:27)
at
org.apache.spark.sql.execution.streaming.sources.TextSocketMicroBatchReader.commit(socket.scala:197)
at
org.apache.spark.sql.execution.streaming.MicroBatchExecution$$anonfun$org$apache$spark$sql$execution$streaming$MicroBatchExecution$$constructNextBatch$1$$anonfun$apply$mcZ$sp$2$$anonfun$apply$mcV$sp$5.apply(MicroBatchExecution.scala:377)
```
The proposed patch fixes this so that the query can be restarted with saved
state. This could be used for running sample programs with saved state and
examining the results (though its not fully fault tolerant)
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
