[GitHub] spark pull request #19924: [SPARK-22187][SS][REVERT] Revert change in state ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/19924


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19924: [SPARK-22187][SS][REVERT] Revert change in state ...

[tdas]

GitHub user tdas opened a pull request:

https://github.com/apache/spark/pull/19924

[SPARK-22187][SS][REVERT] Revert change in state row format for 
mapGroupsWithState

## What changes were proposed in this pull request?

#19416 changed the format in which rows were encoded in the state store. 
However, this can break existing streaming queries with the old format in 
unpredictable ways (potentially crashing the JVM). Hence I am reverting this 
for now. This will be re-applied in the future after we start saving more 
metadata in checkpoints to signify which version of state row format the 
existing streaming query is running. Then we can decode old and new formats 
accordingly.

## How was this patch tested?
Existing tests.

Please review http://spark.apache.org/contributing.html before opening a 
pull request.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tdas/spark SPARK-22187-1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/19924.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #19924


commit 4a606de0c797fa6e8338d9a4987ff1a35e954c54
Author: Tathagata Das 
Date:   2017-11-08T08:00:23Z

Revert row format




---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org