GitHub user joseph-torres opened a pull request:
https://github.com/apache/spark/pull/19461
[SPARK-22230] Swap per-row order in state store restore.
## What changes were proposed in this pull request?
In state store restore, for each row, put the saved state before the row in
the iterator instead of after.
This fixes an issue where agg(last('attr)) will forever return the last
value of 'attr from the first microbatch.
## How was this patch tested?
new unit test
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/joseph-torres/spark SPARK-22230
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/19461.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #19461
----
commit 17ef8a843e7dec8da0625caeda213cb1f5c64a4a
Author: Jose Torres <[email protected]>
Date: 2017-10-09T20:55:19Z
Swap per-row order in state store restore.
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
