GitHub user jerryshao opened a pull request:
https://github.com/apache/spark/pull/4032
[SPARK-5233][Streaming] Fix error replaying of WAL introduced bug
Because of lacking of `BlockAllocationEvent` in WAL recovery, the dangled
event will mix into the new batch, which will lead to the wrong result. Details
can be seen in [SPARK-5233](https://issues.apache.org/jira/browse/SPARK-5233).
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jerryshao/apache-spark SPARK-5233
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/4032.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4032
----
commit 622cbbca5b7e169dcae5f08b001723eebfbbc4cd
Author: jerryshao <[email protected]>
Date: 2015-01-14T05:24:21Z
Correctly replay the WAL log when recovering from failure
commit 4c82be8e41383a4e4cb6b1e53e47928b3f245f2a
Author: jerryshao <[email protected]>
Date: 2015-01-14T05:41:20Z
Fix bug in unit test
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
