GitHub user tdas opened a pull request:
https://github.com/apache/spark/pull/2940
[SPARK-4028][Streaming] ReceivedBlockHandler interface to abstract the
functionality of storage of received data
As part of the initiative to prevent data loss on streaming driver failure,
this JIRA tracks the subtask of implementing a ReceivedBlockHandler, that
abstracts the functionality of storage of received data blocks. The default
implementation will maintain the current behavior of storing the data into
BlockManager. The optional implementation will store the data to both
BlockManager as well as a write ahead log.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/tdas/spark driver-ha-rbh
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/2940.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2940
----
commit 95a49878922751c7fb20ff7650630696b9a92dbe
Author: Tathagata Das <tathagata.das1...@gmail.com>
Date: 2014-10-25T03:57:40Z
Added ReceivedBlockHandler and its associated tests
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
