GitHub user buptljy opened a pull request:
https://github.com/apache/bahir-flink/pull/25
add redis sink checkpoint
It seems that the redis sink is not very suitable for some specific
circumstances.
For example, I want to do a word count and I enable the checkpoint
machenism, it seems to be wrong if I start my program from checkpoint when my
program fails, because datas that already sent will be sent to redisSink again.
I don't know if it is acceptable for you, so I write a simple version with
a flushOnCheckpoint option. I have a further improvement idea that we can use
redis pipeline if we are going to store datas into redis when snapshot.
I will optimize it if this idea looks good to you.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/buptljy/bahir-flink redissink
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/bahir-flink/pull/25.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #25
----
commit 683b0ed9ac5665ea731920a92c193589de285d01
Author: Liao Jiayi <[email protected]>
Date: 2017-12-05T03:43:47Z
add redis sink checkpoint
----
---
