Re: Clean shutdown of streaming job

2018-10-24 Thread Ning Shi
Hi Neils, Thanks for the response. > I think your problem is that the Cassandra sink doesn't support exactly > once guarantees when the Cassandra query isn't idempotent. If possible, the > cleanest solution would be implementing a new or extending the existing > Cassandra sink with the >

Re: Clean shutdown of streaming job

2018-10-22 Thread Niels van Kaam
Hi Ning, I don't think it is possible to pause a Kafka source upon taking a savepoint without making any changes to the implementation. I think your problem is that the Cassandra sink doesn't support exactly once guarantees when the Cassandra query isn't idempotent. If possible, the cleanest

Clean shutdown of streaming job

2018-10-21 Thread Ning Shi
I'm implementing a streaming job that consumes events from Kafka and writes results to Cassandra. The writes to Cassandra are not idempotent. In preparation for planned maintenance events like Flink version upgrade or job update, I'd like to be able to shutdown the job cleanly. I understand that