Repository: flink
Updated Branches:
  refs/heads/master 27be32e8a -> b005ea353

[FLINK-9183] [docs] Add warning about idle partitions to Kafka connector docs.

This closes #5858.


Branch: refs/heads/master
Commit: b005ea35374f619298cebab649e0ba477aeaf860
Parents: afad30a
Author: juhoautio <>
Authored: Mon Apr 16 17:40:23 2018 +0300
Committer: Fabian Hueske <>
Committed: Mon Apr 16 21:40:57 2018 +0200

 docs/dev/connectors/ | 5 +++++
 1 file changed, 5 insertions(+)
diff --git a/docs/dev/connectors/ b/docs/dev/connectors/
index 27fca7a..47a6651 100644
--- a/docs/dev/connectors/
+++ b/docs/dev/connectors/
@@ -451,6 +451,11 @@ the `Watermark getCurrentWatermark()` (for periodic) or the
 `Watermark checkAndGetNextWatermark(T lastElement, long extractedTimestamp)` 
(for punctuated) is called to determine
 if a new watermark should be emitted and with which timestamp.
+**Note**: If a watermark assigner depends on records read from Kafka to 
advance its watermarks (which is commonly the case), all topics and partitions 
need to have a continuous stream of records. Otherwise, the watermarks of the 
whole application cannot advance and all time-based operations, such as time 
windows or functions with timers, cannot make progress. A single idle Kafka 
partition causes this behavior. 
+A Flink improvement is planned to prevent this from happening 
+(see [FLINK-5479: Per-partition watermarks in FlinkKafkaConsumer should 
consider idle partitions](
+In the meanwhile, a possible workaround is to send *heartbeat messages* to all 
consumed partitions that advance the watermarks of idle partitions.
 ## Kafka Producer

Reply via email to