[FLINK-9183] [docs] Add warning about idle partitions to Kafka connector docs.

This closes #5858.

Project: http://git-wip-us.apache.org/repos/asf/flink/repo
Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/fc58f987
Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/fc58f987
Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/fc58f987

Branch: refs/heads/release-1.5
Commit: fc58f987144a307e7208de0bdfe439922bed4b55
Parents: 4c3d018
Author: juhoautio <autioj...@gmail.com>
Authored: Mon Apr 16 17:40:23 2018 +0300
Committer: Fabian Hueske <fhue...@apache.org>
Committed: Mon Apr 16 21:42:21 2018 +0200

 docs/dev/connectors/kafka.md | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/docs/dev/connectors/kafka.md b/docs/dev/connectors/kafka.md
index 27fca7a..47a6651 100644
--- a/docs/dev/connectors/kafka.md
+++ b/docs/dev/connectors/kafka.md
@@ -451,6 +451,11 @@ the `Watermark getCurrentWatermark()` (for periodic) or the
 `Watermark checkAndGetNextWatermark(T lastElement, long extractedTimestamp)` 
(for punctuated) is called to determine
 if a new watermark should be emitted and with which timestamp.
+**Note**: If a watermark assigner depends on records read from Kafka to 
advance its watermarks (which is commonly the case), all topics and partitions 
need to have a continuous stream of records. Otherwise, the watermarks of the 
whole application cannot advance and all time-based operations, such as time 
windows or functions with timers, cannot make progress. A single idle Kafka 
partition causes this behavior. 
+A Flink improvement is planned to prevent this from happening 
+(see [FLINK-5479: Per-partition watermarks in FlinkKafkaConsumer should 
consider idle partitions](
+In the meanwhile, a possible workaround is to send *heartbeat messages* to all 
consumed partitions that advance the watermarks of idle partitions.
 ## Kafka Producer

Reply via email to