Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/4080#discussion_r120643661
--- Diff: docs/monitoring/large_state_tuning.md ---
@@ -94,21 +94,11 @@ When a savepoint is manually triggered, it may be in
process concurrently with a
## Tuning Network Buffers
-The number of network buffers is a parameter that can currently have an
effect on checkpointing at large scale.
-The Flink community is working on eliminating that parameter in the next
versions of Flink.
-
-The number of network buffers defines how much data a TaskManager can hold
in-flight before back-pressure kicks in.
-A very high number of network buffers means that a lot of data may be in
the stream network channels when a checkpoint
-is started. Because the checkpoint barriers travel with that data (see
[description of how checkpointing
works](../internals/stream_checkpointing.html)),
-a lot of in-flight data means that the barriers have to wait for that data
to be transported/processed before arriving
-at the target operator.
-
-Having a lot of data in-flight also does not speed up the data processing
as a whole. It only means that data is picked up faster
-from the data source (log, files, message queue) and buffered longer in
Flink. Having fewer network buffers means that
-data is picked up from the source more immediately before it is actually
being processed, which is generally desirable.
-The number of network buffers should hence not be set arbitrarily large,
but to a low multiple (such as 2x) of the
-minimum number of required buffers.
-
+Before Flink 1.3, an increased number of network buffers also caused
increased checkpointing times since
+keeping more in-flight data also meant that checkpoint barriers got
delayed. Since Flink 1.3, the
--- End diff --
remove "also" for a better flow.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
