[
https://issues.apache.org/jira/browse/FLINK-6798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16040979#comment-16040979
]
ASF GitHub Bot commented on FLINK-6798:
---------------------------------------
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/4080#discussion_r120643661
--- Diff: docs/monitoring/large_state_tuning.md ---
@@ -94,21 +94,11 @@ When a savepoint is manually triggered, it may be in
process concurrently with a
## Tuning Network Buffers
-The number of network buffers is a parameter that can currently have an
effect on checkpointing at large scale.
-The Flink community is working on eliminating that parameter in the next
versions of Flink.
-
-The number of network buffers defines how much data a TaskManager can hold
in-flight before back-pressure kicks in.
-A very high number of network buffers means that a lot of data may be in
the stream network channels when a checkpoint
-is started. Because the checkpoint barriers travel with that data (see
[description of how checkpointing
works](../internals/stream_checkpointing.html)),
-a lot of in-flight data means that the barriers have to wait for that data
to be transported/processed before arriving
-at the target operator.
-
-Having a lot of data in-flight also does not speed up the data processing
as a whole. It only means that data is picked up faster
-from the data source (log, files, message queue) and buffered longer in
Flink. Having fewer network buffers means that
-data is picked up from the source more immediately before it is actually
being processed, which is generally desirable.
-The number of network buffers should hence not be set arbitrarily large,
but to a low multiple (such as 2x) of the
-minimum number of required buffers.
-
+Before Flink 1.3, an increased number of network buffers also caused
increased checkpointing times since
+keeping more in-flight data also meant that checkpoint barriers got
delayed. Since Flink 1.3, the
--- End diff --
remove "also" for a better flow.
> Remove/update documentation about network buffer tuning
> -------------------------------------------------------
>
> Key: FLINK-6798
> URL: https://issues.apache.org/jira/browse/FLINK-6798
> Project: Flink
> Issue Type: Sub-task
> Components: Documentation, Network
> Reporter: Robert Metzger
> Assignee: Nico Kruber
> Fix For: 1.3.0
>
>
> {quote}The number of network buffers is a parameter that can currently have
> an effect on checkpointing at large scale. The Flink community is working on
> eliminating that parameter in the next versions of Flink.
> {quote}
> in
> https://ci.apache.org/projects/flink/flink-docs-release-1.3/monitoring/large_state_tuning.html#tuning-network-buffers
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
