[jira] [Commented] (FLINK-8458) Add the switch for keeping both the old mode and the new credit-based mode
[ https://issues.apache.org/jira/browse/FLINK-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16385973#comment-16385973 ] ASF GitHub Bot commented on FLINK-8458: --- Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/5317 > Add the switch for keeping both the old mode and the new credit-based mode > -- > > Key: FLINK-8458 > URL: https://issues.apache.org/jira/browse/FLINK-8458 > Project: Flink > Issue Type: Sub-task > Components: Network >Reporter: zhijiang >Assignee: zhijiang >Priority: Major > Fix For: 1.5.0 > > > After the whole feature of credit-based flow control is done, we should add a > config parameter to switch on/off the new credit-based mode. To do so, we can > roll back to the old network mode for any expected risks. > The parameter is defined as > {{taskmanager.network.credit-based-flow-control.enabled}} and the default > value is true. This switch may be removed after next release. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8458) Add the switch for keeping both the old mode and the new credit-based mode
[ https://issues.apache.org/jira/browse/FLINK-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16382064#comment-16382064 ] ASF GitHub Bot commented on FLINK-8458: --- Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/5317#discussion_r171575083 --- Diff: flink-core/src/main/java/org/apache/flink/configuration/TaskManagerOptions.java --- @@ -269,15 +269,21 @@ public static final ConfigOption NETWORK_BUFFERS_PER_CHANNEL = key("taskmanager.network.memory.buffers-per-channel") .defaultValue(2) - .withDescription("Number of network buffers to use for each outgoing/incoming channel (subpartition/input channel)."); + .withDescription("Number of network buffers to use for each outgoing/incoming channel (subpartition/input channel)." + + "In credit-based flow control mode, this indicates how many credits are exclusive in each input channel. It should be" + + " configured at least 2 for good performance. 1 buffer is for receiving in-flight data in the subpartition and 1 buffer is" + + " for parallel serialization."); /** * Number of extra network buffers to use for each outgoing/incoming gate (result partition/input gate). */ public static final ConfigOption NETWORK_EXTRA_BUFFERS_PER_GATE = key("taskmanager.network.memory.floating-buffers-per-gate") .defaultValue(8) - .withDescription("Number of extra network buffers to use for each outgoing/incoming gate (result partition/input gate)."); + .withDescription("Number of extra network buffers to use for each outgoing/incoming gate (result partition/input gate)." + + " In credit-based flow control mode, this indicates how many floating credits are shared among all the input channels." + + " The floating buffers are distributed based on backlog (real-time output buffers in the subpartition) feedback, and can" + + " help relieve back-pressure caused by unbalanced data distribution among the subpartitions."); --- End diff -- Yeah, I already added it. > Add the switch for keeping both the old mode and the new credit-based mode > -- > > Key: FLINK-8458 > URL: https://issues.apache.org/jira/browse/FLINK-8458 > Project: Flink > Issue Type: Sub-task > Components: Network >Reporter: zhijiang >Assignee: zhijiang >Priority: Major > Fix For: 1.5.0 > > > After the whole feature of credit-based flow control is done, we should add a > config parameter to switch on/off the new credit-based mode. To do so, we can > roll back to the old network mode for any expected risks. > The parameter is defined as > {{taskmanager.network.credit-based-flow-control.enabled}} and the default > value is true. This switch may be removed after next release. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8458) Add the switch for keeping both the old mode and the new credit-based mode
[ https://issues.apache.org/jira/browse/FLINK-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16381840#comment-16381840 ] ASF GitHub Bot commented on FLINK-8458: --- Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/5317#discussion_r166921740 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/SequenceNumberingViewReader.java --- @@ -77,14 +78,35 @@ void requestSubpartitionView( } } - InputChannelID getReceiverId() { + @Override + public void addCredit(int creditDeltas) { + } + + @Override + public void setRegisteredAsAvailable(boolean isRegisteredAvailable) { + } + + @Override + public boolean isRegisteredAsAvailable() { + return false; --- End diff -- Why this is not implemented? > Add the switch for keeping both the old mode and the new credit-based mode > -- > > Key: FLINK-8458 > URL: https://issues.apache.org/jira/browse/FLINK-8458 > Project: Flink > Issue Type: Sub-task > Components: Network >Reporter: zhijiang >Assignee: zhijiang >Priority: Major > Fix For: 1.5.0 > > > After the whole feature of credit-based flow control is done, we should add a > config parameter to switch on/off the new credit-based mode. To do so, we can > roll back to the old network mode for any expected risks. > The parameter is defined as > {{taskmanager.network.credit-based-flow-control.enabled}} and the default > value is true. This switch may be removed after next release. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8458) Add the switch for keeping both the old mode and the new credit-based mode
[ https://issues.apache.org/jira/browse/FLINK-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16381839#comment-16381839 ] ASF GitHub Bot commented on FLINK-8458: --- Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/5317#discussion_r171525131 --- Diff: flink-core/src/main/java/org/apache/flink/configuration/TaskManagerOptions.java --- @@ -269,15 +269,21 @@ public static final ConfigOption NETWORK_BUFFERS_PER_CHANNEL = key("taskmanager.network.memory.buffers-per-channel") .defaultValue(2) - .withDescription("Number of network buffers to use for each outgoing/incoming channel (subpartition/input channel)."); + .withDescription("Number of network buffers to use for each outgoing/incoming channel (subpartition/input channel)." + + "In credit-based flow control mode, this indicates how many credits are exclusive in each input channel. It should be" + + " configured at least 2 for good performance. 1 buffer is for receiving in-flight data in the subpartition and 1 buffer is" + + " for parallel serialization."); /** * Number of extra network buffers to use for each outgoing/incoming gate (result partition/input gate). */ public static final ConfigOption NETWORK_EXTRA_BUFFERS_PER_GATE = key("taskmanager.network.memory.floating-buffers-per-gate") .defaultValue(8) - .withDescription("Number of extra network buffers to use for each outgoing/incoming gate (result partition/input gate)."); + .withDescription("Number of extra network buffers to use for each outgoing/incoming gate (result partition/input gate)." + + " In credit-based flow control mode, this indicates how many floating credits are shared among all the input channels." + + " The floating buffers are distributed based on backlog (real-time output buffers in the subpartition) feedback, and can" + + " help relieve back-pressure caused by unbalanced data distribution among the subpartitions."); --- End diff -- add: `This value should be increased in case of higher round trip times between nodes and/or larger number of machines in the cluster.`? > Add the switch for keeping both the old mode and the new credit-based mode > -- > > Key: FLINK-8458 > URL: https://issues.apache.org/jira/browse/FLINK-8458 > Project: Flink > Issue Type: Sub-task > Components: Network >Reporter: zhijiang >Assignee: zhijiang >Priority: Major > Fix For: 1.5.0 > > > After the whole feature of credit-based flow control is done, we should add a > config parameter to switch on/off the new credit-based mode. To do so, we can > roll back to the old network mode for any expected risks. > The parameter is defined as > {{taskmanager.network.credit-based-flow-control.enabled}} and the default > value is true. This switch may be removed after next release. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8458) Add the switch for keeping both the old mode and the new credit-based mode
[ https://issues.apache.org/jira/browse/FLINK-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16373865#comment-16373865 ] ASF GitHub Bot commented on FLINK-8458: --- Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/5317 @pnowojski , thanks for your messages and I already updated the commit as you suggested. > Add the switch for keeping both the old mode and the new credit-based mode > -- > > Key: FLINK-8458 > URL: https://issues.apache.org/jira/browse/FLINK-8458 > Project: Flink > Issue Type: Sub-task > Components: Network >Reporter: zhijiang >Assignee: zhijiang >Priority: Major > Fix For: 1.5.0 > > > After the whole feature of credit-based flow control is done, we should add a > config parameter to switch on/off the new credit-based mode. To do so, we can > roll back to the old network mode for any expected risks. > The parameter is defined as > {{taskmanager.network.credit-based-flow-control.enabled}} and the default > value is true. This switch may be removed after next release. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8458) Add the switch for keeping both the old mode and the new credit-based mode
[ https://issues.apache.org/jira/browse/FLINK-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16372716#comment-16372716 ] ASF GitHub Bot commented on FLINK-8458: --- Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/5317 @zhijiangW I think that configuration description should come from `/flink/docs/_includes/generated/netty_configuration.html`. @zentol has changed something recently and it seems that config descriptions are automatically generated now so we do not have to copy/paste descriptions to docs :) > Add the switch for keeping both the old mode and the new credit-based mode > -- > > Key: FLINK-8458 > URL: https://issues.apache.org/jira/browse/FLINK-8458 > Project: Flink > Issue Type: Sub-task > Components: Network >Reporter: zhijiang >Assignee: zhijiang >Priority: Major > Fix For: 1.5.0 > > > After the whole feature of credit-based flow control is done, we should add a > config parameter to switch on/off the new credit-based mode. To do so, we can > roll back to the old network mode for any expected risks. > The parameter is defined as > {{taskmanager.network.credit-based-flow-control.enabled}} and the default > value is true. This switch may be removed after next release. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8458) Add the switch for keeping both the old mode and the new credit-based mode
[ https://issues.apache.org/jira/browse/FLINK-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370087#comment-16370087 ] ASF GitHub Bot commented on FLINK-8458: --- Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/5317 @NicoK , I found most of the codes in this PR are already merged into master by commit `0093bcbe771f296baf3857ef15fe7ec9b22bbc34` in your `FLINK-8425`. Maybe I only need to add the `config.md` in this PR. > Add the switch for keeping both the old mode and the new credit-based mode > -- > > Key: FLINK-8458 > URL: https://issues.apache.org/jira/browse/FLINK-8458 > Project: Flink > Issue Type: Sub-task > Components: Network >Reporter: zhijiang >Assignee: zhijiang >Priority: Major > Fix For: 1.5.0 > > > After the whole feature of credit-based flow control is done, we should add a > config parameter to switch on/off the new credit-based mode. To do so, we can > roll back to the old network mode for any expected risks. > The parameter is defined as > {{taskmanager.network.credit-based-flow-control.enabled}} and the default > value is true. This switch may be removed after next release. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8458) Add the switch for keeping both the old mode and the new credit-based mode
[ https://issues.apache.org/jira/browse/FLINK-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370041#comment-16370041 ] ASF GitHub Bot commented on FLINK-8458: --- Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/5317 Yes, I also found that and was rebasing on the latest codes. Because I am on the Spring Festival vacation, it can be ready later today or on Thursday. > Add the switch for keeping both the old mode and the new credit-based mode > -- > > Key: FLINK-8458 > URL: https://issues.apache.org/jira/browse/FLINK-8458 > Project: Flink > Issue Type: Sub-task > Components: Network >Reporter: zhijiang >Assignee: zhijiang >Priority: Major > Fix For: 1.5.0 > > > After the whole feature of credit-based flow control is done, we should add a > config parameter to switch on/off the new credit-based mode. To do so, we can > roll back to the old network mode for any expected risks. > The parameter is defined as > {{taskmanager.network.credit-based-flow-control.enabled}} and the default > value is true. This switch may be removed after next release. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8458) Add the switch for keeping both the old mode and the new credit-based mode
[ https://issues.apache.org/jira/browse/FLINK-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16369895#comment-16369895 ] ASF GitHub Bot commented on FLINK-8458: --- Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5317 @zhijiangW can you rebase the new changes? The previous implementation got merged and from the comments I see here, there might have been some changes since then which should be applied as well. > Add the switch for keeping both the old mode and the new credit-based mode > -- > > Key: FLINK-8458 > URL: https://issues.apache.org/jira/browse/FLINK-8458 > Project: Flink > Issue Type: Sub-task > Components: Network >Reporter: zhijiang >Assignee: zhijiang >Priority: Major > Fix For: 1.5.0 > > > After the whole feature of credit-based flow control is done, we should add a > config parameter to switch on/off the new credit-based mode. To do so, we can > roll back to the old network mode for any expected risks. > The parameter is defined as > {{taskmanager.network.credit-based-flow-control.enabled}} and the default > value is true. This switch may be removed after next release. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8458) Add the switch for keeping both the old mode and the new credit-based mode
[ https://issues.apache.org/jira/browse/FLINK-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16335419#comment-16335419 ] ASF GitHub Bot commented on FLINK-8458: --- Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/5317#discussion_r163155607 --- Diff: docs/ops/config.md --- @@ -290,6 +290,12 @@ The following parameters configure Flink's JobManager and TaskManagers. - `taskmanager.network.numberOfBuffers` (deprecated, replaced by the three parameters above): The number of buffers available to the network stack. This number determines how many streaming data exchange channels a TaskManager can have at the same time and how well buffered the channels are. If a job is rejected or you get a warning that the system has not enough buffers available, increase this value (DEFAULT: **2048**). If set, it will be mapped to `taskmanager.network.memory.min` and `taskmanager.network.memory.max` based on `taskmanager.memory.segment-size`. +- `taskmanager.network.memory.buffers-per-channel`: Number of network buffers to use for each outgoing/incoming channel (subpartition/input channel). Especially in credit-based flow control mode, it indicates how many credits are exclusive in each input channel. It should be configured at least 2 for good performance. 1 buffer is for receving in-flight data in the subpartition and 1 buffer is for parallel serialization. + +- `taskmanager.network.memory.floating-buffers-per-gate`: Number of extra network buffers to use for each outgoing/incoming gate (result partition/input gate). In credit-based flow control mode, it indicates how many floating credits are shared for all the input channels. The floating buffers are distributed based on backlog (real-time output buffers in the subpartition) feedback. So the floating buffers can help relief back-pressure caused by imbalance data distribution among subpartitions. + --- End diff -- thanks for your polish, alpinegizmo. I will apply the above fixes. > Add the switch for keeping both the old mode and the new credit-based mode > -- > > Key: FLINK-8458 > URL: https://issues.apache.org/jira/browse/FLINK-8458 > Project: Flink > Issue Type: Sub-task > Components: Network >Reporter: zhijiang >Assignee: zhijiang >Priority: Major > Fix For: 1.5.0 > > > After the whole feature of credit-based flow control is done, we should add a > config parameter to switch on/off the new credit-based mode. To do so, we can > roll back to the old network mode for any expected risks. > The parameter is defined as > {{taskmanager.network.credit-based-flow-control.enabled}} and the default > value is true. This switch may be removed after next release. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8458) Add the switch for keeping both the old mode and the new credit-based mode
[ https://issues.apache.org/jira/browse/FLINK-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16335418#comment-16335418 ] ASF GitHub Bot commented on FLINK-8458: --- Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/5317#discussion_r163155437 --- Diff: docs/ops/config.md --- @@ -290,6 +290,12 @@ The following parameters configure Flink's JobManager and TaskManagers. - `taskmanager.network.numberOfBuffers` (deprecated, replaced by the three parameters above): The number of buffers available to the network stack. This number determines how many streaming data exchange channels a TaskManager can have at the same time and how well buffered the channels are. If a job is rejected or you get a warning that the system has not enough buffers available, increase this value (DEFAULT: **2048**). If set, it will be mapped to `taskmanager.network.memory.min` and `taskmanager.network.memory.max` based on `taskmanager.memory.segment-size`. +- `taskmanager.network.memory.buffers-per-channel`: Number of network buffers to use for each outgoing/incoming channel (subpartition/input channel). Especially in credit-based flow control mode, it indicates how many credits are exclusive in each input channel. It should be configured at least 2 for good performance. 1 buffer is for receving in-flight data in the subpartition and 1 buffer is for parallel serialization. + --- End diff -- It is also used in current old mode and it is no need to change the default value in most cases in the old mode. Considering the new credit-based mode, if the value greater than 2, it can get benefits if the bottleneck is caused by slow downstream processing. The greater the value is set, the lower probability of blocking the upstream and causing back-pressure. But we should also consider the total available buffer resources for setting this parameter. > Add the switch for keeping both the old mode and the new credit-based mode > -- > > Key: FLINK-8458 > URL: https://issues.apache.org/jira/browse/FLINK-8458 > Project: Flink > Issue Type: Sub-task > Components: Network >Reporter: zhijiang >Assignee: zhijiang >Priority: Major > Fix For: 1.5.0 > > > After the whole feature of credit-based flow control is done, we should add a > config parameter to switch on/off the new credit-based mode. To do so, we can > roll back to the old network mode for any expected risks. > The parameter is defined as > {{taskmanager.network.credit-based-flow-control.enabled}} and the default > value is true. This switch may be removed after next release. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8458) Add the switch for keeping both the old mode and the new credit-based mode
[ https://issues.apache.org/jira/browse/FLINK-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16334156#comment-16334156 ] ASF GitHub Bot commented on FLINK-8458: --- Github user alpinegizmo commented on a diff in the pull request: https://github.com/apache/flink/pull/5317#discussion_r162906372 --- Diff: docs/ops/config.md --- @@ -290,6 +290,12 @@ The following parameters configure Flink's JobManager and TaskManagers. - `taskmanager.network.numberOfBuffers` (deprecated, replaced by the three parameters above): The number of buffers available to the network stack. This number determines how many streaming data exchange channels a TaskManager can have at the same time and how well buffered the channels are. If a job is rejected or you get a warning that the system has not enough buffers available, increase this value (DEFAULT: **2048**). If set, it will be mapped to `taskmanager.network.memory.min` and `taskmanager.network.memory.max` based on `taskmanager.memory.segment-size`. +- `taskmanager.network.memory.buffers-per-channel`: Number of network buffers to use for each outgoing/incoming channel (subpartition/input channel). Especially in credit-based flow control mode, it indicates how many credits are exclusive in each input channel. It should be configured at least 2 for good performance. 1 buffer is for receving in-flight data in the subpartition and 1 buffer is for parallel serialization. + +- `taskmanager.network.memory.floating-buffers-per-gate`: Number of extra network buffers to use for each outgoing/incoming gate (result partition/input gate). In credit-based flow control mode, it indicates how many floating credits are shared for all the input channels. The floating buffers are distributed based on backlog (real-time output buffers in the subpartition) feedback. So the floating buffers can help relief back-pressure caused by imbalance data distribution among subpartitions. + --- End diff -- Number of extra network buffers used by each outgoing/incoming gate (result partition/input gate). In credit-based flow control mode, this indicates how many floating credits are shared among all the input channels. The floating buffers are distributed based on backlog (real-time output buffers in the subpartition) feedback, and can help relieve back-pressure caused by unbalanced data distribution among the subpartitions. > Add the switch for keeping both the old mode and the new credit-based mode > -- > > Key: FLINK-8458 > URL: https://issues.apache.org/jira/browse/FLINK-8458 > Project: Flink > Issue Type: Sub-task > Components: Network >Reporter: zhijiang >Assignee: zhijiang >Priority: Major > Fix For: 1.5.0 > > > After the whole feature of credit-based flow control is done, we should add a > config parameter to switch on/off the new credit-based mode. To do so, we can > roll back to the old network mode for any expected risks. > The parameter is defined as > {{taskmanager.network.credit-based-flow-control.enabled}} and the default > value is true. This switch may be removed after next release. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8458) Add the switch for keeping both the old mode and the new credit-based mode
[ https://issues.apache.org/jira/browse/FLINK-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16334157#comment-16334157 ] ASF GitHub Bot commented on FLINK-8458: --- Github user alpinegizmo commented on a diff in the pull request: https://github.com/apache/flink/pull/5317#discussion_r16290 --- Diff: docs/ops/config.md --- @@ -290,6 +290,12 @@ The following parameters configure Flink's JobManager and TaskManagers. - `taskmanager.network.numberOfBuffers` (deprecated, replaced by the three parameters above): The number of buffers available to the network stack. This number determines how many streaming data exchange channels a TaskManager can have at the same time and how well buffered the channels are. If a job is rejected or you get a warning that the system has not enough buffers available, increase this value (DEFAULT: **2048**). If set, it will be mapped to `taskmanager.network.memory.min` and `taskmanager.network.memory.max` based on `taskmanager.memory.segment-size`. +- `taskmanager.network.memory.buffers-per-channel`: Number of network buffers to use for each outgoing/incoming channel (subpartition/input channel). Especially in credit-based flow control mode, it indicates how many credits are exclusive in each input channel. It should be configured at least 2 for good performance. 1 buffer is for receving in-flight data in the subpartition and 1 buffer is for parallel serialization. + --- End diff -- This needs some clarification. Is taskmanager.network.memory.buffers-per-channel only used in credit-based flow control mode? Does choosing a value greater than 2 provide any benefit? > Add the switch for keeping both the old mode and the new credit-based mode > -- > > Key: FLINK-8458 > URL: https://issues.apache.org/jira/browse/FLINK-8458 > Project: Flink > Issue Type: Sub-task > Components: Network >Reporter: zhijiang >Assignee: zhijiang >Priority: Major > Fix For: 1.5.0 > > > After the whole feature of credit-based flow control is done, we should add a > config parameter to switch on/off the new credit-based mode. To do so, we can > roll back to the old network mode for any expected risks. > The parameter is defined as > {{taskmanager.network.credit-based-flow-control.enabled}} and the default > value is true. This switch may be removed after next release. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8458) Add the switch for keeping both the old mode and the new credit-based mode
[ https://issues.apache.org/jira/browse/FLINK-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16332411#comment-16332411 ] ASF GitHub Bot commented on FLINK-8458: --- GitHub user zhijiangW opened a pull request: https://github.com/apache/flink/pull/5317 [FLINK-8458] Add the switch for keeping both the old mode and the new credit-based mode ## What is the purpose of the change *After the whole feature of credit-based flow control is done, we should add a config parameter to switch on/off the new credit-based mode. To do so, we can roll back to the old network mode for any expected risks.* *The parameter is defined as taskmanager.network.credit-based-flow-control.enabled and the default value is true. This switch may be removed after next release.* *This PR is based on #4552 whose commit is also included for passing travis.* ## Brief change log - *Abstract the `NetworkClientHandler` interface for different implementations in two modes* - *Abstract the `NetworkSequenceViewReader` interface for different implementations in two modes* - *Define the `taskmanager.network.credit-based-flow-control.enabled` in `TaskManagerOptions`* ## Verifying this change This change is already covered by existing tests*. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no) - The serializers: (no) - The runtime per-record code paths (performance sensitive): (no) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no) - The S3 file system connector: (no) ## Documentation - Does this pull request introduce a new feature? (no) - If yes, how is the feature documented? (not applicable) You can merge this pull request into a Git repository by running: $ git pull https://github.com/zhijiangW/flink FLINK-8458 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/5317.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5317 commit a3b41f51173adaa382b42877142664d2b09101f7 Author: Zhijiang Date: 2017-09-30T06:36:19Z [FLINK-7456][network] Implement Netty sender incoming pipeline for credit-based commit a8154989f9c93e71e3051d7184c0f02316f1a3c7 Author: Zhijiang Date: 2018-01-17T06:15:04Z [FLINK-8458] Add the switch for keeping both the old mode and the new credit-based mode > Add the switch for keeping both the old mode and the new credit-based mode > -- > > Key: FLINK-8458 > URL: https://issues.apache.org/jira/browse/FLINK-8458 > Project: Flink > Issue Type: Sub-task > Components: Network >Reporter: zhijiang >Assignee: zhijiang >Priority: Major > Fix For: 1.5.0 > > > After the whole feature of credit-based flow control is done, we should add a > config parameter to switch on/off the new credit-based mode. To do so, we can > roll back to the old network mode for any expected risks. > The parameter is defined as > {{taskmanager.network.credit-based-flow-control.enabled}} and the default > value is true. This switch may be removed after next release. -- This message was sent by Atlassian JIRA (v7.6.3#76005)