tillrohrmann commented on a change in pull request #10462:
[FLINK-15031][runtime] Calculate required shuffle memory before allocating
slots if resources are specified
URL: https://github.com/apache/flink/pull/10462#discussion_r354855860
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/shuffle/NettyShuffleMaster.java
##########
@@ -32,12 +35,18 @@
*/
public class NettyShuffleMaster implements
ShuffleMaster<NettyShuffleDescriptor> {
+ private final int networkBuffersPerInputChannel;
+
+ private final int networkBufferSize;
+
@VisibleForTesting
public NettyShuffleMaster() {
this(new Configuration());
}
public NettyShuffleMaster(final Configuration configuration) {
+ networkBuffersPerInputChannel =
configuration.getInteger(NettyShuffleEnvironmentOptions.NETWORK_BUFFERS_PER_CHANNEL);
+ networkBufferSize =
ConfigurationParserUtils.getPageSize(configuration);
Review comment:
What we could do is to introduce two static methods `int
getNetworkBuffersPerInputChannel(Configuration)` and `int
getNetworkBufferSize(Configuration)` which we use here and in the
`NettyShuffleEnvironmentConfiguration.fromConfiguration()`. That way there is
only a single place where we access the `NETWORK_BUFFERS_PER_CHANNEL` config
constant.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
