tillrohrmann commented on a change in pull request #8484: [FLINK-12547] Add
connection and socket timeouts for the blob client
URL: https://github.com/apache/flink/pull/8484#discussion_r288022559
##########
File path:
flink-core/src/main/java/org/apache/flink/configuration/BlobServerOptions.java
##########
@@ -102,4 +102,20 @@
public static final ConfigOption<Integer> OFFLOAD_MINSIZE =
key("blob.offload.minsize")
.defaultValue(1_024 * 1_024) // 1MiB by default
.withDescription("The minimum size for messages to be offloaded
to the BlobServer.");
+
+ /**
+ * The socket timeout in milliseconds for the blob client.
+ */
+ public static final ConfigOption<Integer> SO_TIMEOUT =
+ key("blob.client.socket.timeout")
+ .defaultValue(120_000)
Review comment:
The reason I'm asking is because we might break existing setups with this
default value. Imagine you have a really large setup (let's say 100 TMs) and
all of them need to download a large set of blobs. Since the `BlobServer` only
serves 50 connections concurrently, it might be the case that 50 others will
time out if the transmission takes longer than 2 minutes. I think the Netty
case is a bit different because we don't limit the number of concurrent
transactions. So concretely, I'm wondering whether we should set the default
value to `0` to maintain the existing behaviour or to something a bit larger
(e.g. 10 minutes). Or alternatively, we might add a bit more meaningful error
message allowing users to pin point the problem and what configuration value to
increase.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
