featzhang commented on code in PR #27568:
URL: https://github.com/apache/flink/pull/27568#discussion_r3121306667
##########
flink-models/flink-model-triton/src/main/java/org/apache/flink/model/triton/TritonOptions.java:
##########
@@ -165,4 +165,89 @@ private TritonOptions() {
+ "Example: %s",
code("'X-Custom-Header:value,X-Another:value2'"))
.build());
+
+ // ========== Connection Pool Management Options ==========
+
+ public static final ConfigOption<Integer> CONNECTION_POOL_MAX_IDLE =
+ ConfigOptions.key("connection-pool-max-idle")
+ .intType()
+ .defaultValue(20)
+ .withDescription(
+ Description.builder()
+ .text(
+ "Maximum number of idle
connections to keep in the pool. "
+ + "Higher values reduce
connection setup overhead but consume more memory. "
+ + "Recommended: 10-50
depending on parallelism and QPS. "
+ + "Defaults to 20.")
+ .build());
+
+ public static final ConfigOption<Duration> CONNECTION_POOL_KEEP_ALIVE =
+ ConfigOptions.key("connection-pool-keep-alive")
+ .durationType()
+ .defaultValue(Duration.ofSeconds(300))
+ .withDescription(
+ Description.builder()
+ .text(
+ "Duration to keep idle connections
alive in the pool before eviction. "
+ + "Longer durations reduce
connection setup overhead but may keep stale connections. "
+ + "Recommended: 60s-600s. "
+ + "Defaults to 300s (5
minutes).")
+ .build());
+
+ public static final ConfigOption<Integer> CONNECTION_POOL_MAX_TOTAL =
+ ConfigOptions.key("connection-pool-max-total")
+ .intType()
+ .defaultValue(100)
+ .withDescription(
+ Description.builder()
+ .text(
+ "Maximum total number of
connections across all routes. "
+ + "This limits the overall
number of concurrent connections. "
+ + "Should be >=
max-concurrent-requests to avoid blocking. "
Review Comment:
This is a boolean ConfigOption with a default value of true. The validation
is implicit in the code logic in TritonUtils.java where
`poolConfig.reuseEnabled` is checked to determine whether to enable connection
pooling (maxIdleConnections > 0) or disable it (maxIdleConnections = 0).
##########
flink-models/flink-model-triton/src/main/java/org/apache/flink/model/triton/TritonOptions.java:
##########
@@ -165,4 +165,89 @@ private TritonOptions() {
+ "Example: %s",
code("'X-Custom-Header:value,X-Another:value2'"))
.build());
+
+ // ========== Connection Pool Management Options ==========
+
+ public static final ConfigOption<Integer> CONNECTION_POOL_MAX_IDLE =
+ ConfigOptions.key("connection-pool-max-idle")
+ .intType()
+ .defaultValue(20)
+ .withDescription(
+ Description.builder()
+ .text(
+ "Maximum number of idle
connections to keep in the pool. "
+ + "Higher values reduce
connection setup overhead but consume more memory. "
+ + "Recommended: 10-50
depending on parallelism and QPS. "
+ + "Defaults to 20.")
+ .build());
+
+ public static final ConfigOption<Duration> CONNECTION_POOL_KEEP_ALIVE =
+ ConfigOptions.key("connection-pool-keep-alive")
+ .durationType()
+ .defaultValue(Duration.ofSeconds(300))
+ .withDescription(
+ Description.builder()
+ .text(
+ "Duration to keep idle connections
alive in the pool before eviction. "
+ + "Longer durations reduce
connection setup overhead but may keep stale connections. "
+ + "Recommended: 60s-600s. "
+ + "Defaults to 300s (5
minutes).")
+ .build());
+
+ public static final ConfigOption<Integer> CONNECTION_POOL_MAX_TOTAL =
+ ConfigOptions.key("connection-pool-max-total")
+ .intType()
+ .defaultValue(100)
+ .withDescription(
+ Description.builder()
+ .text(
+ "Maximum total number of
connections across all routes. "
+ + "This limits the overall
number of concurrent connections. "
+ + "Should be >=
max-concurrent-requests to avoid blocking. "
+ + "Recommended: 50-200
depending on expected load. "
+ + "Defaults to 100.")
+ .build());
+
+ public static final ConfigOption<Duration> CONNECTION_TIMEOUT =
+ ConfigOptions.key("connection-timeout")
+ .durationType()
+ .defaultValue(Duration.ofSeconds(10))
+ .withDescription(
+ Description.builder()
+ .text(
+ "Timeout for establishing a new
connection to Triton server. "
+ + "Shorter timeouts fail
fast but may cause false negatives on slow networks. "
+ + "Should be less than
overall %s. "
Review Comment:
✅ Fixed: Updated documentation to emphasize that connection-timeout must be
less than the overall request timeout to avoid conflicts. Updated in commit
3eb6e7b7f1a.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
