Samrat002 commented on code in PR #28136:
URL: https://github.com/apache/flink/pull/28136#discussion_r3310703542
##########
flink-filesystems/flink-s3-fs-native/src/main/java/org/apache/flink/fs/s3native/NativeS3FileSystemFactory.java:
##########
@@ -298,6 +298,33 @@ public class NativeS3FileSystemFactory implements
FileSystemFactory {
+ "When not set, the default chain is
used: delegation tokens -> "
+ "static credentials (if configured) ->
DefaultCredentialsProvider.");
+ public static final ConfigOption<Boolean> CRT_ENABLED =
+ ConfigOptions.key("s3.crt.enabled")
+ .booleanType()
+ .defaultValue(false)
+ .withDescription(
+ "Enable AWS Common Runtime (CRT) HTTP transport. "
+ + "When true, uses AwsCrtHttpClient for
sync S3 operations and "
+ + "S3AsyncClient.crtBuilder() for
async/transfer operations, "
+ + "providing higher throughput for large
S3 transfers. "
+ + "Requires aws-crt-client and aws-crt
JARs in the plugin directory "
+ + "(they are not bundled in the fat JAR
due to JNI shading constraints).");
+
+ public static final ConfigOption<Double> CRT_TARGET_THROUGHPUT_GBPS =
+ ConfigOptions.key("s3.crt.target-throughput-gbps")
Review Comment:
Yes. `targetThroughputInGbps` is exactly how the CRT API abstracts away
internal parallelism: the SDK uses it to size its thread pool, connection
count, and multi-part parallelism internally. There is no separate public API
for the number of upload threads in S3CrtAsyncClientBuilder. Exposing an
additional Flink-level concurrency knob would require us to reverse-engineer
that relationship (throughput ÷ per-connection bandwidth ≈ threads).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
