davidradl commented on code in PR #860: URL: https://github.com/apache/flink-web/pull/860#discussion_r3412006679
########## docs/content/posts/2026-06-14-announcing-native-s3-fs.md: ########## @@ -0,0 +1,200 @@ +--- +title: "Introducing Flink's Native S3 FileSystem: Built for Performance, Designed for Production" +date: "2026-06-14T08:00:00.000Z" +slug: "announcing-native-s3-fs" +url: "/2026/06/14/announcing-native-s3-fs/" +authors: +- gabor: + name: "Gabor Somogyi" +- samrat: + name: "Samrat Deb" +aliases: +- /news/2026/06/14/announcing-native-s3-fs.html +--- + +Apache Flink relies on the underlying filesystem for much of its work: reading and writing application data, materializing streaming sinks, and storing checkpoints and savepoints for recovery. For years, S3 support in Flink meant choosing between two Hadoop-based plugins, each with its own trade-offs and configuration quirks. With Flink 2.3, there is a better option. + +Today we're introducing `flink-s3-fs-native`, A ground-up, Hadoop-free S3 filesystem built specifically for Flink. It ships as an experimental opt-in plugin in Flink 2.3, is already running in production at scale at major technology companies, and delivers measurable, reproducible performance gains. + + +**At a glance** + +| | | +|---|---| +| **~2x faster checkpoints** | 48.8 s average vs 90.1 s with the Presto plugin; up to 4.5x at small state sizes | +| **Drop-in replacement** | Swap the JAR, keep your existing `flink-conf.yaml`, restart your cluster | +| **No Hadoop dependency** | ~13 MB JAR vs ~30–93 MB; no CVE triage on Hadoop transitive dependencies | +| **AWS SDK v2** | Async-first I/O; AWS SDK v1 entered maintenance mode December 2025 | +| **One plugin for everything** | Exactly-once sinks and fast checkpoints — no trade-offs, no compromises | Review Comment: I thought there were some trade offs - have these been all addressed now? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
