Thank you @[email protected] <[email protected]> Yes, This totally aligned with making native-s3-fs mainline along with existing connectors. Looking forward to hearing from the community to move forward.
Bests, Samrat On Tue, Sep 8, 2026 at 2:22 PM Gabor Somogyi <[email protected]> wrote: > Hey Samrat, > > +1 to promote in short. > > I personally think that one to one feature parity was not a target. > Hadoop is a general purpose FS with huge amount of tuning possibilities > however the new native S3 connector is specific to our Flink needs > with smaller amount of tuning. > > The feature set which allows users to use it in production is there > both from data processing and checkpointing perspective. > It's used in many production pipelines already so from my point of view > we can promote it as the preferred way. It's important to say it out > that the old connectors must remain from code and documentation > perspective but as a fallback if something can't be done with the new > native s3 connector. > > After the promotion we should gather experience from the community, fill > those > gaps if there are, fix bugs and that is going to allow us to sunset the old > connectors. I'm pretty sure that won't be lightning fast since we should > allow everybody time to do the migration and express the missing > parts/bugfixes. > > Happy to hear other voices. > > BR, > G > > > On Mon, Sep 7, 2026 at 7:25 AM Samrat Deb <[email protected]> wrote: > > > Hi All, > > > > flink-s3-fs-native (the AWS-SDK-v2, Hadoop-free S3 filesystem for s3:// > and > > s3a://) landed experimentally in Flink 2.3[1]. Since then, the [Phase 2] > > Native S3 Filesystem FLINK-39109 [2] has closed 19 subtasks, closing the > > feature gap compared to flink-s3-fs-hadoop or flink-s3-fs-presto. > > > > Proposal > > > > 1. promote NativeS3FileSystem from Experimental to mainstream in an > > upcoming 2.4 release[3]. What is the community view and overall > consensus? > > > > 2. If you run flink-s3-fs-hadoop or flink-s3-fs-presto in production, > > please reply with anything you rely on that flink-s3-fs-native doesn't > > cover yet. I've checked all three plugins' sources for the above, but > > deployment-specific setups (custom credential providers, particular > > S3-compatible endpoints, etc.) may surface gaps I've missed. > > > > Bests, > > Samrat > > > > [1] > > > > > https://cwiki.apache.org/confluence/spaces/FLINK/pages/396790457/FLIP-555+Flink+Native+S3+FileSystem > > [2] https://issues.apache.org/jira/browse/FLINK-39109 > > [3] https://lists.apache.org/thread/14vym623dh08rry7jsxj2sm6ty4m6cho > > >
