Hi Ordencia, S3 is getting more trending as before and that's the reason why the community is putting quite some efforts into it. There are megatrends in open source, one of them is to break up with Hadoop dependencies. A good example here is that the new native S3 connector has zero Hadoop dependencies. It's important to say it out that it doesn't mean HDFS support is intended to be dropped in the near future since still used heavily.
All in all if you choose from these two and works for your use-case then you've made good decision🙂 Of course both of them has certain constraints and trade-offs which should be considered on your side... BR, G On Tue, Aug 11, 2026 at 11:46 PM Ordencia Wu <[email protected]> wrote: > Hi all, > > Just wanted to give this a quick bump, the main question I’m hoping to > understand is whether the community expects Flink to remain storage system > neutral (especially between s3 and hdfs) for the use case of durable > checkpoint and state storage? This question is inspired by the recent S3 > native plugin and disaggregated state work since it’s so far unclear > whether these roadmaps reflect a longer-term preference toward object > storage for new deployments. > > Any pointers to relevant discussions or FLIPs would be helpful! > > Thank you, > Ordencia > > On 2026/08/08 19:00:12 Ordencia Wu wrote: > > Hi all! I am selecting the durable checkpoint store for a multi-tenant > Flink platform. It appears recent introduction of `flink-s3-fs-native` > brought significant improvements to checkpointing on S3, and FLIP-555 [1] > describes a path towards making it the recommended S3 implementation. I am > weighing between S3 and HDFS as my options, and was wondering if the recent > improvements to Flink’s S3 integration and performance would change the > community’s guidance for durable checkpoint storage in new deployments, > particularly on Kubernetes. Any thoughts on the following would be very > helpful: > > > > 1. Does the community expect S3/object storage to become the preferred > durable checkpoint storage model for new deployments, or is the intent for > S3 and HDFS to remain equally first-class options? > > 2. Are there planned FLIPs or architectural directions around > checkpointing, recovery, or disaggregated state management that are likely > to favor S3/object-storage characteristics over HDFS > > 3. If both are expected to remain equally supported, what factors would > you recommend using to choose between remote HDFS and S3 for checkpoint > storage? > > > > For context, the platform supports a range of different Flink workloads: > > 1. Jobs on the platform range from stateless to 100GB+ in state size > > 2. SLOs vary greatly across tenants and the jobs could have different > checkpoint intervals ranging from 10s to 10m > > > > Any thoughts or pointers to related discussions would be greatly > appreciated! > > > > Best, > > Ordencia > > > > ——————————————————— > > [1] > https://cwiki.apache.org/confluence/spaces/FLINK/pages/396790457/FLIP-555+Flink+Native+S3+FileSystem > > > > > > > >
