Hi everyone, We’ve released a new version of Cobble-Flink [1], adding async state support for Flink 2.x.
For those unfamiliar with the project, Cobble-Flink is a unified storage layer for Apache Flink, bringing together a state backend and SQL source/sink connectors. Our goal is to make both streaming state and sink output readily available for fast sharing, querying, and reuse across jobs, while improving performance and preserving the familiar, native Flink experience. Some highlights: - *Performant stateful streaming*: up to 2x throughput in Nexmark and 2.2x throughput in the Flink state benchmark, compared with RocksDB. - *Disaggregated state*: available starting from *Flink 1.17*, helping reduce resource usage and speed up restarts, with remote compaction to further offload CPU work from processing jobs. - *Source and sink support*: store intermediate results, and scan or look up persisted state and sink output from other Flink jobs. - *Web Monitor*: inspect records, explore state by column, and track changes across snapshots. - *New async state support for Flink 2.x.* These capabilities are powered by Cobble [2], an embedded key-value storage engine designed for distributed systems, with flexible local and remote storage, hot/cold data tiering, and remote compaction. Cobble’s use extends beyond Flink, and we welcome anyone interested in exploring integrations with other engines and applications. You can get started by adding the Cobble-Flink dependency—see the documentation [3] for setup instructions. A short introduction video [4] walks through the main capabilities. The project is evolving quickly. We welcome anyone interested to give it a try and share feedback, questions, or suggestions for improvements. Best, Zakelly [1] https://github.com/cobble-project/cobble-flink [2] https://github.com/cobble-project/cobble [3] https://cobble-project.github.io/cobble-flink/latest/ [4] https://github.com/user-attachments/assets/09f4e0fc-4605-497e-9161-9a69f92dab5c
