And I found some previous discussion, FYI: 1. https://issues.apache.org/jira/browse/FLINK-3035 2. https://www.mail-archive.com/dev@flink.apache.org/msg10666.html
Hope this helps. Best, Zakelly On Tue, Jan 30, 2024 at 4:08 PM Zakelly Lan <zakelly....@gmail.com> wrote: > Hi Chirag > > That's an interesting idea. IIUC, storing key-values can be simply > implemented for Redis, but supporting checkpoint and recovery is relatively > challenging. Flink's checkpoint should be consistent among all stateful > operators at the same time. For an *embedded* and *file-based* key value > store like RocksDB, it is easier to implement by uploading files of > specific time asynchronously. > > Moreover if you want to store your state basically in memory, then why not > using the HashMapStateBackend. It saves the overhead of serialization and > deserialization and may achieve better performance compared with Redis I > guess. > > > Best, > Zakelly > > On Tue, Jan 30, 2024 at 2:15 PM Chirag Dewan via user < > user@flink.apache.org> wrote: > >> Hi, >> >> I was looking at the FLIP-254: Redis Streams Connector and I was >> wondering if Flink ever considered Redis as a state backend? And if yes, >> why was it discarded compared to RocksDB? >> >> If someone can point me towards any deep dives on why RocksDB is a better >> fit as a state backend, it would be helpful. >> >> Thanks, >> Chirag >> >
