Hi ConradJam, Thanks for taking a look at the FRocksDB project.
>From my point of view, the reasons why Flink community would not upgrade the >FRocksDB too frequently are mainly due to: 1. We have suffered from the performance regression of new versions of RocksDB compared with current versions[1]. Unfortunately, the RocksDB community (or we can say the RocksDB team at Meta Inc) does not have clear benchmark testing when releasing the new version each time. I spent some time to figure out why RocksDB-6.20.x behaves worse than RocksDB-5.17, however, we just found one major performance regression reason [2] and cannot advance more. That's why Flink community launches an ANNOUNCE of this problem [3]. 2. Releasing RocksDB-java binaries is really a bit complicated as RocksDB has supported more and more platforms (linux32/64, windows, apple silicon, arm, ppc64, s390). However, we don't have so many environments to release these binaries within one jar package. We even considered launching discussions to drop some platform support. Thus, if we don't have strong demands, we would not upgrade the FRocksDB too frequently. BTW, we aim to make the FRocksDB could run on Apple silicon chips in Flink-1.17 [4] Last but not least, RocksDB actually has an apache-2.0 license. Apart from the possible license problem, FRocksDB [5] was created in 2019, in which the community does have so many Flink sub-projects yet. [1] https://issues.apache.org/jira/browse/FLINK-14482 [2] https://issues.apache.org/jira/browse/FLINK-19710 [3] https://lists.apache.org/thread/v9w4dm6wdqgn5b6jqjdrzfycxsdh38vj [4] https://issues.apache.org/jira/browse/FLINK-24932 [5] https://issues.apache.org/jira/browse/FLINK-10471 Best Yun Tang ________________________________ From: Zheng Yu Chen <jam.gz...@gmail.com> Sent: Tuesday, November 22, 2022 17:02 To: dev@flink.apache.org <dev@flink.apache.org> Subject: Re: FRocksDB Some Question Thank you for your reply. I will join the discussion after careful review Yanfei Lei <fredia...@gmail.com> 于2022年11月22日周二 12:20写道： > Hi ConradJam, > > Thanks for bringing this up. > In fact, @Sebastian has tried to update RocksDB to 6.29.5[1], but there are > some regressions on MapState[2]. If you want to use the newest feature of > RocksDB, I think you can refer to this release document[3] to release your > own version. > > > I would like to ask if there is any chance for this project to be moved > to the Apache Flink sub-project to allow more people to participate? > IMO, the open-source licenses of RocksDB and Apache Flink are different, > moving FRocksDB to the Apache Flink sub-project may be hindered. > > [1] > > https://issues.apache.org/jira/browse/FLINK-24932?focusedCommentId=17560179&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17560179 > > [2] > > https://issues.apache.org/jira/browse/FLINK-24932?focusedCommentId=17557331&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17557331 > [3] > > https://github.com/ververica/frocksdb/blob/FRocksDB-6.20.3/FROCKSDB-RELEASE.md > > > Zheng Yu Chen <jam.gz...@gmail.com> 于2022年11月22日周二 11:40写道： > > > Hi community, I want to ask a question about frocksdb > > > > I noticed that frocksdb seems to have been inactive for a long time. > > flink-statebackend-rocksdb is depend > https://github.com/ververica/frocksdb > > to run state backend > > This Rocksdb is stuck at 6.20.3.But the current upstream version > > https://github.com/facebook/rocksdb is already at 7.7.3.As the basic > state > > backend of Flink, this part > > seems to be ignored by people (although users cannot directly perceive > it) > > > > I don't know how often this library is maintained and updated and the > > differences between them and Rocksdb. It would be best if there are > > relevant documents to tell me > > > > If the community is willing to upgrade this underlying component, I am > > willing to assist in this work as one of the volunteers. > > At the same time, I would like to ask if there is any chance for this > > project to be moved to the Apache Flink sub-project to allow more people > to > > participate? > > > > > > > > -- > > Best > > > > ConradJam > > > > > -- > Best, > Yanfei > -- Best ConradJam
