This is indeed an issue at the moment. Personally, I haven't found a proper way to migrate data from LevelDB to RocksDB, as their storage structures are different. Should we wait until a reasonable migration solution becomes available before moving forward with this?
Jungtaek Lim <kabhwan.opensou...@gmail.com> 于2025年5月28日周三 15:41写道： > > Thanks for initiating this. > > I wonder if we don't have any compatibility issue on every component - SS > area does not have an issue, but I don't quite remember if the history server > would be OK with this. What is the story of the migration if they had been > using leveldb? I guess it could be probably re-parsed, but do we need to ask > users to perform some manual work to do that? > > On Wed, May 28, 2025 at 2:27 PM Yang Jie <yangji...@apache.org> wrote: >> >> The project "org.fusesource.leveldbjni:leveldbjni" released its last version >> 12 years ago, and its code repository was last updated 8 years ago. >> Consequently, I believe it's challenging for us to receive ongoing >> maintenance and support from this project. >> >> On the flip side, when developers implement new features related to Spark >> code, they have become accustomed to using rocksdb instead of leveldb. >> >> Furthermore, in Spark 4.0, support for leveldb was deprecated, and the >> default implementation of the corresponding functionality was switched to >> rocksdb. >> >> Given these factors, I support discontinuing support for leveldb. >> >> >> Thanks >> JIe Yang >> >> On 2025/05/27 08:26:06 Jia Fan wrote: >> > Hi all, >> > >> > I'd like to start a discussion about removing LevelDB support from Apache >> > Spark. >> > >> > As noted in SPARK-44223(https://issues.apache.org/jira/browse/SPARK-44223), >> > LevelDB support was deprecated in Spark 4.0. It’s no longer actively >> > maintained or widely used, and continuing to support it brings >> > unnecessary maintenance and dependency complexity. >> > >> > A PR has been opened here to remove it entirely: >> > https://github.com/apache/spark/pull/51027 >> > >> > WDYT? >> > >> > Best regards, >> > Jia Fan >> > >> > --------------------------------------------------------------------- >> > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org >> > >> > >> >> --------------------------------------------------------------------- >> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org >> --------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
