Thanks for initiating this. I wonder if we don't have any compatibility issue on every component - SS area does not have an issue, but I don't quite remember if the history server would be OK with this. What is the story of the migration if they had been using leveldb? I guess it could be probably re-parsed, but do we need to ask users to perform some manual work to do that?
On Wed, May 28, 2025 at 2:27 PM Yang Jie <yangji...@apache.org> wrote: > The project "org.fusesource.leveldbjni:leveldbjni" released its last > version 12 years ago, and its code repository was last updated 8 years ago. > Consequently, I believe it's challenging for us to receive ongoing > maintenance and support from this project. > > On the flip side, when developers implement new features related to Spark > code, they have become accustomed to using rocksdb instead of leveldb. > > Furthermore, in Spark 4.0, support for leveldb was deprecated, and the > default implementation of the corresponding functionality was switched to > rocksdb. > > Given these factors, I support discontinuing support for leveldb. > > > Thanks > JIe Yang > > On 2025/05/27 08:26:06 Jia Fan wrote: > > Hi all, > > > > I'd like to start a discussion about removing LevelDB support from > Apache Spark. > > > > As noted in SPARK-44223( > https://issues.apache.org/jira/browse/SPARK-44223), > > LevelDB support was deprecated in Spark 4.0. It’s no longer actively > > maintained or widely used, and continuing to support it brings > > unnecessary maintenance and dependency complexity. > > > > A PR has been opened here to remove it entirely: > > https://github.com/apache/spark/pull/51027 > > > > WDYT? > > > > Best regards, > > Jia Fan > > > > --------------------------------------------------------------------- > > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > > > > > > --------------------------------------------------------------------- > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > >
