Hi, Thanks for the input from xuba. Yes, Indeed, at this stage, we may still need some methods to allow users to add support for RocksDB when needed. However, we can consider removing it from the default installation package.
In my opinion, there are two possible methods： 1. The first one is to add a Maven profile related to RocksDB, allowing users to manually use this profile to build the project and enable this feature when needed. 2. The second method is to provide a bundled package for RocksDB, allowing it to be dynamically added at runtime. Method one is much easier to implement and we should implement it first, and implement method two when we needed later. What do you think? Best, Jinsong On Tue, Jul 9, 2024 at 11:09 AM Xavier Bai <x...@apache.org> wrote: > There are still many optimisers in PROD environments that have rocksDB > storage enabled. Removing dependencies in projects is acceptable, but we > should also provide documentation and description of what to do if users > want to continue using the feature. For example, there could be support for > users to add dependencies individually, etc. > > Jinsong Zhou <jinsongz...@apache.org> 于2024年7月8日周一 17:36写道： > > > Hi devs, > > > > Recently, I have been working on reducing the size of the Amoro > > installation package. Considering the Amoro installation package is > almost > > 1GB in size, this task really should be done ASAP. > > > > I found the largest dependent of Amoro is the rocksdb lib (more than > 50MB). > > It is used to cache some data to disk storage when the memory is not > > enough. It is originally used to cache iceberg delete records in > > optimizers. But when we have improved the delete records caching with > bloom > > filter, this feature is really not needed anymore. > > > > So I am considering removing the rocksdb dependencies from the project to > > reduce the installation package size. > > > > I am looking forward to hearing any point from anyone regarding this > > issue. > > > > Best regards, > > Jinsong > > >
