There are still many optimisers in PROD environments that have rocksDB storage enabled. Removing dependencies in projects is acceptable, but we should also provide documentation and description of what to do if users want to continue using the feature. For example, there could be support for users to add dependencies individually, etc.
Jinsong Zhou <jinsongz...@apache.org> 于2024年7月8日周一 17:36写道： > Hi devs, > > Recently, I have been working on reducing the size of the Amoro > installation package. Considering the Amoro installation package is almost > 1GB in size, this task really should be done ASAP. > > I found the largest dependent of Amoro is the rocksdb lib (more than 50MB). > It is used to cache some data to disk storage when the memory is not > enough. It is originally used to cache iceberg delete records in > optimizers. But when we have improved the delete records caching with bloom > filter, this feature is really not needed anymore. > > So I am considering removing the rocksdb dependencies from the project to > reduce the installation package size. > > I am looking forward to hearing any point from anyone regarding this > issue. > > Best regards, > Jinsong >
