Hi Alessandro, There was a chat around the compaction recently, https://apachebookkeeper.slack.com/archives/C3KJNCWBU/p1617906924006700?thread_ts=1617301616.002900&cid=C3KJNCWBU so I'll just paste the relevant part and let others clarify details if i got anything wrong:
Compaction works (at least worked a few years ago when I dealt with this) in two modes: a. simple deletion - all ledgers in the entry log already deleted and the EL can be simply deleted b. rewrite and delete. Some data in the EL is needed, BK reads that data, ends up writing it to the new EL file, deletes the old file to release disk space. In case it cannot create the file (and reserve some space for it, IIRC EL size is configured in bk server config and is reserved on EL creation) the compaction fails -> no disk space is freed. You'll have to understand if this is a case. Normally bookie is configured to go into the r/o mode well before such situation happens, thus reserving some space for compaction to work.Compaction logs details about its decisions, free/used space, %% used, whether it will do major or minor compaction etc. Major and minor compaction thresholds control how much compactable space EL should have to be considered for compaction and compaction schedule. I.e. minor - 80% of compactable space, let's rewrite these 20% of usable data into another EL, runs hourly major - 30% of compactable space, let's rewrite these 70%, daily when traffic is low to reduce performance impact EL == entry log Simple deletion (a) happens during both minor and major compaction. In the example above, EL with 20% of compactable space would not be touched by the garbage collection. You can tune the garbage collection thresholds but keep in mind need to maintain balance between freeing space sooner and using more IO for garbage collection (affects latency of write/read requests). Hope this helps. On Tue, Apr 13, 2021 at 1:49 AM Alessandro Luccaroni - Diennea < alessandro.luccar...@diennea.com> wrote: > Hi BookKeepers, > > can you confirm me that the space occupied by deleted ledgers is reclaimed > on BK only during majorCompaction? > > > > If this is the case maybe you we should clarify that in the documentation: > http://bookkeeper.apache.org/docs/4.13.0/reference/config/#entry-log-compaction-settings > > > > Regards, > > *Alessandro Luccaroni* > Platform Manager @ Diennea - MagNews > Tel.: (+39) 0546 066100 Int. 924 - Mob.: (+39) 393 7273519 > Viale G.Marconi 30/14 - 48018 Faenza (RA) - Italy > > > > ------------------------------ > > CONFIDENTIALITY & PRIVACY NOTICE > This e-mail (including any attachments) is strictly confidential and may > also contain privileged information. If you are not the intended recipient > you are not authorised to read, print, save, process or disclose this > message. If you have received this message by mistake, please inform the > sender immediately and destroy this e-mail, its attachments and any copies. > Any use, distribution, reproduction or disclosure by any person other than > the intended recipient is strictly prohibited and the person responsible > may incur in penalties. > The use of this e-mail is only for professional purposes; there is no > guarantee that the correspondence towards this e-mail will be read only by > the recipient, because, under certain circumstances, there may be a need to > access this email by third subjects belonging to the Company. > -- -- Andrey Yegorov
