dlg99 commented on code in PR #3631:
URL: https://github.com/apache/bookkeeper/pull/3631#discussion_r1042762683
##########
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/GarbageCollectorThread.java:
##########
@@ -548,14 +548,24 @@ private boolean removeIfLedgerNotExists(EntryLogMetadata
meta) throws EntryLogMe
@VisibleForTesting
void doCompactEntryLogs(double threshold, long maxTimeMillis) throws
EntryLogMetadataMapException {
LOG.info("Do compaction to compact those files lower than {}",
threshold);
+ double tmpGcEntryLogSizeRatio = conf.getGcEntryLogSizeRatio() <= 0 ? 0
: conf.getGcEntryLogSizeRatio();
+ final double gcEntryLogSizeRatio = tmpGcEntryLogSizeRatio > 1 ? 1.0 :
tmpGcEntryLogSizeRatio;
+ if (gcEntryLogSizeRatio > 0.5) {
+ LOG.warn("Configured gcEntryLogSizeRatio: {}, updated
gcEntryLogSizeRatio: {} gratter than 0.5, "
+ + "which means any entryLog file size less than {} will be
compacted and it will bring heavy pressure "
+ + "on garbage collection.", conf.getGcEntryLogSizeRatio(),
gcEntryLogSizeRatio,
+ gcEntryLogSizeRatio * conf.getEntryLogSizeLimit());
+ }
final int numBuckets = 10;
int[] entryLogUsageBuckets = new int[numBuckets];
int[] compactedBuckets = new int[numBuckets];
ArrayList<LinkedList<Long>> compactableBuckets = new
ArrayList<>(numBuckets);
+ ArrayList<LinkedList<Long>> smallFilesCompactableBuckets = new
ArrayList<>(numBuckets);
Review Comment:
I think you can avoid addition of another type of buckets.
Have something like useTargetEntryLogSizeForGc config;
Replace
```java
calculateUsageIndex(numBuckets, meta.getUsage());
```
with something like
```java
double usage = meta.getUsage();
if (conf.getUseTargetEntryLogSizeForGc()) {
usage = (double) meta.getRemainingSize() / Math.max(meta.getTotalSize(),
conf.getEntryLogSizeLimit());
}
```
and the rest will happen naturally.
As an improvement, transactional compaction can fallback to a regular one if
the estimated size of entry log after compaction is below some limit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
