keith-turner commented on code in PR #5026:
URL: https://github.com/apache/accumulo/pull/5026#discussion_r1824784852
##########
core/src/main/java/org/apache/accumulo/core/util/compaction/CompactionJobPrioritizer.java:
##########
@@ -44,6 +49,11 @@ public static short createPriority(CompactionKind kind, int
totalFiles, int comp
return (short) prio;
case SELECTOR:
case SYSTEM:
+ // Given that tablets with too many files cause several problems,
+ // boost their priority to the maximum allowed value.
+ if (condition == Condition.TABLET_OVER_SIZE) {
+ return Short.MAX_VALUE;
Review Comment:
Always returning MAX_VALUES does not differentiate between a tablet that is
1 over the max files and one that is 300 over. Would be nice to prioritize
within this group. One possible way to do this is the following.
* instead of passing in the TABLET_OVER_SIZE condition to this function
pass in the maxFilePerTablet setting
* create a dedicated priority range for tablets over size, like from
Short.MAX-2000 to Short.MAX.
* adjust the priotiy range of user compactions top out at Short.MAX-2000
```java
if(totalFiles > maxFilesPerTablet && kind == SYSTEM) {
int numberOfFilesOverMax = totalFiles - maxFilesPerTablet;
int prio = Shot.MAX-2000+ numberOfFilesOverMax+compactingFiles;
if(prio>Short.MAX){prio = Short.MAX);
}
```
This will get the worst offenders scheduled to run first. These changes
would easier in later versions of Accumulo where the CompactionJobPrioritizer
already has multiple ranges to prioritize root and metadata table. Maybe those
changes should be backported first, not sure if there was a reason they were
not done in 2.1. Even if those changes are not backported, the 2.1 code could
be adapted to follow that pattern to support more than two priority groups.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
