keith-turner commented on issue #4454:
URL: https://github.com/apache/accumulo/issues/4454#issuecomment-2065607462
Below are some code pointers for the code.
1. This
[code](https://github.com/apache/accumulo/blob/cb25f26ab3be77dbfce54e9d1e7234e5974510ec/server/manager/src/main/java/org/apache/accumulo/manager/tableOps/compact/CompactionDriver.java#L190)
selects files for user compactions, creating the selected column.
2. This
[code](https://github.com/apache/accumulo/blob/cb25f26ab3be77dbfce54e9d1e7234e5974510ec/server/base/src/main/java/org/apache/accumulo/server/compaction/CompactionJobGenerator.java#L92)
creates compaction jobs that are queued. If we add a count to the selected
files column, then this code could generate system jobs when that count is zero
(and no user jobs are running) in addition to user jobs.
3. This
[code](https://github.com/apache/accumulo/blob/cb25f26ab3be77dbfce54e9d1e7234e5974510ec/server/manager/src/main/java/org/apache/accumulo/manager/compaction/coordinator/CompactionCoordinator.java#L531)
creates a running compaction entry in the metadata table when a compaction
starts. This code could allow system compactions to start if selected files
exists and its count is zero (and no user jobs are running), for this case
would need to remove the selected files column.
4. This
[code](https://github.com/apache/accumulo/blob/cb25f26ab3be77dbfce54e9d1e7234e5974510ec/server/manager/src/main/java/org/apache/accumulo/manager/compaction/coordinator/CompactionCoordinator.java#L726)
commits a compaction. It will modify the selected files column, removing the
compacted files for user compactions. It will also update the tablets files
and remove the running compaction entry. This code could increment the count
in the selected files column.
We could probably drop the timeout in elasticity for now and replace it with
a log messages indicating a system compaction removed the selected files of a
user compaction. This would allow detection of the case where a system
compaction is starving a user compaction. This logging would be done by the
code at number 3 above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
