keith-turner opened a new issue, #4505:
URL: https://github.com/apache/accumulo/issues/4505
**Is your feature request related to a problem? Please describe.**
Write ahead log recovery can take a while because of the following two
behaviors.
* Tablet servers processes only do a single log recovery at time
* All tablets, even if they have no data in the write ahead log, will go
through the log recovery process when being loaded on a tablet.
Those behaviors make log recovery times correlate with the number of tablets
per tserver. So as the number of tablets per tserver increases, log recovery
time increases.
**Describe the solution you'd like**
Allow parallel log recovery and faster log recovery. The parallelism is
related to #4429, but that change does not completely solve the issue as the
lock is still acquired for log recovery.
* Use a cache during log recovery when reading from sorted walog rfiles
* Inspect tablet w/ logs before acquiring recovery lock to see if they
contain data
**Describe alternatives you've considered**
Could potentially produce an F file for log recovery outside of the tablet
server somewhere (similar to external compactions). This may have been
discussed on an elasticity related issue, but could not find it. This would be
a much larger change and probably would be suitable to do in 2.1. It may
require completly refactoring the tablet minor compaction code to make it
usable elsewhere.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
