EdColeman commented on issue #4318: URL: https://github.com/apache/accumulo/issues/4318#issuecomment-1970149852
The metadata is periodically flushed. If compactions do not run, then the number of files will continue to grow. Keeping the number of files to a minimum would reduce potential impact to metadata scan performance as well as reduce recovery time. If the number of files increased unbounded, then that could stress things to the point where metadata recovery may have issues? Ideally any error conditions are currently handled, but those paths may not be as robust as we assume. This is speculative, but we currently strive to keep the metadata compacted to maintain performance. If we are not compacting regularly, then things could be pushed to a tipping point that we can prevent if we have some guarantee that metadata compactions are occurring, loudly calling attention if they are not, and as a last step, take some action that places a bound on how bad we let things get before we get pushed into uncharted territory. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
