mauricebarnum opened a new pull request #2932: URL: https://github.com/apache/bookkeeper/pull/2932
Descriptions of the changes in this PR: ### Motivation BookKeeper's entry log writing is buffered in application code before being submitted to the operating system, which by will buffer it again. The cost of this "double buffering" becomes a limiting write throughput factor. This set of changes adds optional support to bypass the operating system buffering on supported systems (currently Linux and MacOS) by using the open(2) flag O_DIRECT. fallocate(2) is used, if available, to request that the filesystem allocate the required space before data is written. Access to the I/O system calls is via a JNI binding included in bookkeeper/native-io ### Changes - Refactor EntryLogger by introducing an interface, EntryLoggerIface. The direct-io entry logger will implement this interface. - Add package bookkeeper-slogger to provide support for structured logging with a pluggable logging backend. Provide an implementation using SLF4J. - Add native-io package to providing JNI bindings to operating system I/O api. - Refactor garbage collection and compaction to allow the entry logger to control which files are available to be garbage collected. - Add EventLoggerIface implementation DirectEntryLogger, enabled with `dbStorage_directIOEntryLogger=true` > --- > In order to uphold a high standard for quality for code contributions, Apache BookKeeper runs various precommit > checks for pull requests. A pull request can only be merged when it passes precommit checks. > > --- > Be sure to do all of the following to help us incorporate your contribution > quickly and easily: > > If this PR is a BookKeeper Proposal (BP): > > - [ ] Make sure the PR title is formatted like: > `<BP-#>: Description of bookkeeper proposal` > `e.g. BP-1: 64 bits ledger is support` > - [ ] Attach the master issue link in the description of this PR. > - [ ] Attach the google doc link if the BP is written in Google Doc. > > Otherwise: > > - [ ] Make sure the PR title is formatted like: > `<Issue #>: Description of pull request` > `e.g. Issue 123: Description ...` > - [ ] Make sure tests pass via `mvn clean apache-rat:check install spotbugs:check`. > - [ ] Replace `<Issue #>` in the title with the actual Issue number. > > --- -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
