Issue in entryLogFilePreallocationEnabled logic in EntryLogger.java

[Charan Reddy G]

As part of BOOKKEEPER-643 (
https://github.com/apache/bookkeeper/commit/694568b0ff0d048c284c8d5db0c9455d30dfa3ce),
"entryLogFilePreallocationEnabled" logic is introduced in EntryLogger.java.

Because of this logic, preallocated entryLogChannel will be used as a newly
created logchannel even though the ledgerDir, in which it is created, is
full.

Consider the following example -

1) We have configured Bookies with two LedgerDirs - 'ledgerDir1',
'ledgerDir2'
2) lets say current active channel in EntryLogger is 'logChannel1' which is
created in 'ledgerDir1'
3) since 'entryLogFilePreallocationEnabled' is enabled by default, when we
set 'logChannel1' as the current active channel, we create async task to
create a new logChannel for future purpose (check
allocatorExecutor.submit(allocateNewLog task))
4) currently both the ledgerDirs are in writable state, so for precreation
of new logchannel it might pick any ledgerdir
5) lets say it chose 'ledgerDir1' again for the creation of new logchannel
- 'logChannel2'
6) now lets assume ledgerDir1 is full, in that case shouldCreateNewEntryLog
will be set to true
7) if a new entry is added to entrylogger, then since
shouldCreateNewEntryLog is set to true, it will close the current active
logchannel - 'logChannel1' and add it to logChannelsToFlush list.
8) For new logChannel it will get the preallocated log - 'logChannel2' and
will continue to use it as current active channel, though 'ledgerDir1' is
not writable anymore.

relevant code snippets in EntryLogger.java

    private LedgerDirsListener getLedgerDirsListener() {
        return new LedgerDirsListener() {
            @Override
            public void diskFull(File disk) {
                // If the current entry log disk is full, then create new
entry
                // log.
                if (currentDir != null && currentDir.equals(disk)) {
                    shouldCreateNewEntryLog.set(true);
                }
            }

            @Override
            public void diskAlmostFull(File disk) {
                // If the current entry log disk is almost full, then
create new entry
                // log.
                if (currentDir != null && currentDir.equals(disk)) {
                    shouldCreateNewEntryLog.set(true);
                }
            }
....
}

    synchronized long addEntry(long ledger, ByteBuffer entry, boolean
rollLog) throws IOException {
.....
        boolean createNewLog = shouldCreateNewEntryLog.get();
        if (createNewLog || reachEntryLogLimit) {
            if (doRegularFlushes) {
                flushCurrentLog();
            }
            createNewLog();
            // Reset the flag
            if (createNewLog) {
                shouldCreateNewEntryLog.set(false);
            }
        }
.....
    }


        synchronized BufferedLogChannel createNewLog() throws IOException {
            BufferedLogChannel bc;
            if (!entryLogPreAllocationEnabled || null == preallocation) {
                // initialization time to create a new log
                bc = allocateNewLog();
            } else {
                // has a preallocated entry log
                try {
                    bc = preallocation.get();
...
                preallocation = allocatorExecutor.submit(new
Callable<BufferedLogChannel>() {
                    @Override
                    public BufferedLogChannel call() throws IOException {
                        return allocateNewLog();
                    }
                });
            }
            LOG.info("Created new entry logger {}.", bc.getLogId());
            return bc;
        }

Thanks,
Charan