[ https://issues.apache.org/jira/browse/BOOKKEEPER-821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sijie Guo updated BOOKKEEPER-821: --------------------------------- Fix Version/s: (was: 4.3.2) > Failing to write lastId to ledger directories should not fail startup of > bookies > -------------------------------------------------------------------------------- > > Key: BOOKKEEPER-821 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-821 > Project: Bookkeeper > Issue Type: Bug > Components: bookkeeper-server > Affects Versions: 4.3.0 > Reporter: Jia Zhai > Assignee: Jia Zhai > Fix For: 4.4.0 > > Attachments: BOOKKEEPER-821_no-prefix.patch > > Original Estimate: 5h > Remaining Estimate: 5h > > At the startup of bookie, the bookie constructor would call setLastLogId() > as this trace: > Bookie() -> InterleavedLedgerStorage() -> EntryLogger() -> Initialize() > - >createNewLog() -> allocateNewLog() -> setLastLogId(). > If "bw.write() and bw.flush()" in setLastLogId() failed, an IOException would > throw and not catch, and cause Bookie constructor fail. > {code} > private void setLastLogId(File dir, long logId) throws IOException { > FileOutputStream fos; > fos = new FileOutputStream(new File(dir, "lastId")); > BufferedWriter bw = new BufferedWriter(new > OutputStreamWriter(fos, UTF_8)); > try { // < ==== original: if write fail in this try, > IOException thrown but not catch, and cause bookie startup fail. > bw.write(Long.toHexString(logId) + "\n"); > bw.flush(); > } finally { > try { > bw.close(); > } catch (IOException e) { > LOG.error("Could not close lastId file in {}", > dir.getPath()); > } > } > } > {code} > But failing setLastLogId() could be tolerated, and will not cause any problem > in next time Bookie startup. > Next time, when calling EntryLogger constructor again, in getLastLogId(), if > read ledgerDir.lastId fail, it will walk through all log files to get > LastLogID; If reading an old ledgerDir.lastId, which caused by last failure > of setLastLogId(), and it happened to be the largest logId, then > allocateNewLog() will find the file already exist, and will allocate > newlogfile with bigger ID. > {code} > BufferedLogChannel allocateNewLog() throws IOException { > List<File> list = ledgerDirsManager.getWritableLedgerDirs(); > Collections.shuffle(list); > // It would better not to overwrite existing entry log files > File newLogFile = null; > do { > String logFileName = Long.toHexString(++preallocatedLogId) + > ".log"; > for (File dir : list) { > newLogFile = new File(dir, logFileName); > currentDir = dir; > if (newLogFile.exists()) { < === this will handle last > set fail issue, in which LastId update fail, and get a wrong > preallocatedLogId. > LOG.warn("Found existed entry log " + newLogFile > + " when trying to create it as a new log."); > newLogFile = null; > break; > } > } > } while (newLogFile == null); > FileChannel channel = new RandomAccessFile(newLogFile, > "rw").getChannel(); > BufferedLogChannel logChannel = new BufferedLogChannel(channel, > conf.getWriteBufferBytes(), conf.getReadBufferBytes(), > preallocatedLogId); > logChannel.write((ByteBuffer) LOGFILE_HEADER.clear()); > for (File f : list) { > setLastLogId(f, preallocatedLogId); > } > LOG.info("Preallocated entry logger {}.", preallocatedLogId); > return logChannel; > } > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)