ArvinDevel commented on a change in pull request #832: Issue 620: Close the
fileChannels for read when they are idle
URL: https://github.com/apache/bookkeeper/pull/832#discussion_r157133489
##########
File path:
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/EntryLogger.java
##########
@@ -327,54 +333,100 @@ private int readFromLogChannel(long entryLogId,
BufferedReadChannel channel, Byt
* A thread-local variable that wraps a mapping of log ids to
bufferedchannels
* These channels should be used only for reading. logChannel is the one
* that is used for writes.
+ * We use this Guava cache to store the BufferedReadChannel.
+ * When the BufferedReadChannel is removed, the underlying fileChannel's
refCnt decrease 1,
+ * temporally use 1h to relax replace after reading.
*/
- private final ThreadLocal<Map<Long, BufferedReadChannel>> logid2Channel =
- new ThreadLocal<Map<Long, BufferedReadChannel>>() {
+ private final ThreadLocal<Cache<Long, BufferedReadChannel>> logid2Channel =
+ new ThreadLocal<Cache<Long, BufferedReadChannel>>() {
@Override
- public Map<Long, BufferedReadChannel> initialValue() {
+ public Cache<Long, BufferedReadChannel> initialValue() {
// Since this is thread local there only one modifier
// We dont really need the concurrency, but we need to use
// the weak values. Therefore using the concurrency level of 1
- return new MapMaker().concurrencyLevel(1)
- .weakValues()
- .makeMap();
+ return CacheBuilder.newBuilder().concurrencyLevel(1)
+ .expireAfterAccess(expireReadChannelCacheInHour,
TimeUnit.HOURS)
+ //decrease the refCnt
+ .removalListener(removal ->
logid2FileChannel.get(removal.getKey()).release())
+ .build(readChannelLoader);
}
};
+ private final CacheLoader<Long, BufferedReadChannel> readChannelLoader =
+ new CacheLoader<Long, BufferedReadChannel> () {
+ public BufferedReadChannel load(Long entryLogId) throws Exception {
+
+ return getChannelForLogId(entryLogId);
+
+ }
+ };
+
+
+ private static class ReferenceCountedFileChannel extends
AbstractReferenceCounted {
+ private final FileChannel fc;
+
+ public ReferenceCountedFileChannel(FileChannel fileChannel) {
+ this.fc = fileChannel;
+ }
+
+
+ @Override
+ public ReferenceCounted touch(Object hint) {
+ return this;
+ }
+
+ // when the refCnt decreased to
+ @Override
+ protected void deallocate() {
+ try {
+ fc.close();
+ } catch (IOException e) {
+ LOG.warn("Exception occurred in ReferenceCountedFileChannel"
+ + " while closing channel for log file: {}", fc);
+ } finally {
+ IOUtils.close(LOG, fc);
+ }
+ }
+
+ }
+
+
/**
* Each thread local buffered read channel can share the same file handle
because reads are not relative
- * and don't cause a change in the channel's position. We use this map to
store the file channels. Each
- * file channel is mapped to a log id which represents an open log file.
+ * and don't cause a change in the channel's position.
+ * Each file channel is mapped to a log id which represents an open log
file.
+ * when ReferenceCountedFileChannel's refCnt decrease to 0, close the
fileChannel.
*/
- private final ConcurrentMap<Long, FileChannel> logid2FileChannel = new
ConcurrentHashMap<Long, FileChannel>();
+ private ConcurrentMap<Long, ReferenceCountedFileChannel>
+ logid2FileChannel = new ConcurrentHashMap<>();
+
/**
* Put the logId, bc pair in the map responsible for the current thread.
* @param logId
* @param bc
*/
- public BufferedReadChannel putInReadChannels(long logId,
BufferedReadChannel bc) {
- Map<Long, BufferedReadChannel> threadMap = logid2Channel.get();
- return threadMap.put(logId, bc);
+ public void putInReadChannels(long logId, BufferedReadChannel bc) {
+ Cache<Long, BufferedReadChannel> threadCahe = logid2Channel.get();
+ threadCahe.put(logId, bc);
}
/**
* Remove all entries for this log file in each thread's cache.
* @param logId
*/
public void removeFromChannelsAndClose(long logId) {
- FileChannel fileChannel = logid2FileChannel.remove(logId);
- if (null != fileChannel) {
- try {
- fileChannel.close();
- } catch (IOException e) {
- LOG.warn("Exception while closing channel for log file:" +
logId);
- }
- }
+ //remove the fileChannel from logId2Channel
Review comment:
I think the new method is not necessary, just keep one deallocate is simple.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
