sam-an-cloudera commented on a change in pull request #662: Hive-21787:
Metastore table cache enhancement
URL: https://github.com/apache/hive/pull/662#discussion_r293638248
##########
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java
##########
@@ -79,17 +92,21 @@
private AtomicBoolean isDatabaseCacheDirty = new AtomicBoolean(false);
// For caching TableWrapper objects. Key is aggregate of database name and
table name
- private Map<String, TableWrapper> tableCache = new TreeMap<>();
+ private ReentrantReadWriteLock tableCacheRWLock = new
ReentrantReadWriteLock(true);
Review comment:
Yes, it's a global lock. It serves a different purpose than the cacheLock.
There can be concurrent threads trying to read from the cache, they should be
able to enter the cacheLock.readLock() when current thread is modifying the
tableCache.
When modification of tableWrapper happens, we usually do the following
try {
cacheLock.readLock().lock();
tableCacheRWLock.writeLock().lock();
TableWrapper tblWrapper =
tableCache.getIfPresent(CacheUtils.buildTableKey(catName, dbName, tblName));
if (tblWrapper != null) {
tblWrapper.cachePartitions(parts, this, false);
}
} finally {
tableCacheRWLock.writeLock().unlock();
cacheLock.readLock().unlock();
My idea is the first lock, global cacheLock is there to guard read/write at
the database level, and the tableCacheRWLock is there to guard the update
block, so that the update to the tableWrapper.cachePartitions can happen
atomically. Without this tableCacheRWLock, it'd be possible for other
concurrent thread to change the tableWrapper between the time
tableCache.getIfPresent( ) is called, and tblWrapper.cachePartitions(parts,
this, false); is called, causing inconsistency. So using two locks, we can be
sure threads are entering the cache with some lock at least (the global
cacheLock), and if modification of tableWrapper is needed, need to enter write
lock on tableCacheRWLock.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
