bharathv commented on a change in pull request #934: HBASE-23568: Improve
Threading of Replication
URL: https://github.com/apache/hbase/pull/934#discussion_r376664834
##########
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java
##########
@@ -304,31 +318,56 @@ private void initializeWALEntryFilter(UUID
peerClusterId) {
this.walEntryFilter = new ChainWALEntryFilter(filters);
}
- private void tryStartNewShipper(String walGroupId,
PriorityBlockingQueue<Path> queue) {
- workerThreads.compute(walGroupId, (key, value) -> {
- if (value != null) {
- if (LOG.isDebugEnabled()) {
- LOG.debug(
- "{} Someone has beat us to start a worker thread for wal group
{}",
- logPeerId(), key);
- }
- return value;
- } else {
- if (LOG.isDebugEnabled()) {
- LOG.debug("{} Starting up worker for wal group {}", logPeerId(),
key);
+ /**
+ * Synchronized method so that only one item is inserted at a time. Should
be the only place that
Review comment:
Oh man, this is super subtle. While I agree that there won't be considerable
perf drop (since most methods are just spinning up threads and not doing heavy
weight stuff), I still didn't understand how it can cause a deadlock.
> However, in the original code, the main thread was running within the lock
of the Map, so trying to then remove an entry from the Map that is already had
a lock on caused a deadlock.
I agree that this was in the original implementation, it was inside a
"compute()", so it grabbed the lock and it would deadlock. But with your
change, you are not inside a lock scope right? (or did I miss something). IIUC,
you are saying "workerThreads.remove(walGroupId);" deadlocks with something.
Whats that something?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
