sidkhillon commented on code in PR #8611:
URL: https://github.com/apache/hbase/pull/8611#discussion_r4037212397
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java:
##########
@@ -794,8 +794,11 @@ private SyncFuture publishSyncOnRingBuffer(boolean
forceSync) {
protected SyncFuture publishSyncOnRingBuffer(long sequence, boolean
forceSync) {
// here we use ring buffer sequence as transaction id
- SyncFuture syncFuture = getSyncFuture(sequence, forceSync);
+ // getSyncFuture must stay inside the try: the sequence is already
claimed, so we must publish
+ // it even if this throws, else the consumer wedges.
+ SyncFuture syncFuture = null;
try {
+ syncFuture = getSyncFuture(sequence, forceSync);
Review Comment:
It's all in-memory, but the memory it mutates is Guava's `LocalCache`, and
its write-order queue (present because `SyncFutureCache` uses
`expireAfterWrite`) can NPE under a race. We hit it in production on branch-2.6
(`hbase.wal.provider=filesystem`):
```
java.lang.NullPointerException: Cannot invoke
"org.apache.hbase.thirdparty.com.google.common.cache.ReferenceEntry.setNextInWriteQueue(org.apache.hbase.thirdparty.com.google.common.cache.ReferenceEntry)"
because "previous" is null
at
org.apache.hbase.thirdparty.com.google.common.cache.LocalCache.connectWriteOrder(LocalCache.java:1818)
at
org.apache.hbase.thirdparty.com.google.common.cache.LocalCache$WriteQueue.remove(LocalCache.java:3725)
at
org.apache.hbase.thirdparty.com.google.common.cache.LocalCache$Segment.removeValueFromChain(LocalCache.java:3249)
at
org.apache.hbase.thirdparty.com.google.common.cache.LocalCache$Segment.remove(LocalCache.java:3079)
at
org.apache.hbase.thirdparty.com.google.common.cache.LocalCache.remove(LocalCache.java:4273)
at
org.apache.hadoop.hbase.regionserver.wal.SyncFutureCache.getIfPresentOrNew(SyncFutureCache.java:61)
at
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.getSyncFuture(AbstractFSWAL.java:1093)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.publishSyncOnRingBuffer(FSHLog.java:789)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.publishSyncOnRingBuffer(FSHLog.java:784)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.publishSyncThenBlockOnCompletion(FSHLog.java:801)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.doSync(FSHLog.java:836)
at
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.sync(AbstractFSWAL.java:605)
at
org.apache.hadoop.hbase.regionserver.HRegion.doWALAppend(HRegion.java:7956)
at
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4597)
... RSRpcServices.multi -> RpcServer.call -> CallRunner.run ->
RpcHandler.run
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
