saintstack commented on a change in pull request #739: HBASE-23157 WAL
unflushed seqId tracking may wrong when Durability.AS…
URL: https://github.com/apache/hbase/pull/739#discussion_r337663711
##########
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/SequenceIdAccounting.java
##########
@@ -184,6 +185,30 @@ void update(byte[] encodedRegionName, Set<byte[]>
families, long sequenceid,
}
}
+ /**
+ * Clear all the records of the given region as it is going to be closed.
+ * <p/>
+ * We will call this once we get the region close marker. We need this
because that, if we use
+ * Durability.ASYNC_WAL, after calling startCacheFlush, we may still get
some ongoing wal entries
+ * that has not been processed yet, this will lead to orphan records in the
+ * lowestUnflushedSequenceIds and then cause too many WAL files.
+ * <p/>
+ * See HBASE-23157 for more details.
+ */
+ void onRegionClose(byte[] encodedRegionName) {
+ synchronized (tieLock) {
+ this.lowestUnflushedSequenceIds.remove(encodedRegionName);
+ Map<ImmutableByteArray, Long> flushing =
this.flushingSequenceIds.remove(encodedRegionName);
+ if (flushing != null) {
+ LOG.warn("Still have flushing records when closing {}, {}",
+ Bytes.toString(encodedRegionName),
+ flushing.entrySet().stream().map(e -> e.getKey().toStringBinary() +
"->" + e.getValue())
+ .collect(Collectors.joining(",", "{", "}")));
+ }
+ }
+ this.highestSequenceIds.remove(encodedRegionName);
Review comment:
Good. This is like the purge method over in HBASE-23181.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
