cuibo01 commented on a change in pull request #2889:
URL: https://github.com/apache/hbase/pull/2889#discussion_r562303540
##########
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
##########
@@ -921,8 +921,10 @@ public boolean isClusterShutdown() {
public void startChore() {
Configuration c = master.getConfiguration();
if (persistFlushedSequenceId) {
- // when reach here, RegionStates should loaded, firstly, we call remove
deleted regions
- removeDeletedRegionFromLoadedFlushedSequenceIds();
+ new Thread(() -> {
+ // when reach here, RegionStates should loaded, firstly, we call
remove deleted regions
+ removeDeletedRegionFromLoadedFlushedSequenceIds();
+ }, "RemoveDeletedRegionSyncThread").start();
Review comment:
> Should this be a demon thread?
no need, hmaster won't need the thread when it's done...
> And seems that removeDeletedRegionFromLoadedFlushedSequenceIds could be a
part of FlushedSequenceIdFlusher?
no, removeDeletedRegionFromLoadedFlushedSequenceIds takes a long time and is
disposable. if in FlushedSequenceIdFlusher, FlushedSequenceIdFlusher takes up
choreService for a long time...Other tasks of choreService
cannot be executed.
##########
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
##########
@@ -921,8 +921,10 @@ public boolean isClusterShutdown() {
public void startChore() {
Configuration c = master.getConfiguration();
if (persistFlushedSequenceId) {
- // when reach here, RegionStates should loaded, firstly, we call remove
deleted regions
- removeDeletedRegionFromLoadedFlushedSequenceIds();
+ new Thread(() -> {
+ // when reach here, RegionStates should loaded, firstly, we call
remove deleted regions
+ removeDeletedRegionFromLoadedFlushedSequenceIds();
+ }, "RemoveDeletedRegionSyncThread").start();
Review comment:
> Should this be a demon thread?
no need, hmaster won't need the thread when it's done...
> And seems that removeDeletedRegionFromLoadedFlushedSequenceIds could be a
part of FlushedSequenceIdFlusher?
no, removeDeletedRegionFromLoadedFlushedSequenceIds takes a long time and is
disposable. if in FlushedSequenceIdFlusher, FlushedSequenceIdFlusher takes up
choreService for a long time...Other tasks of choreService
cannot be executed.
##########
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
##########
@@ -921,8 +921,10 @@ public boolean isClusterShutdown() {
public void startChore() {
Configuration c = master.getConfiguration();
if (persistFlushedSequenceId) {
- // when reach here, RegionStates should loaded, firstly, we call remove
deleted regions
- removeDeletedRegionFromLoadedFlushedSequenceIds();
+ new Thread(() -> {
+ // when reach here, RegionStates should loaded, firstly, we call
remove deleted regions
+ removeDeletedRegionFromLoadedFlushedSequenceIds();
+ }, "RemoveDeletedRegionSyncThread").start();
Review comment:
> Should this be a demon thread?
no need, hmaster won't need the thread when it's done...
> And seems that removeDeletedRegionFromLoadedFlushedSequenceIds could be a
part of FlushedSequenceIdFlusher?
no, removeDeletedRegionFromLoadedFlushedSequenceIds takes a long time and is
disposable. if in FlushedSequenceIdFlusher, FlushedSequenceIdFlusher takes up
choreService for a long time...Other tasks of choreService
cannot be executed.
##########
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
##########
@@ -921,8 +921,10 @@ public boolean isClusterShutdown() {
public void startChore() {
Configuration c = master.getConfiguration();
if (persistFlushedSequenceId) {
- // when reach here, RegionStates should loaded, firstly, we call remove
deleted regions
- removeDeletedRegionFromLoadedFlushedSequenceIds();
+ new Thread(() -> {
Review comment:
1)flushedSequenceIdByRegion and storeFlushedSequenceIdsByRegion are
ConcurrentSkipListMap
2)flushedSequenceIdByRegion and storeFlushedSequenceIdsByRegion are used
only when splitting logs and updating flushSeqId, and the rs only get
regionSeqId once during split log...
i think it is ok, lik memstore put and scan at the same time
##########
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
##########
@@ -921,8 +921,10 @@ public boolean isClusterShutdown() {
public void startChore() {
Configuration c = master.getConfiguration();
if (persistFlushedSequenceId) {
- // when reach here, RegionStates should loaded, firstly, we call remove
deleted regions
- removeDeletedRegionFromLoadedFlushedSequenceIds();
+ new Thread(() -> {
+ // when reach here, RegionStates should loaded, firstly, we call
remove deleted regions
+ removeDeletedRegionFromLoadedFlushedSequenceIds();
+ }, "RemoveDeletedRegionSyncThread").start();
Review comment:
n is 50W,100W...so removeDeletedRegionFromLoadedFlushedSequenceIds is
very slow.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
