[
https://issues.apache.org/jira/browse/HBASE-26811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17516390#comment-17516390
]
Hudson commented on HBASE-26811:
--------------------------------
Results for branch branch-2.5
[build #85 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/85/]:
(x) *{color:red}-1 overall{color}*
----
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/85/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/85/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/62/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/85/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Secondary replica may be disabled for read incorrectly forever
> --------------------------------------------------------------
>
> Key: HBASE-26811
> URL: https://issues.apache.org/jira/browse/HBASE-26811
> Project: HBase
> Issue Type: Bug
> Components: read replicas
> Affects Versions: 3.0.0-alpha-2, 2.4.10
> Reporter: chenglei
> Assignee: chenglei
> Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.12
>
>
> For read replica, when I set {{hbase.region.replica.wait.for.primary.flush}}
> to false, and set {{TableDescriptorBuilder.setRegionMemStoreReplication}} to
> true explicitly at table level, the secondary replica would be disabled for
> read, reading on this replica region would throw :
> {code:java}
> java.io.IOException: The region's reads are disabled. Cannot serve the
> request
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkReadsEnabled(HRegion.java:5187)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.java:8279)
> {code}
> Very strange, if I don't set
> {{TableDescriptorBuilder.setRegionMemStoreReplication}} to true explicitly
> (which default value is true), the secondary replica is normal.
> This problem is because when set
> {{hbase.region.replica.wait.for.primary.flush}} to false, the
> {{HRegionServer.startServices}} would not create the
> {{ExecutorType.RS_REGION_REPLICA_FLUSH_OPS}} for
> {{RegionReplicaFlushHandler}} at HRegionServer-level:
> {code:java}
> if
> (ServerRegionReplicaUtil.isRegionReplicaWaitForPrimaryFlushEnabled(conf)) {
> final int regionReplicaFlushThreads = conf.getInt(
> "hbase.regionserver.region.replica.flusher.threads", conf.getInt(
> "hbase.regionserver.executor.openregion.threads", 3));
> executorService.startExecutorService(executorService.new
> ExecutorConfig().setExecutorType(
>
> ExecutorType.RS_REGION_REPLICA_FLUSH_OPS).setCorePoolSize(regionReplicaFlushThreads));
> }
> {code}
> but when I set {{TableDescriptorBuilder.setRegionMemStoreReplication}} to
> true explicitly, it also set {{hbase.region.replica.wait.for.primary.flush}}
> to true at table-level(there is no public
> {{hbase.region.replica.wait.for.primary.flush} config for hbase user at
> table-level):
> {code:java}
> public ModifyableTableDescriptor setRegionMemStoreReplication(boolean
> memstoreReplication) {
> setValue(REGION_MEMSTORE_REPLICATION_KEY,
> Boolean.toString(memstoreReplication));
> // If the memstore replication is setup, we do not have to wait for
> observing a flush event
> // from primary before starting to serve reads, because gaps from
> replication is not applicable
> return setValue(REGION_REPLICA_WAIT_FOR_PRIMARY_FLUSH_CONF_KEY,
> Boolean.toString(memstoreReplication));
> }
> {code}
> So when the secondary replica region is
> open,{{HRegionServer.triggerFlushInPrimaryRegion}} is invoked for this
> region, because {{hbase.region.replica.wait.for.primary.flush}} to true at
> table-level, the line 2234 is skipped, secondary replica is disabled for read
> at line 2238, but there is no {{ExecutorType.RS_REGION_REPLICA_FLUSH_OPS}}
> for {{RegionReplicaFlushHandler}} at HRegionServer-level, so line 2243 would
> not schedule {{RegionReplicaFlushHandler}}, the secondary replica would be
> disabled for read.
> {code:java}
> 2227 private void triggerFlushInPrimaryRegion(final HRegion region) {
> ...
> 2232 if
> (!ServerRegionReplicaUtil.isRegionReplicaReplicationEnabled(region.conf, tn)
> ||
> 2233
> !ServerRegionReplicaUtil.isRegionReplicaWaitForPrimaryFlushEnabled(region.conf))
> {
> 2234 region.setReadsEnabled(true);
> 2235 return;
> 2236 }
> 2237
> 2238 region.setReadsEnabled(false); // disable reads before marking the
> region as opened.
> 2239 // RegionReplicaFlushHandler might reset this.
> 2240
> 2241 // Submit it to be handled by one of the handlers so that we do not
> block OpenRegionHandler
> 2242 if (this.executorService != null) {
> 2243 this.executorService.submit(new RegionReplicaFlushHandler(this,
> region));
> 2244 } else {
> ...
> {code}
> I think for above {{ModifyableTableDescriptor.setRegionMemStoreReplication}},
> when set it to true, there is no reason to also set the
> {{hbase.region.replica.wait.for.primary.flush}} to true at table-level.
> This problem may be more serious on master because the new replication
> framework (HBASE-26233) does not enable the secondary replica read when
> receives the flush marker, that is to say, the secondary replica read would
> be disabled for read forever.
> But for branch-2, the secondary replica would be enabled for read when
> receives the flush marker.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
