[jira] [Resolved] (HBASE-25780) Add 2.2.7 to download page
[ https://issues.apache.org/jira/browse/HBASE-25780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25780. Assignee: Guanghao Zhang Resolution: Fixed > Add 2.2.7 to download page > -- > > Key: HBASE-25780 > URL: https://issues.apache.org/jira/browse/HBASE-25780 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25780) Add 2.2.7 to download page
Guanghao Zhang created HBASE-25780: -- Summary: Add 2.2.7 to download page Key: HBASE-25780 URL: https://issues.apache.org/jira/browse/HBASE-25780 Project: HBase Issue Type: Sub-task Reporter: Guanghao Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25764) Generate CHANGES.md and RELEASENOTES.md for 2.2.7
[ https://issues.apache.org/jira/browse/HBASE-25764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25764. Resolution: Fixed > Generate CHANGES.md and RELEASENOTES.md for 2.2.7 > - > > Key: HBASE-25764 > URL: https://issues.apache.org/jira/browse/HBASE-25764 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25765) Set version as 2.2.7 in branch-2.2
[ https://issues.apache.org/jira/browse/HBASE-25765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25765. Fix Version/s: 2.2.7 Resolution: Fixed > Set version as 2.2.7 in branch-2.2 > -- > > Key: HBASE-25765 > URL: https://issues.apache.org/jira/browse/HBASE-25765 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 2.2.7 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25765) Set version as 2.2.7 in branch-2.2
Guanghao Zhang created HBASE-25765: -- Summary: Set version as 2.2.7 in branch-2.2 Key: HBASE-25765 URL: https://issues.apache.org/jira/browse/HBASE-25765 Project: HBase Issue Type: Sub-task Reporter: Guanghao Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25764) Generate CHANGES.md and RELEASENOTES.md for 2.2.7
Guanghao Zhang created HBASE-25764: -- Summary: Generate CHANGES.md and RELEASENOTES.md for 2.2.7 Key: HBASE-25764 URL: https://issues.apache.org/jira/browse/HBASE-25764 Project: HBase Issue Type: Sub-task Reporter: Guanghao Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25427) [branch-2.2] Fix the ruby problem for pre-commit
[ https://issues.apache.org/jira/browse/HBASE-25427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25427. Fix Version/s: 2.2.7 Assignee: Guanghao Zhang Resolution: Fixed > [branch-2.2] Fix the ruby problem for pre-commit > > > Key: HBASE-25427 > URL: https://issues.apache.org/jira/browse/HBASE-25427 > Project: HBase > Issue Type: Bug >Affects Versions: 2.2.6 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 2.2.7 > > > *00:33:07* [91mERROR: Error installing rubocop:*00:33:07* > parallel requires Ruby version >= 2.4.*00:33:08* [0mSuccessfully installed > jaro_winkler-1.5.4 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25605) Try ignore the ExportSnapshot related unit tests for branch-2.2
[ https://issues.apache.org/jira/browse/HBASE-25605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25605. Assignee: Guanghao Zhang Resolution: Fixed > Try ignore the ExportSnapshot related unit tests for branch-2.2 > --- > > Key: HBASE-25605 > URL: https://issues.apache.org/jira/browse/HBASE-25605 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 2.2.7 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25605) Try ignore the ExportSnapshot related unit tests
Guanghao Zhang created HBASE-25605: -- Summary: Try ignore the ExportSnapshot related unit tests Key: HBASE-25605 URL: https://issues.apache.org/jira/browse/HBASE-25605 Project: HBase Issue Type: Sub-task Reporter: Guanghao Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25545) Fix the build problem for branch-2.2
[ https://issues.apache.org/jira/browse/HBASE-25545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25545. Resolution: Fixed Pushed to branch-2.2. Thanks [~meiyi] for reviewing. > Fix the build problem for branch-2.2 > > > Key: HBASE-25545 > URL: https://issues.apache.org/jira/browse/HBASE-25545 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.2.7 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25545) Fix the build problem for branch-2.2
Guanghao Zhang created HBASE-25545: -- Summary: Fix the build problem for branch-2.2 Key: HBASE-25545 URL: https://issues.apache.org/jira/browse/HBASE-25545 Project: HBase Issue Type: Sub-task Affects Versions: 2.2.7 Reporter: Guanghao Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25544) Release 2.2.7
Guanghao Zhang created HBASE-25544: -- Summary: Release 2.2.7 Key: HBASE-25544 URL: https://issues.apache.org/jira/browse/HBASE-25544 Project: HBase Issue Type: Umbrella Reporter: Guanghao Zhang As discussed in [https://s.apache.org/7rqow] , the stable pointer had been moved to 2.3.x and the final release for 2.2.x will be 2.2.7. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25426) [branch-2.2] Fix the ruby problem for pre-commit
[ https://issues.apache.org/jira/browse/HBASE-25426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25426. Resolution: Duplicate Duplicate with HBASE-25426. > [branch-2.2] Fix the ruby problem for pre-commit > > > Key: HBASE-25426 > URL: https://issues.apache.org/jira/browse/HBASE-25426 > Project: HBase > Issue Type: Bug >Affects Versions: 2.2.6 >Reporter: Guanghao Zhang >Priority: Major > > *00:33:07* [91mERROR: Error installing rubocop:*00:33:07* > parallel requires Ruby version >= 2.4.*00:33:08* [0mSuccessfully installed > jaro_winkler-1.5.4 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25427) [branch-2.2] Fix the ruby problem for pre-commit
Guanghao Zhang created HBASE-25427: -- Summary: [branch-2.2] Fix the ruby problem for pre-commit Key: HBASE-25427 URL: https://issues.apache.org/jira/browse/HBASE-25427 Project: HBase Issue Type: Bug Affects Versions: 2.2.6 Reporter: Guanghao Zhang *00:33:07* [91mERROR: Error installing rubocop:*00:33:07*parallel requires Ruby version >= 2.4.*00:33:08* [0mSuccessfully installed jaro_winkler-1.5.4 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25426) [branch-2.2] Fix the ruby problem for pre-commit
Guanghao Zhang created HBASE-25426: -- Summary: [branch-2.2] Fix the ruby problem for pre-commit Key: HBASE-25426 URL: https://issues.apache.org/jira/browse/HBASE-25426 Project: HBase Issue Type: Bug Affects Versions: 2.2.6 Reporter: Guanghao Zhang *00:33:07* [91mERROR: Error installing rubocop:*00:33:07*parallel requires Ruby version >= 2.4.*00:33:08* [0mSuccessfully installed jaro_winkler-1.5.4 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25324) Remove unnecessary array to list conversion in SplitLogManager
[
https://issues.apache.org/jira/browse/HBASE-25324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang resolved HBASE-25324.
Fix Version/s: 2.4.0
3.0.0-alpha-1
Resolution: Fixed
Pushed to branch-2 and master. Thanks [~yuqi] for contributing.
> Remove unnecessary array to list conversion in SplitLogManager
> --
>
> Key: HBASE-25324
> URL: https://issues.apache.org/jira/browse/HBASE-25324
> Project: HBase
> Issue Type: Improvement
>Reporter: yuqi
>Assignee: yuqi
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
> {code:java}
> public static List getFileList(final Configuration conf, final
> List logDirs,
> final PathFilter filter)
> throws IOException {
> List fileStatus = new ArrayList<>();
>
> FileStatus[] a = new FileStatus[fileStatus.size()];
> //list to array
> return fileStatus.toArray(a);
> }
> public List getWALsToSplit(ServerName serverName, boolean
> splitMeta)
> throws IOException {
> List logDirs =
> master.getMasterWalManager().getLogDirs(Collections.singleton(serverName));
> FileStatus[] fileStatuses =
> SplitLogManager.getFileList(this.conf, logDirs, splitMeta ?
> META_FILTER : NON_META_FILTER);
> LOG.info("{} WAL count={}, meta={}", serverName, fileStatuses.length,
> splitMeta);
> //array convert to list which is just contrary to the logic of
> getFileList
> return Lists.newArrayList(fileStatuses);
>
> {code}
> After judge the logic, there seems no need to make such a conversion.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-25323) Fix potential NPE when the zookeeper path of RegionServerTracker does not exist when start
[ https://issues.apache.org/jira/browse/HBASE-25323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25323. Fix Version/s: 2.3.4 2.2.7 2.4.0 3.0.0-alpha-1 Resolution: Fixed Pushed to branch-2.2+. Thanks [~yuqi] for contributing. > Fix potential NPE when the zookeeper path of RegionServerTracker does not > exist when start > -- > > Key: HBASE-25323 > URL: https://issues.apache.org/jira/browse/HBASE-25323 > Project: HBase > Issue Type: Improvement >Reporter: yuqi >Assignee: yuqi >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.4.0, 2.2.7, 2.3.4 > > Attachments: image-2020-11-24-16-56-04-411.png, > image-2020-11-24-16-58-04-621.png > > > !image-2020-11-24-16-56-04-411.png! > !image-2020-11-24-16-58-04-621.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25281) Bulkload split hfile too many times due to unreasonable split point
[
https://issues.apache.org/jira/browse/HBASE-25281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang resolved HBASE-25281.
Fix Version/s: 2.4.0
3.0.0-alpha-1
Resolution: Fixed
Pushed to branch-2 and master. Thanks [~niuyulin] for contributing.
> Bulkload split hfile too many times due to unreasonable split point
> ---
>
> Key: HBASE-25281
> URL: https://issues.apache.org/jira/browse/HBASE-25281
> Project: HBase
> Issue Type: Improvement
> Components: tooling
>Reporter: niuyulin
>Assignee: niuyulin
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.4.0
>
> Attachments: bulkload.png
>
>
> [https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/tool/BulkLoadHFilesTool.java#L688]
> if hfile span multi regions, for example A,B,C,D,E,F(the start key of these
> regions are in ascending order), we should use region C endkey to split, not
> region A. In this way, we can get equal .top and .bottom hfiles, reduce time
> complexity of split from O( n ) to O(logn),decrease invoke of {{bulkLoad}}
> rpc to regionserver ,and also avoid write amplification during
> {{copyHFileHalf}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-25325) Remove unused class ClusterSchemaException
[ https://issues.apache.org/jira/browse/HBASE-25325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25325. Fix Version/s: 2.4.0 3.0.0-alpha-1 Resolution: Fixed Pushed to branch-2 and master. Thanks [~yuqi] for contributing. > Remove unused class ClusterSchemaException > -- > > Key: HBASE-25325 > URL: https://issues.apache.org/jira/browse/HBASE-25325 > Project: HBase > Issue Type: Improvement >Reporter: yuqi >Assignee: yuqi >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.4.0 > > > Currently, ClusterSchemaException is useless, so we can freely delete it -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25282) Remove processingServers in DeadServer as we can get this information by Procedure of master
[ https://issues.apache.org/jira/browse/HBASE-25282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25282. Fix Version/s: 3.0.0-alpha-1 Resolution: Fixed Pushed to master. Thanks [~yuqi] for contributing. > Remove processingServers in DeadServer as we can get this information by > Procedure of master > > > Key: HBASE-25282 > URL: https://issues.apache.org/jira/browse/HBASE-25282 > Project: HBase > Issue Type: Improvement >Reporter: yuqi >Assignee: yuqi >Priority: Minor > Fix For: 3.0.0-alpha-1 > > > We can directly get the servers in processing through proceduces in master as > long as proceduce is a instance of `ServerCrashProcedure` > *Our final goal is to remove class `DeadServer`*, the next step is to remove > the deadServers attributes in DeadServer and then we can freely delete the > class -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25213) Should request Compaction when bulkLoadHFiles is done
[ https://issues.apache.org/jira/browse/HBASE-25213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25213. Resolution: Fixed Pushed the addendum patch to branch-2 and master. > Should request Compaction when bulkLoadHFiles is done > - > > Key: HBASE-25213 > URL: https://issues.apache.org/jira/browse/HBASE-25213 > Project: HBase > Issue Type: Improvement >Reporter: niuyulin >Assignee: niuyulin >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.4.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24654) Allow unset table's rsgroup
[ https://issues.apache.org/jira/browse/HBASE-24654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-24654. Resolution: Fixed Pushed to master. Thanks [~Ddupg] for contributing. > Allow unset table's rsgroup > --- > > Key: HBASE-24654 > URL: https://issues.apache.org/jira/browse/HBASE-24654 > Project: HBase > Issue Type: Improvement > Components: rsgroup >Affects Versions: 3.0.0-alpha-1 >Reporter: Sun Xin >Assignee: Sun Xin >Priority: Major > Fix For: 3.0.0-alpha-1 > > > In TableDescriptorBuilder, we have only one method to set rsgroup, but have > no one to unset it. this unset method is necessary In some cases. > If the table had rsgroup config before, but now I want to use the namespace > config. It doesn't work that I set table rsgroup config to default rsgroup, > must remove rsgroup config. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25113) [testing] HBaseCluster support ReplicationServer for UTs
[ https://issues.apache.org/jira/browse/HBASE-25113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25113. Resolution: Fixed Merged. Thanks [~Ddupg] for contributing. > [testing] HBaseCluster support ReplicationServer for UTs > > > Key: HBASE-25113 > URL: https://issues.apache.org/jira/browse/HBASE-25113 > Project: HBase > Issue Type: Sub-task > Components: Replication >Affects Versions: 3.0.0-alpha-1 >Reporter: Sun Xin >Assignee: Sun Xin >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25213) Should request Compaction when bulkLoadHFiles is done
[ https://issues.apache.org/jira/browse/HBASE-25213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25213. Fix Version/s: 2.4.0 3.0.0-alpha-1 Resolution: Fixed Merged. Thanks [~niuyulin] for contributing. > Should request Compaction when bulkLoadHFiles is done > - > > Key: HBASE-25213 > URL: https://issues.apache.org/jira/browse/HBASE-25213 > Project: HBase > Issue Type: Improvement >Reporter: niuyulin >Assignee: niuyulin >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.4.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25300) 'Unknown table hbase:quota' happens when desc table in shell if quota disabled
[ https://issues.apache.org/jira/browse/HBASE-25300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25300. Fix Version/s: 2.3.4 2.2.7 2.4.0 Resolution: Fixed Pushed to branch-2.2+. Thanks [~Ddupg] for contributing. > 'Unknown table hbase:quota' happens when desc table in shell if quota disabled > -- > > Key: HBASE-25300 > URL: https://issues.apache.org/jira/browse/HBASE-25300 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 3.0.0-alpha-1 >Reporter: Sun Xin >Assignee: Sun Xin >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0, 2.2.7, 2.3.4 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25289) [testing] Clean up resources after tests in rsgroup_shell_test.rb
[ https://issues.apache.org/jira/browse/HBASE-25289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25289. Fix Version/s: 2.3.4 2.4.0 Resolution: Fixed Pushed to branch-2.3 and branch-2. Thanks [~Ddupg] for contributing. > [testing] Clean up resources after tests in rsgroup_shell_test.rb > - > > Key: HBASE-25289 > URL: https://issues.apache.org/jira/browse/HBASE-25289 > Project: HBase > Issue Type: Improvement > Components: rsgroup, test >Affects Versions: 3.0.0-alpha-1 >Reporter: Sun Xin >Assignee: Sun Xin >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0, 2.3.4 > > > In rsgroup_shell_test.rb, some tests don't remove rsgroups and drop tables, > messing up adding new tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25296) [Documentation] fix duplicate conf entry
[
https://issues.apache.org/jira/browse/HBASE-25296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang resolved HBASE-25296.
Resolution: Fixed
Pushed to master. Thanks [~tangtianhang] for contributing.
> [Documentation] fix duplicate conf entry
>
>
> Key: HBASE-25296
> URL: https://issues.apache.org/jira/browse/HBASE-25296
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: tianhang tang
>Assignee: tianhang tang
>Priority: Trivial
>
> [hbase.rolling.restart|https://hbase.apache.org/book.html#hbase.rolling.restart]
> {panel:title=HBase 2.0+ can no longer read Sequence File based WAL file.}
> HBase can no longer read the deprecated WAL files written in the Apache
> Hadoop Sequence File format. The hbase.regionserver.hlog.reader.impl and
> hbase.regionserver.hlog.reader.impl configuration entries should be set to
> use the Protobuf based WAL reader / writer classes. This implementation has
> been the default since HBase 0.96, so legacy WAL files should not be a
> concern for most downstream users.
> {panel}
> It should be:
> "The _hbase.regionserver.hlog.reader.impl_ and
> _hbase.regionserver.hlog.writer.impl_ "...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-25295) Refactor the locate WAL logic in ReplicationSource
Guanghao Zhang created HBASE-25295: -- Summary: Refactor the locate WAL logic in ReplicationSource Key: HBASE-25295 URL: https://issues.apache.org/jira/browse/HBASE-25295 Project: HBase Issue Type: Bug Components: Replication Reporter: Guanghao Zhang When cluster replication enabled and one RegionServer crashed, its WALs will be move from WALs dir to oldWALs dir and its replication queue will moved to other RegionServer's replication queue. HDFS layout (WAL Storage) /hbase/WALs/RS1/1.log /hbase/WALs/RS1/2.log /hbase/WALs/RS1/3.log ZooKeeper layout (Replication queue storage) /hbase/replication/rs/RS1/peerId/1.log /hbase/replication/rs/RS1/peerId/2.log /hbase/replication/rs/RS1/peerId/3.log Failover finished: HDFS layout (WAL Storage) /hbase/.oldWALs/1.log /hbase/.oldWALs/2.log /hbase/.oldWALs/3.log ZooKeeper layout (Replication queue storage) /hbase/replication/rs/RS2/peerId-RS1/1.log /hbase/replication/rs/RS2/peerId-RS1/2.log /hbase/replication/rs/RS2/peerId-RS1/3.log And if enabled hbase.separate.oldlogdir.by.regionserver, the HDFS layout may be: HDFS layout (WAL Storage) /hbase/.oldWALs/RS1/1.log /hbase/.oldWALs/RS1/2.log /hbase/.oldWALs/RS1/3.log Then if RS2 crashed, the HDFS layout will not change but ZooKeeper layout may changed. ZooKeeper layout (Replication queue storage) /hbase/replication/rs/RS3/peerId-RS1-RS2/1.log /hbase/replication/rs/RS3/peerId-RS1-RS2/2.log /hbase/replication/rs/RS3/peerId-RS1-RS2/3.log So even the replication queue was transfered many times, the HDFS layout never change. Another case is master-cluster disaster, the failover work not finished. Then ReplicationSyncUp tool can start replication source to replicate the WAL data. The HDFS layout need to consider two more case: /hbase/WALs/RS1/1.log /hbase/WALs/RS1/2.log /hbase/WALs/RS1/3.log or /hbase/WALs/RS1-splitting/1.log /hbase/WALs/RS1-splitting/2.log /hbase/WALs/RS1-splitting/3.log -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25276) Need to throw the original exception in HRegion#openHRegion
[
https://issues.apache.org/jira/browse/HBASE-25276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang resolved HBASE-25276.
Fix Version/s: 2.3.4
2.2.7
2.4.0
3.0.0-alpha-1
Resolution: Fixed
Pushed to branch-2.2+. Thanks [~niuyulin] for contributing.
> Need to throw the original exception in HRegion#openHRegion
> ---
>
> Key: HBASE-25276
> URL: https://issues.apache.org/jira/browse/HBASE-25276
> Project: HBase
> Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: niuyulin
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.0, 2.2.7, 2.3.4
>
>
> {code:java}
> protected HRegion openHRegion(final CancelableProgressable reporter)
> throws IOException {
> try {
> ..
> } catch(Throwable t) {
> // By coprocessor path wrong region will open failed,
> // MetricsRegionWrapperImpl is already init and not close,
> // add region close when open failed
> this.close(); // This may throw IOException too.
> throw t;
> }
> return this;
> }
> {code}
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java#L8144
> this.close() may throw IOException too. The original exception will not be
> recorded.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-25276) Need to throw the original exception in HRegion#openHRegion
Guanghao Zhang created HBASE-25276:
--
Summary: Need to throw the original exception in
HRegion#openHRegion
Key: HBASE-25276
URL: https://issues.apache.org/jira/browse/HBASE-25276
Project: HBase
Issue Type: Bug
Reporter: Guanghao Zhang
{code:java}
protected HRegion openHRegion(final CancelableProgressable reporter)
throws IOException {
try {
..
} catch(Throwable t) {
// By coprocessor path wrong region will open failed,
// MetricsRegionWrapperImpl is already init and not close,
// add region close when open failed
this.close(); // This may throw IOException too.
throw t;
}
return this;
}
{code}
https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java#L8144
this.close() may throw IOException too. The original exception will not be
recorded.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-25071) ReplicationServer support start ReplicationSource internal
[ https://issues.apache.org/jira/browse/HBASE-25071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25071. Resolution: Fixed Merged. Thanks all for reviewing. > ReplicationServer support start ReplicationSource internal > -- > > Key: HBASE-25071 > URL: https://issues.apache.org/jira/browse/HBASE-25071 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24999) Master manages ReplicationServers
[ https://issues.apache.org/jira/browse/HBASE-24999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-24999. Resolution: Fixed Merged. Thanks [~Ddupg] for contributing. > Master manages ReplicationServers > - > > Key: HBASE-24999 > URL: https://issues.apache.org/jira/browse/HBASE-24999 > Project: HBase > Issue Type: Sub-task > Components: Replication >Affects Versions: 3.0.0-alpha-1 >Reporter: Sun Xin >Assignee: Sun Xin >Priority: Major > > In [HBASE-24683|https://issues.apache.org/jira/browse/HBASE-24683] add an > isolated ReplicationServer. > What this issue is to do: > # ReplicationServer reports to Master periodically. > # Add a basic ReplicationServerManager in Master to manage ReplicationServer. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25225) Create table very slowly if there are multi regions
Guanghao Zhang created HBASE-25225:
--
Summary: Create table very slowly if there are multi regions
Key: HBASE-25225
URL: https://issues.apache.org/jira/browse/HBASE-25225
Project: HBase
Issue Type: Bug
Affects Versions: 2.2.6
Reporter: Guanghao Zhang
Run the same UT TestRegionReplicaFailover on my local PC, mvn clean test
-Dtest=TestRegionReplicaFailover, branch-2.2 takes 8 mins but branch-2.3 only
needs 2 mins.
I found the problem is related to procedure schedule. See the below log:
2020-10-21 13:52:28,097 INFO [PEWorker-1] procedure2.ProcedureExecutor(1427):
Finished pid=296, ppid=45, state=SUCCESS;
org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure in 1.6250sec
2020-10-21 13:52:28,538 INFO [PEWorker-3] procedure2.ProcedureExecutor(1427):
Finished pid=45, ppid=20, state=SUCCESS; TransitRegionStateProcedure
table=testLotsOfRegionRepli2, region=50703895da3cb8c942d3197600d549bc, ASSIGN
in 59.4330sec
The real assign procedure only cost 1.6 seconds but the
TransitRegionStateProcedure cost 59.4 seconds. The pid=45 procedure was
initialized at 2020-10-21 13:51:28,666. It was added to TableQueue at
2020-10-21 13:51:28,789. But took xlock to run at 2020-10-21 13:52:24,761. See
the below log:
{color:#ff}2020-10-21 13:51:28,789{color} DEBUG [PEWorker-4]
procedure.MasterProcedureScheduler(352): Add TableQueue(testLotsOfRegionRepli2,
xlock=true (20) sharedLock=0 size=25) to run queue because: pid=45, ppid=20,
state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE;
TransitRegionStateProcedure table=testLotsOfRegionRepli2,
region=50703895da3cb8c942d3197600d549bc, ASSIGN has the excusive lock access
{color:#ff}2020-10-21 13:52:24,761{color} INFO [PEWorker-2]
procedure.MasterProcedureScheduler(737): Took xlock for pid=45, ppid=20,
state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE;
TransitRegionStateProcedure table=testLotsOfRegionRepli2,
region=50703895da3cb8c942d3197600d549bc, ASSIGN
But when I tried this UT on another PC, it only cost 2 mins, which is the same
with branch-2.3. It is weird.
Marked this as blocker for release 2.2.7.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-25128) RSGroupInfo's toString() and hashCode() does not take into account configuration map.
[ https://issues.apache.org/jira/browse/HBASE-25128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25128. Fix Version/s: 2.4.0 3.0.0-alpha-1 Resolution: Fixed Pushed to branch-2 and master. Thanks [~sanjeetnishad] for contributing. > RSGroupInfo's toString() and hashCode() does not take into account > configuration map. > - > > Key: HBASE-25128 > URL: https://issues.apache.org/jira/browse/HBASE-25128 > Project: HBase > Issue Type: Improvement > Components: rsgroup >Affects Versions: 2.2.3 >Reporter: Sanjeet Nishad >Assignee: Sanjeet Nishad >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.4.0 > > > RSGroupInfo's toString() and hashcode() methods should include Configurations > as well which is added as a part of HBASE-24431. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (HBASE-25186) TestMasterRegionOnTwoFileSystems is failing after HBASE-25065
[ https://issues.apache.org/jira/browse/HBASE-25186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang reopened HBASE-25186: Reopen for branch-2. Failed to mvn package. > TestMasterRegionOnTwoFileSystems is failing after HBASE-25065 > - > > Key: HBASE-25186 > URL: https://issues.apache.org/jira/browse/HBASE-25186 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 3.0.0-alpha-1, 2.4.0 >Reporter: ramkrishna.s.vasudevan >Assignee: Duo Zhang >Priority: Blocker > Fix For: 3.0.0-alpha-1, 2.4.0 > > > After HBASE-25065, we are having a test case failure with > TestMasterRegionOnTwoFileSystems. > The reason is that we manually trigger a WAL roll on the master region. As > part of the WAL roll we expect the Master region's WAL will also be moved > from region oldWAL dir to the global oldWAL directory. This happens after > afterRoll() method in AbstractWALRoller. > Since now the WAL archival is asynchronous the afterRoll() method does not > find any WAL file to be moved in the local region oldWAL dir. So the movement > to global oldWAL dir does not happen. > The test case checks for the file in the oldWAL dir and since it is not found > the test timesout. WE need a way to fix this. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25204) Nightly job failed as the name of jdk and maven changed
[ https://issues.apache.org/jira/browse/HBASE-25204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25204. Fix Version/s: 2.2.7 1.4.14 2.4.0 1.7.0 2.3.3 3.0.0-alpha-1 Resolution: Fixed Pushed to all active branchs. Thanks [~zhangduo] for reviewing. > Nightly job failed as the name of jdk and maven changed > > > Key: HBASE-25204 > URL: https://issues.apache.org/jira/browse/HBASE-25204 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0, 1.4.14, 2.2.7 > > > See > [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/85/console] > [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/103/console] > > org.codehaus.groovy.control.MultipleCompilationErrorsException: startup > failed: WorkflowScript: 508: Tool type "maven" does not have an install of > "Maven (latest)" configured - did you mean "maven_latest"? @ line 508, column > 19. maven 'Maven (latest)' ^ WorkflowScript: 510: Tool type "jdk" does not > have an install of "JDK 1.8 (latest)" configured - did you mean > "jdk_1.8_latest"? @ line 510, column 17. jdk "JDK 1.8 (latest)" > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25204) Nightly job failed as the name of jdk and maven changed
Guanghao Zhang created HBASE-25204: -- Summary: Nightly job failed as the name of jdk and maven changed Key: HBASE-25204 URL: https://issues.apache.org/jira/browse/HBASE-25204 Project: HBase Issue Type: Bug Reporter: Guanghao Zhang See [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/85/console] [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/103/console] org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed: WorkflowScript: 508: Tool type "maven" does not have an install of "Maven (latest)" configured - did you mean "maven_latest"? @ line 508, column 19. maven 'Maven (latest)' ^ WorkflowScript: 510: Tool type "jdk" does not have an install of "JDK 1.8 (latest)" configured - did you mean "jdk_1.8_latest"? @ line 510, column 17. jdk "JDK 1.8 (latest)" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25200) Try enlarge the flaky test timeout for branch-2.2
Guanghao Zhang created HBASE-25200: -- Summary: Try enlarge the flaky test timeout for branch-2.2 Key: HBASE-25200 URL: https://issues.apache.org/jira/browse/HBASE-25200 Project: HBase Issue Type: Bug Reporter: Guanghao Zhang Now there are too many flaky tests to run. And the flaky test job cannot finished. Then these tests will be marked to flaky again. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25177) Try create table with 100 regions for branch-2.2 nightly job's hadoop integration test
[ https://issues.apache.org/jira/browse/HBASE-25177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25177. Resolution: Won't Fix > Try create table with 100 regions for branch-2.2 nightly job's hadoop > integration test > -- > > Key: HBASE-25177 > URL: https://issues.apache.org/jira/browse/HBASE-25177 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > > It still failed now. > [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/88/execution/node/171/log/] > > [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/88//artifact/output-integration/hadoop-2.log] > > It failed when create table with 1000 regions. And not import the example TSV > to HDFS. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25178) Remove the hadoop 3.3.0 personality hadoopcheck for branch-2.2/branch-2.3
[ https://issues.apache.org/jira/browse/HBASE-25178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25178. Resolution: Duplicate Already fixed by HBASE-25144. > Remove the hadoop 3.3.0 personality hadoopcheck for branch-2.2/branch-2.3 > - > > Key: HBASE-25178 > URL: https://issues.apache.org/jira/browse/HBASE-25178 > Project: HBase > Issue Type: Bug >Affects Versions: 2.2.6 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > > For branch-2.2, see > [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/88/execution/node/163/log/] > It will fail when run "mvn clean install -DskipTests -DHBasePatchProcess > -Dhadoop-three.version=3.3.0 -Dhadoop.profile=3.0". > > For branch-2.3, see HBASE-23834. HBase failed to start on hadoop 3.3.0 > because the jetty problem. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25178) Fix the LICENSE error when branch-2.2 build with hadoop 3.3.0
Guanghao Zhang created HBASE-25178: -- Summary: Fix the LICENSE error when branch-2.2 build with hadoop 3.3.0 Key: HBASE-25178 URL: https://issues.apache.org/jira/browse/HBASE-25178 Project: HBase Issue Type: Bug Affects Versions: 2.2.6 Reporter: Guanghao Zhang See [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/88/execution/node/163/log/] It will fail when run "mvn clean install -DskipTests -DHBasePatchProcess -Dhadoop-three.version=3.3.0 -Dhadoop.profile=3.0". -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25177) Try create table with 100 regions for branch-2.2 nightly job's hadoop integration test
Guanghao Zhang created HBASE-25177: -- Summary: Try create table with 100 regions for branch-2.2 nightly job's hadoop integration test Key: HBASE-25177 URL: https://issues.apache.org/jira/browse/HBASE-25177 Project: HBase Issue Type: Bug Reporter: Guanghao Zhang It still failed now. [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/88/execution/node/171/log/] [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/88//artifact/output-integration/hadoop-2.log] It failed when create table with 1000 regions. And not import the example TSV to HDFS. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25172) No need timelineservice for branch-2.2 nightly job's hadoop integration test
[
https://issues.apache.org/jira/browse/HBASE-25172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang resolved HBASE-25172.
Resolution: Fixed
> No need timelineservice for branch-2.2 nightly job's hadoop integration test
>
>
> Key: HBASE-25172
> URL: https://issues.apache.org/jira/browse/HBASE-25172
> Project: HBase
> Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 2.2.7
>
>
> [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/86/execution/node/171/log/]
>
>
> /home/jenkins/jenkins-home/workspace/HBase_HBase_Nightly_branch-2.2/component/dev-support/hbase_nightly_pseudo-distributed-test.sh
> --single-process --working-dir output-integration/hadoop-2
> --hbase-client-install hbase-client hbase-install hadoop-2/bin/hadoop
> {color:#ff}hadoop-2/share/hadoop/yarn/timelineservice{color}
> hadoop-2/share/hadoop/yarn/test/hadoop-yarn-server-tests-2.8.5-tests.jar
> hadoop-2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.8.5-tests.jar
> hadoop-2/bin/mapred
>
> branch-2.2 still use hadoop 2.8.5 and hadoop 2.8.5 doesn't have
> timelineservice. The dev-support/hbase_nightly_pseudo-distributed-test.sh not
> consider this timelineservice and only consider 5 paramerters. But
> branch-2.3+ use 2.10.x hadoop, so they consider 6 parameters.
>
> And for hadoop-3, the timelineservice is not used, too. See
> [https://github.com/apache/hbase/blob/master/dev-support/hbase_nightly_pseudo-distributed-test.sh#L286]
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-25172) No need timelineservice for branch-2.2 nightly job's hadoop integration test
Guanghao Zhang created HBASE-25172: -- Summary: No need timelineservice for branch-2.2 nightly job's hadoop integration test Key: HBASE-25172 URL: https://issues.apache.org/jira/browse/HBASE-25172 Project: HBase Issue Type: Bug Reporter: Guanghao Zhang [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/86/execution/node/171/log/] /home/jenkins/jenkins-home/workspace/HBase_HBase_Nightly_branch-2.2/component/dev-support/hbase_nightly_pseudo-distributed-test.sh --single-process --working-dir output-integration/hadoop-2 --hbase-client-install hbase-client hbase-install hadoop-2/bin/hadoop hadoop-2/share/hadoop/yarn/timelineservice hadoop-2/share/hadoop/yarn/test/hadoop-yarn-server-tests-2.8.5-tests.jar hadoop-2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.8.5-tests.jar hadoop-2/bin/mapred branch-2.2 still use hadoop 2.8.5 and doesn't have timelineservice. The dev-support/hbase_nightly_pseudo-distributed-test.sh not consider this. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25110) Add heartbeat for ReplicationServer and dispatch replication sources to ReplicationServer
Guanghao Zhang created HBASE-25110: -- Summary: Add heartbeat for ReplicationServer and dispatch replication sources to ReplicationServer Key: HBASE-25110 URL: https://issues.apache.org/jira/browse/HBASE-25110 Project: HBase Issue Type: Sub-task Reporter: Guanghao Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24684) Fetch ReplicationSink servers list from HMaster instead of ZooKeeper
[ https://issues.apache.org/jira/browse/HBASE-24684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-24684. Resolution: Fixed Merged to HBASE-24666. > Fetch ReplicationSink servers list from HMaster instead of ZooKeeper > > > Key: HBASE-24684 > URL: https://issues.apache.org/jira/browse/HBASE-24684 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Sun Xin >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25100) conf and conn are assigned twice in HBaseReplicationEndpoint and HBaseInterClusterReplicationEndpoint
[ https://issues.apache.org/jira/browse/HBASE-25100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25100. Resolution: Fixed Pushed to master. Thanks [~Ddupg] for contributing. > conf and conn are assigned twice in HBaseReplicationEndpoint and > HBaseInterClusterReplicationEndpoint > - > > Key: HBASE-25100 > URL: https://issues.apache.org/jira/browse/HBASE-25100 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 3.0.0-alpha-1 >Reporter: Sun Xin >Assignee: Sun Xin >Priority: Major > Fix For: 3.0.0-alpha-1 > > > In > [HBaseReplicationEndpoint.init()|https://github.com/apache/hbase/blob/c312760819ed185cab3a0717a1ea0ff6e8c47a23/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/HBaseReplicationEndpoint.java#L109] > and > [HBaseInterClusterReplicationEndpoint.init|https://github.com/apache/hbase/blob/c312760819ed185cab3a0717a1ea0ff6e8c47a23/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HBaseInterClusterReplicationEndpoint.java#L145] > , the latter is a sub-class of the former, conf and conn are assigned twice. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25077) hbck.jsp page loading fails, logs NPE in master log.
[ https://issues.apache.org/jira/browse/HBASE-25077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25077. Fix Version/s: 2.2.7 2.4.0 2.3.3 3.0.0-alpha-1 Resolution: Fixed Pushed to branch-2.2+. Thanks [~arshad.mohammad] for contributing. > hbck.jsp page loading fails, logs NPE in master log. > > > Key: HBASE-25077 > URL: https://issues.apache.org/jira/browse/HBASE-25077 > Project: HBase > Issue Type: Bug > Components: hbck >Affects Versions: 2.2.3 >Reporter: Ajeet Rai >Assignee: Mohammad Arshad >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.3, 2.4.0, 2.2.7 > > > While testing with large data, it is observed that some time hbck page was > not accessible and NPE was thrown in log. > > 【Test step】Excat Steps are not known. Observed this while executing below > steps > 1: Test data: 9K regions,~300 issues in my test setup > 2: run hbck_chore and catalogjanitor_run multiple times > 3: Also run balancer so that region movement is happening. > 4: Observe that HBCK page not available > > 2020-09-21 20:05:31,706 WARN [qtp1698322791-94] server.HttpChannel: /hbck.jsp > java.lang.NullPointerException > at > org.apache.hadoop.hbase.generated.master.hbck_jsp.formatServerName(hbck_jsp.java:37) > at > org.apache.hadoop.hbase.generated.master.hbck_jsp._jspService(hbck_jsp.java:144) > at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:111) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) > at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:844) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25098) ReplicationStatisticsChore runs in wrong time unit
[ https://issues.apache.org/jira/browse/HBASE-25098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25098. Resolution: Fixed Pushed to master. Thanks [~Ddupg] for contributing. > ReplicationStatisticsChore runs in wrong time unit > -- > > Key: HBASE-25098 > URL: https://issues.apache.org/jira/browse/HBASE-25098 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 3.0.0-alpha-1 >Reporter: Sun Xin >Assignee: Sun Xin >Priority: Major > Fix For: 3.0.0-alpha-1 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25074) Refactor ReplicationSinkManager: reduce code and make it easy to understand
[ https://issues.apache.org/jira/browse/HBASE-25074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25074. Fix Version/s: 3.0.0-alpha-1 Resolution: Fixed Pushed to master. Thanks [~zhangduo] and [~wchevreuil] for reviewing. > Refactor ReplicationSinkManager: reduce code and make it easy to understand > --- > > Key: HBASE-25074 > URL: https://issues.apache.org/jira/browse/HBASE-25074 > Project: HBase > Issue Type: Improvement >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 3.0.0-alpha-1 > > > ReplicationSinkManager only used in HBaseReplicationEndpoint. Move its logic > to HBaseReplicationEndpoint inside. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25086) Refactor Replication: move the default ReplicationSinkService implementation out
[ https://issues.apache.org/jira/browse/HBASE-25086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25086. Fix Version/s: 3.0.0-alpha-1 Resolution: Fixed Pushed to master. Thanks [~Ddupg] and [~meiyi] for reviewing. > Refactor Replication: move the default ReplicationSinkService implementation > out > > > Key: HBASE-25086 > URL: https://issues.apache.org/jira/browse/HBASE-25086 > Project: HBase > Issue Type: Improvement >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 3.0.0-alpha-1 > > > Now the Replication implements both ReplicationSourceService and > ReplicationSinkService interface. But most of code is only related to > ReplicationSourceService. Meanwhile, there is a bug when > HRegionServer#buildServerLoad, as it not consider the case: > ReplicationSourceService implementation is different with > ReplicationSinkService. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25086) Refactor Replication: move the default ReplicationSinkService implementation out
Guanghao Zhang created HBASE-25086: -- Summary: Refactor Replication: move the default ReplicationSinkService implementation out Key: HBASE-25086 URL: https://issues.apache.org/jira/browse/HBASE-25086 Project: HBase Issue Type: Improvement Reporter: Guanghao Zhang Now the Replication implements both ReplicationSourceService and ReplicationSinkService interface. But most of code is only related to ReplicationSourceService. Meanwhile, there is a bug when buildServerLoad, as it not consider the case: ReplicationSourceService implementation is different with ReplicationSinkService. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25074) Refactor ReplicationSinkManager: reduce code and make it easy to understand
Guanghao Zhang created HBASE-25074: -- Summary: Refactor ReplicationSinkManager: reduce code and make it easy to understand Key: HBASE-25074 URL: https://issues.apache.org/jira/browse/HBASE-25074 Project: HBase Issue Type: Improvement Reporter: Guanghao Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25071) ReplicationServer support start ReplicationSource internal
Guanghao Zhang created HBASE-25071: -- Summary: ReplicationServer support start ReplicationSource internal Key: HBASE-25071 URL: https://issues.apache.org/jira/browse/HBASE-25071 Project: HBase Issue Type: Sub-task Reporter: Guanghao Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24998) Introduce a ReplicationSourceController interface and decouple ReplicationSourceManager and ReplicationSource
[ https://issues.apache.org/jira/browse/HBASE-24998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-24998. Assignee: Guanghao Zhang Resolution: Fixed > Introduce a ReplicationSourceController interface and decouple > ReplicationSourceManager and ReplicationSource > -- > > Key: HBASE-24998 > URL: https://issues.apache.org/jira/browse/HBASE-24998 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25069) Display region name instead of encoded region name in HBCK report page.
[ https://issues.apache.org/jira/browse/HBASE-25069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25069. Fix Version/s: 2.4.0 2.3.3 3.0.0-alpha-1 Resolution: Fixed Pushed to branch-2.3+. Thanks [~arshad.mohammad] for contributing. > Display region name instead of encoded region name in HBCK report page. > > > Key: HBASE-25069 > URL: https://issues.apache.org/jira/browse/HBASE-25069 > Project: HBase > Issue Type: Improvement > Components: hbck >Affects Versions: 3.0.0-alpha-1, 2.3.1 >Reporter: Mohammad Arshad >Assignee: Mohammad Arshad >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.3.3, 2.4.0 > > Attachments: image-2020-09-19-11-39-01-755.png > > > In HMaster UI, in HBCK report, holes, overlaps and unknown server display > only encoded region names. > In display region encode name does not give any information. like which > table, what's the start key etc. > I think it is better to display the region name instead of region encoded > name. > !image-2020-09-19-11-39-01-755.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24991) Replace MovedRegionsCleaner with guava cache
[ https://issues.apache.org/jira/browse/HBASE-24991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-24991. Fix Version/s: 2.2.7 2.4.0 2.3.3 3.0.0-alpha-1 Resolution: Fixed Pushed to branch-2.2+. Thanks [~Joseph295] for contributing. > Replace MovedRegionsCleaner with guava cache > > > Key: HBASE-24991 > URL: https://issues.apache.org/jira/browse/HBASE-24991 > Project: HBase > Issue Type: Improvement >Reporter: Junhong Xu >Assignee: Junhong Xu >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.3.3, 2.4.0, 2.2.7 > > > Chore is a task executed periodically, here a cache with expired time is > enough. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25057) Fix typo "memeber"
[ https://issues.apache.org/jira/browse/HBASE-25057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25057. Fix Version/s: 2.2.7 2.4.0 2.3.3 3.0.0-alpha-1 Resolution: Fixed Pushed to branch-2.2+. Thanks [~doll6777] for contributing. > Fix typo "memeber" > -- > > Key: HBASE-25057 > URL: https://issues.apache.org/jira/browse/HBASE-25057 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Hyeran Lee >Assignee: Hyeran Lee >Priority: Trivial > Fix For: 3.0.0-alpha-1, 2.3.3, 2.4.0, 2.2.7 > > > Fix typo > Memeber -> Member in several comments -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25047) WAL split edits number is negative in RegionServerUI
[ https://issues.apache.org/jira/browse/HBASE-25047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25047. Fix Version/s: 2.2.7 2.4.0 2.3.3 3.0.0-alpha-1 Resolution: Fixed Pushed to branch-2.2+. Thanks [~Joseph295] for contributing. > WAL split edits number is negative in RegionServerUI > > > Key: HBASE-25047 > URL: https://issues.apache.org/jira/browse/HBASE-25047 > Project: HBase > Issue Type: Bug > Components: UI, wal >Reporter: Yi Mei >Assignee: Junhong Xu >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.3.3, 2.4.0, 2.2.7 > > Attachments: 2020-09-16 11-38-13屏幕截图.png > > > !2020-09-16 11-38-13屏幕截图.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24584) Release 2.2.6
[ https://issues.apache.org/jira/browse/HBASE-24584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-24584. Resolution: Fixed > Release 2.2.6 > - > > Key: HBASE-24584 > URL: https://issues.apache.org/jira/browse/HBASE-24584 > Project: HBase > Issue Type: Umbrella >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25036) Set version as 2.2.7-SNAPSHOT in branch-2.2
[ https://issues.apache.org/jira/browse/HBASE-25036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25036. Fix Version/s: 2.2.7 Resolution: Fixed Merged. Thanks all for reviewing. > Set version as 2.2.7-SNAPSHOT in branch-2.2 > --- > > Key: HBASE-25036 > URL: https://issues.apache.org/jira/browse/HBASE-25036 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 2.2.7 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25035) Add 2.2.6 to download page
[ https://issues.apache.org/jira/browse/HBASE-25035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25035. Resolution: Fixed Merged. Thanks all for reviewing. > Add 2.2.6 to download page > -- > > Key: HBASE-25035 > URL: https://issues.apache.org/jira/browse/HBASE-25035 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25012) HBASE-24359 causes replication missed log of some RemoteException
[ https://issues.apache.org/jira/browse/HBASE-25012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25012. Fix Version/s: 2.4.0 2.3.3 Resolution: Fixed Pushed to branch-2.3+. Thanks [~Ddupg] for contributing. > HBASE-24359 causes replication missed log of some RemoteException > - > > Key: HBASE-25012 > URL: https://issues.apache.org/jira/browse/HBASE-25012 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 3.0.0-alpha-1, 2.3.0, 2.3.1 >Reporter: Sun Xin >Assignee: Sun Xin >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.3, 2.4.0 > > Attachments: image-2020-09-11-14-30-27-898.png > > > HBASE-24359 broken the logic of handling exception. In branch2, it even > causes some RemoteException log missed. > [File > changed|[https://github.com/apache/hbase/pull/1855/files#diff-1e3f171b19474698601a0752b618af0eL435]] > in branch2. > !image-2020-09-11-14-30-27-898.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25009) Hbck chore logs wrong message when loading regions from RS report
[
https://issues.apache.org/jira/browse/HBASE-25009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang resolved HBASE-25009.
Fix Version/s: 2.2.7
2.4.0
2.3.3
3.0.0-alpha-1
Resolution: Fixed
Pushed to branch-2.2+. Thanks [~arshad.mohammad] for contributing.
> Hbck chore logs wrong message when loading regions from RS report
> -
>
> Key: HBASE-25009
> URL: https://issues.apache.org/jira/browse/HBASE-25009
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha-1, 2.3.1
>Reporter: Mohammad Arshad
>Assignee: Mohammad Arshad
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.3.3, 2.4.0, 2.2.7
>
>
> {code:java}
> LOG.info("Loaded {} regions from {} regionservers' reports and found {}
> orphan regions",
> numRegions, rsReports.size(), orphanRegionsOnFS.size());
> {code}
> In above log message orphanRegionsOnFS.size() should be replaced with
> orphanRegionsOnRS.size() as the regions are loaded from RS not form FS
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-25014) ScheduledChore is never triggered when initalDelay > 1.5*period
[ https://issues.apache.org/jira/browse/HBASE-25014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25014. Fix Version/s: 2.2.7 2.4.0 2.3.3 Resolution: Fixed > ScheduledChore is never triggered when initalDelay > 1.5*period > --- > > Key: HBASE-25014 > URL: https://issues.apache.org/jira/browse/HBASE-25014 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0-alpha-1, 2.2.3, 2.2.4, 2.2.5 >Reporter: Sun Xin >Assignee: Sun Xin >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.3, 2.4.0, 2.2.7 > > > In our recent tests, ScheduledChore is never triggered when initalDelay > > 1.5*period. > The cause of the bug is the following: > The trigger time for a ScheduleChore must be within an acceptable time window > that is 1.5 * period. see > [here|https://github.com/apache/hbase/blob/e5ca9adc54f9f580f85d21d38217afa97aa79d68/hbase-common/src/main/java/org/apache/hadoop/hbase/ScheduledChore.java#L234] > timeOfLastRun and timeOfThisRun are two variables that record two adjacent > trigger time. [The first initialization of > timeOfThisRun|https://github.com/apache/hbase/blob/e5ca9adc54f9f580f85d21d38217afa97aa79d68/hbase-common/src/main/java/org/apache/hadoop/hbase/ScheduledChore.java#L273] > is when the ScheduleChore is created, it's not a real trigger time. > If we set initialDelay > 1.5 period , after initialDelay, the first time when > chore is triggered has exceeded the allowed window. Then [cancel the chore > and schedule it > again|https://github.com/apache/hbase/blob/e5ca9adc54f9f580f85d21d38217afa97aa79d68/hbase-common/src/main/java/org/apache/hadoop/hbase/ChoreService.java#L176]. > So it's stuck in loop when initialDelay > 1.5 period : > 1. init timeOfThisRun at a wrong time. > 2. wait initalDelay > 3. chore trigger, but exceeded the allowed window. > 4. cancel chore and schedule it again > 5. go step 1. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25036) Set version as 2.2.7-SNAPSHOT in branch-2.2
Guanghao Zhang created HBASE-25036: -- Summary: Set version as 2.2.7-SNAPSHOT in branch-2.2 Key: HBASE-25036 URL: https://issues.apache.org/jira/browse/HBASE-25036 Project: HBase Issue Type: Sub-task Reporter: Guanghao Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25035) Add 2.2.6 to download page
Guanghao Zhang created HBASE-25035: -- Summary: Add 2.2.6 to download page Key: HBASE-25035 URL: https://issues.apache.org/jira/browse/HBASE-25035 Project: HBase Issue Type: Sub-task Reporter: Guanghao Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24982) Disassemble the method replicateWALEntry from AdminService to a new interface ReplicationServerService
[ https://issues.apache.org/jira/browse/HBASE-24982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-24982. Resolution: Fixed Merged. Thanks [~Ddupg] for contributing. > Disassemble the method replicateWALEntry from AdminService to a new interface > ReplicationServerService > -- > > Key: HBASE-24982 > URL: https://issues.apache.org/jira/browse/HBASE-24982 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Sun Xin >Assignee: Sun Xin >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24998) Introduce a ReplicationSourceOverallController interface and decouple ReplicationSourceManager and ReplicationSource
Guanghao Zhang created HBASE-24998: -- Summary: Introduce a ReplicationSourceOverallController interface and decouple ReplicationSourceManager and ReplicationSource Key: HBASE-24998 URL: https://issues.apache.org/jira/browse/HBASE-24998 Project: HBase Issue Type: Sub-task Reporter: Guanghao Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24656) [Flakey Tests] branch-2 TestMasterNoCluster.testStopDuringStart
[ https://issues.apache.org/jira/browse/HBASE-24656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-24656. Fix Version/s: 2.2.6 Resolution: Fixed Cherry-picked to branch-2.2. > [Flakey Tests] branch-2 TestMasterNoCluster.testStopDuringStart > --- > > Key: HBASE-24656 > URL: https://issues.apache.org/jira/browse/HBASE-24656 > Project: HBase > Issue Type: Bug >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Fix For: 2.2.6, 2.3.0 > > > org.apache.hadoop.hbase.master.TestMasterNoCluster.testStopDuringStart is > (only) flakey on branch-2 currently. Fails here: > Error Message > KeeperErrorCode = Directory not empty for /hbase/backup-masters > Stacktrace > org.apache.zookeeper.KeeperException$NotEmptyException: KeeperErrorCode = > Directory not empty for /hbase/backup-masters > at > org.apache.hadoop.hbase.master.TestMasterNoCluster.tearDown(TestMasterNoCluster.java:121) > I can see the zk events in teardown as we purge children as part of cleanup. > Can also see that the backup master registers later. Other than that, log is > opaque on why the teardown is failing. This is just clean up so adding in > retry to see if that helps. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (HBASE-24656) [Flakey Tests] branch-2 TestMasterNoCluster.testStopDuringStart
[ https://issues.apache.org/jira/browse/HBASE-24656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang reopened HBASE-24656: Reopen for branch-2.2. > [Flakey Tests] branch-2 TestMasterNoCluster.testStopDuringStart > --- > > Key: HBASE-24656 > URL: https://issues.apache.org/jira/browse/HBASE-24656 > Project: HBase > Issue Type: Bug >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Fix For: 2.3.0 > > > org.apache.hadoop.hbase.master.TestMasterNoCluster.testStopDuringStart is > (only) flakey on branch-2 currently. Fails here: > Error Message > KeeperErrorCode = Directory not empty for /hbase/backup-masters > Stacktrace > org.apache.zookeeper.KeeperException$NotEmptyException: KeeperErrorCode = > Directory not empty for /hbase/backup-masters > at > org.apache.hadoop.hbase.master.TestMasterNoCluster.tearDown(TestMasterNoCluster.java:121) > I can see the zk events in teardown as we purge children as part of cleanup. > Can also see that the backup master registers later. Other than that, log is > opaque on why the teardown is failing. This is just clean up so adding in > retry to see if that helps. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24980) Fix dead links in HBase book
[ https://issues.apache.org/jira/browse/HBASE-24980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-24980. Fix Version/s: (was: 2.3.2) 3.0.0-alpha-1 Resolution: Fixed Pushed to master branch. Thanks [~echohlne] for contributing. > Fix dead links in HBase book > > > Key: HBASE-24980 > URL: https://issues.apache.org/jira/browse/HBASE-24980 > Project: HBase > Issue Type: Bug > Components: documentation >Affects Versions: 2.3.0 >Reporter: echohlne >Assignee: echohlne >Priority: Major > Fix For: 3.0.0-alpha-1 > > > 1. > -[https://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/io/file/tfile/TFile.html|https://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/io/file/tfile/TFile.html-]- > => > [https://hadoop.apache.org/docs/current/api/org/apache/hadoop/io/file/tfile/TFile.html] > 2. -[https://vimeo.com/26804675|https://vimeo.com/26804675-]- => > [https://www.youtube.com/watch?v=DdGKAorSSZ0] > 3. > -[http://www.cloudera.com/videos/hw10_video_how_stumbleupon_built_and_advertising_platform_using_hbase_and_hadoop|http://www.cloudera.com/videos/hw10_video_how_stumbleupon_built_and_advertising_platform_using_hbase_and_hadoop-]- > has been invalid and cannot be found in other website, just remove it. > 4. > -[https://hadoop.apache.org/core/docs/stable/api/org/apache/hadoop/metrics/package-summary.html]- > => > [https://hadoop.apache.org/docs/stable/api/org/apache/hadoop/metrics2/package-summary.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24973) Remove read point parameter in method StoreFlush#performFlush and StoreFlush#createScanner
[
https://issues.apache.org/jira/browse/HBASE-24973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang resolved HBASE-24973.
Fix Version/s: 2.4.0
3.0.0-alpha-1
Resolution: Fixed
Pushed to branch-2+. Thanks [~yuqi] for contributing.
> Remove read point parameter in method StoreFlush#performFlush and
> StoreFlush#createScanner
> --
>
> Key: HBASE-24973
> URL: https://issues.apache.org/jira/browse/HBASE-24973
> Project: HBase
> Issue Type: Improvement
>Reporter: yuqi
>Assignee: yuqi
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
> Currently, read point parameter in method StoreFlush#performFlush is useless
> and can be safely removed.
> and then method StoreFlush#createScanner can also remove this parameter
> See below
> {code:java}
> // Some comments here
> /**
>* Performs memstore flush, writing data from scanner into sink.
>* @param scanner Scanner to get data from.
>* @param sink Sink to write data to. Could be StoreFile.Writer.
>* @param smallestReadPoint Smallest read point used for the flush.
>* @param throughputController A controller to avoid flush too fast
>*/
> protected void performFlush(InternalScanner scanner, CellSink sink,
> long smallestReadPoint, ThroughputController throughputController)
> throws IOException
> {code}
> Parameter smallestReadPoint is not used in this method. When
> `smallestReadPoint` is removed, inner method `createScanner` can remove this
> necessary parameter too
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-24831) Avoid invoke Counter using reflection in SnapshotInputFormat
[ https://issues.apache.org/jira/browse/HBASE-24831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-24831. Fix Version/s: 2.3.2 2.4.0 3.0.0-alpha-1 Resolution: Fixed Pushed to branch-2.3+. Thanks [~chenyechao] for contributing. > Avoid invoke Counter using reflection in SnapshotInputFormat > - > > Key: HBASE-24831 > URL: https://issues.apache.org/jira/browse/HBASE-24831 > Project: HBase > Issue Type: Improvement >Reporter: Yechao Chen >Assignee: Yechao Chen >Priority: Major > Labels: Performance, mapreduce, snapshot > Fix For: 3.0.0-alpha-1, 2.4.0, 2.3.2 > > > In TableRecordReaderImpl we invoke Counter increment by reflection > This will be called nextKeyValue() in TableSnapshotInputFormat > reflection invoke is very slower than normal method call > we can avoid these to improve the read performance -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24760) Add a config hbase.rsgroup.fallback.enable for RSGroup fallback feature
[ https://issues.apache.org/jira/browse/HBASE-24760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-24760. Fix Version/s: 2.4.0 Resolution: Fixed Pushed to branch-2+. Thanks [~Ddupg] for contributing. > Add a config hbase.rsgroup.fallback.enable for RSGroup fallback feature > --- > > Key: HBASE-24760 > URL: https://issues.apache.org/jira/browse/HBASE-24760 > Project: HBase > Issue Type: New Feature > Components: rsgroup >Affects Versions: 3.0.0-alpha-1 >Reporter: Sun Xin >Assignee: Sun Xin >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > In HBASE-22738 we allow tables fallback to specific rs groups, If there is no > online servers in the table's rsgroup. > -But for system tables, if there is no specified fallback rsgroup or the > servers in the fallback rsgroup all went down, It is necessary to allow > system tables fallback to any rsgroup in order to keey available at all > times.- > For Availability, refactor design of rsgroup fallback, finally only > introduced one config property `hbase.rsgroup.fallback.enable`, allow all > table, whether or not system tables, fallback to the default rsgroup first, > then fallback to any group if no online servers in default rsgroup. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24913) Refactor TestJMXConnectorServer
[ https://issues.apache.org/jira/browse/HBASE-24913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-24913. Fix Version/s: 2.3.2 2.2.7 Resolution: Fixed Pushed to branch-2.2+. Thanks [~Ddupg] for contributing. > Refactor TestJMXConnectorServer > --- > > Key: HBASE-24913 > URL: https://issues.apache.org/jira/browse/HBASE-24913 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 3.0.0-alpha-1 >Reporter: Sun Xin >Assignee: Sun Xin >Priority: Major > Fix For: 3.0.0-alpha-1, 2.2.7, 2.3.2 > > > Two optimization points for TestJMXConnectorServer in this issue: > # Just run cluster once, not once per test case. > # Use random free port to run ConnectorServer, avoid specifying a fixed port. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24689) Generate CHANGES.md and RELEASENOTES.md for 2.2.6
[ https://issues.apache.org/jira/browse/HBASE-24689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-24689. Resolution: Fixed > Generate CHANGES.md and RELEASENOTES.md for 2.2.6 > - > > Key: HBASE-24689 > URL: https://issues.apache.org/jira/browse/HBASE-24689 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 2.2.6 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (HBASE-24689) Generate CHANGES.md and RELEASENOTES.md for 2.2.6
[ https://issues.apache.org/jira/browse/HBASE-24689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang reopened HBASE-24689: > Generate CHANGES.md and RELEASENOTES.md for 2.2.6 > - > > Key: HBASE-24689 > URL: https://issues.apache.org/jira/browse/HBASE-24689 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 2.2.6 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24897) RegionReplicaFlushHandler should handle NoServerForRegionException to avoid aborting RegionServer
[
https://issues.apache.org/jira/browse/HBASE-24897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang resolved HBASE-24897.
Fix Version/s: 2.2.6
Resolution: Fixed
> RegionReplicaFlushHandler should handle NoServerForRegionException to avoid
> aborting RegionServer
> -
>
> Key: HBASE-24897
> URL: https://issues.apache.org/jira/browse/HBASE-24897
> Project: HBase
> Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 2.2.6
>
>
> Debug flaky test TestRegionReplicaReplicationEndpoint, I found the RS aborted
> because RegionReplicaFlushHandler flush failed. When create a new table with
> region replica, the assign order may be:
> # assign 0002 replica region and trigger primary region flush.
> # assign 0001 replica region and trigger primary region flush.
> # assign primary region.
> But the primary region flush may failed because the primary region not opened
> now. So it may abort the RS..
>
> {code:java}
> 2020-08-18 16:56:30,041 INFO
> [RS_OPEN_REGION-regionserver/hao-OptiPlex-7050:0-0]
> handler.AssignRegionHandler(141): Opened
> testRegionReplicaReplicationIgnoresDisabledTables_drop_false_disabledReplication_false,,1597740978463_0002.66e9757a05fbae7623cfea3369fc8354.
> 2020-08-18 16:56:30,558 INFO
> [RS_OPEN_REGION-regionserver/hao-OptiPlex-7050:0-0]
> handler.AssignRegionHandler(141): Opened
> testRegionReplicaReplicationIgnoresDisabledTables_drop_false_disabledReplication_false,,1597740978463_0001.22ff45423b0f1f0e93794f673449d140.
> 2020-08-18 16:56:31,192 INFO
> [RS_OPEN_REGION-regionserver/hao-OptiPlex-7050:0-0]
> handler.AssignRegionHandler(141): Opened
> testRegionReplicaReplicationIgnoresDisabledTables_drop_false_disabledReplication_false,,1597740978463.901f9cd06bbf27ef7c2d70b5af725cd2.
> 2020-08-18 16:58:53,857 ERROR
> [RS_REGION_REPLICA_FLUSH_OPS-regionserver/hao-OptiPlex-7050:0-0]
> helpers.MarkerIgnoringBase(159): * ABORTING region server
> hao-optiplex-7050,36368,1597740961432: ServerAborting because an exception
> was thrown *
> org.apache.hadoop.hbase.client.NoServerForRegionException: No server address
> listed in hbase:meta for region
> testRegionReplicaReplicationWithReplicas_10,,1597741128945.0f541dc1a7ca64797c4cf054adb9edfb.
> containing row
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.locateRegionInMeta(ConnectionImplementation.java:926)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.locateRegion(ConnectionImplementation.java:784)
> at
> org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.locateRegion(ConnectionUtils.java:140)
> at
> org.apache.hadoop.hbase.client.RegionAdminServiceCallable.getRegionLocations(RegionAdminServiceCallable.java:147)
> at
> org.apache.hadoop.hbase.client.RegionAdminServiceCallable.getLocation(RegionAdminServiceCallable.java:98)
> at
> org.apache.hadoop.hbase.client.RegionAdminServiceCallable.prepare(RegionAdminServiceCallable.java:84)
> at
> org.apache.hadoop.hbase.client.FlushRegionCallable.prepare(FlushRegionCallable.java:62)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:105)
> at
> org.apache.hadoop.hbase.regionserver.handler.RegionReplicaFlushHandler.triggerFlushInPrimaryRegion(RegionReplicaFlushHandler.java:129)
> at
> org.apache.hadoop.hbase.regionserver.handler.RegionReplicaFlushHandler.process(RegionReplicaFlushHandler.java:78)
> at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> I thought the fix should be assign primary region firstly when enable region
> replica featue. Will check the implmenation of region replica.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-24881) Fix flaky TestMasterAbortAndRSGotKilled for branch-2.2
[ https://issues.apache.org/jira/browse/HBASE-24881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-24881. Fix Version/s: 2.2.6 Resolution: Fixed > Fix flaky TestMasterAbortAndRSGotKilled for branch-2.2 > -- > > Key: HBASE-24881 > URL: https://issues.apache.org/jira/browse/HBASE-24881 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 2.2.6 > > > I meet this problem on branch-2.2 too. This case happened because the > DelayCloseCP. The event execute order is: > # Close regiong. But because the DelayCloseCP, it will close after 10 > seconds. > # Finish ut and shutdown cluster. > # Shutdown master. > # Shutdown RS. Call waitOnAllRegionsToClose method. But abortRequested is > false now. > # Close region and failed because master is down and report master error. > Then abort RegionServer and set abortRequested to ture. > # waitOnAllRegionsToClose hanged because the online regions cannot be empty. > > waitOnAllRegionsToClose(final boolean abort) already consider the abort case > but the problem is abortRequested is false when call this method. I thought > the fix should be that keep to check the abortRequested in > waitOnAllRegionsToClose method internal. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24870) Ignore TestAsyncTableRSCrashPublish
[ https://issues.apache.org/jira/browse/HBASE-24870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-24870. Fix Version/s: 2.2.6 Resolution: Fixed > Ignore TestAsyncTableRSCrashPublish > --- > > Key: HBASE-24870 > URL: https://issues.apache.org/jira/browse/HBASE-24870 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 2.2.6 > > > [ERROR] Failures: > [ERROR] TestAsyncTableRSCrashPublish.test:94 Waiting timed out after [60,000] > msec > > I meet this failure many times when runAllTests. And other developers meet > this too when vote RC. Let's ignore this first and enable this after parent > issue resolved. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-23987) NettyRpcClientConfigHelper will not share event loop by default which is incorrect
[ https://issues.apache.org/jira/browse/HBASE-23987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-23987. Fix Version/s: 2.2.6 Resolution: Fixed > NettyRpcClientConfigHelper will not share event loop by default which is > incorrect > -- > > Key: HBASE-23987 > URL: https://issues.apache.org/jira/browse/HBASE-23987 > Project: HBase > Issue Type: Bug > Components: Client, rpc >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.2.6, 2.3.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24928) balanceRSGroup should skip generating balance plan for disabled table and splitParent region
[
https://issues.apache.org/jira/browse/HBASE-24928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang resolved HBASE-24928.
Fix Version/s: 2.3.2
2.2.6
Resolution: Fixed
> balanceRSGroup should skip generating balance plan for disabled table and
> splitParent region
>
>
> Key: HBASE-24928
> URL: https://issues.apache.org/jira/browse/HBASE-24928
> Project: HBase
> Issue Type: Improvement
> Components: Balancer
>Reporter: niuyulin
>Assignee: niuyulin
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.2.6, 2.3.2
>
>
> now ,we generate balance plan for disabled tables, which is useless
> {code:java}
> 2020-08-20,20:47:54,702 WARN
> [RpcServer.default.RWQ.Fifo.read.handler=310,queue=6,port=22500]
> org.apache.hadoop.hbase.master.HMaster: Failed balance plan:
> hri=aa325467924edc865ab2ef6d82f9e2a7,
> source=tj1-hadoop-staging-st02.kscn,22600,1572403947348, destination=, just
> skip it
> org.apache.hadoop.hbase.client.DoNotRetryRegionException: Unexpected state
> for rit=CLOSED, location=tj1-hadoop-staging-st02.kscn,22600,1572403947348,
> table=galaxysds:sds_staging_258z, region=aa325467924edc865ab2ef6d82f9e2a7
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.preTransitCheck(AssignmentManager.java:580)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.createMoveRegionProcedure(AssignmentManager.java:635)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.moveAsync(AssignmentManager.java:652)
> at
> org.apache.hadoop.hbase.master.HMaster.executeRegionPlansWithThrottling(HMaster.java:1776)
> at
> org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.balanceRSGroup(RSGroupAdminServer.java:486)
> at
> org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.balanceRSGroup(RSGroupAdminEndpoint.java:293)
> at
> org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:13890)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:908)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:135)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-23814) Add null checks and logging to misc set of tests
[ https://issues.apache.org/jira/browse/HBASE-23814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-23814. Fix Version/s: 2.2.6 Resolution: Fixed > Add null checks and logging to misc set of tests > > > Key: HBASE-23814 > URL: https://issues.apache.org/jira/browse/HBASE-23814 > Project: HBase > Issue Type: Test >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Trivial > Fix For: 3.0.0-alpha-1, 2.2.6, 2.3.0 > > > I've been studying unit tests of late. A few are failing but then the output > is missing a detail or shutdown complains of NPE because startup didn't > succeed. > Here are super minor items I've been carrying around that I'd like to land. > They do not change the function of tests (there is an attempt at a fix of > TestLogsCleaner). > * TestFullLogReconstruction log the server we've chosen to expire and then > note where we starting counting rows > * TestAsyncTableScanException use a define for row counts; count 100 instead > of 1000 and see if helps > * TestRawAsyncTableLimitedScanWithFilter check connection was made before > closing it in tearDown > * TestLogsCleaner use single mod time. Make it for sure less than now in case > test runs all in the same millisecond (would cause test fail) > * TestReplicationBase test table is non-null before closing in tearDown -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (HBASE-23814) Add null checks and logging to misc set of tests
[ https://issues.apache.org/jira/browse/HBASE-23814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang reopened HBASE-23814: Reopen for cherry-pick to branch-2.2. > Add null checks and logging to misc set of tests > > > Key: HBASE-23814 > URL: https://issues.apache.org/jira/browse/HBASE-23814 > Project: HBase > Issue Type: Test >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Trivial > Fix For: 3.0.0-alpha-1, 2.3.0 > > > I've been studying unit tests of late. A few are failing but then the output > is missing a detail or shutdown complains of NPE because startup didn't > succeed. > Here are super minor items I've been carrying around that I'd like to land. > They do not change the function of tests (there is an attempt at a fix of > TestLogsCleaner). > * TestFullLogReconstruction log the server we've chosen to expire and then > note where we starting counting rows > * TestAsyncTableScanException use a define for row counts; count 100 instead > of 1000 and see if helps > * TestRawAsyncTableLimitedScanWithFilter check connection was made before > closing it in tearDown > * TestLogsCleaner use single mod time. Make it for sure less than now in case > test runs all in the same millisecond (would cause test fail) > * TestReplicationBase test table is non-null before closing in tearDown -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24948) Reduce the resource of TestReplicationBase
[ https://issues.apache.org/jira/browse/HBASE-24948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-24948. Fix Version/s: 2.2.6 Assignee: Guanghao Zhang Resolution: Fixed > Reduce the resource of TestReplicationBase > --- > > Key: HBASE-24948 > URL: https://issues.apache.org/jira/browse/HBASE-24948 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 2.2.6 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24946) Remove the metrics assert in TestClusterRestartFailover
[ https://issues.apache.org/jira/browse/HBASE-24946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-24946. Fix Version/s: 2.2.6 Resolution: Fixed > Remove the metrics assert in TestClusterRestartFailover > --- > > Key: HBASE-24946 > URL: https://issues.apache.org/jira/browse/HBASE-24946 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Priority: Major > Fix For: 2.2.6 > > > MetricsMasterSource masterSource = > UTIL.getHBaseCluster().getMaster().getMasterMetrics() > .getMetricsSource(); > metricsHelper.assertCounter(MetricsMasterSource.SERVER_CRASH_METRIC_PREFIX+"SubmittedCount", > 4, masterSource); > > Introduced by HBASE-24199. But flaky now as this unit test will restart all > clusters. Meanwhile, this metric already tested by TestMasterMetrics. I plan > to remove this assert for branch-2.2. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24948) Reduce the resource of TestReplicationBase
Guanghao Zhang created HBASE-24948: -- Summary: Reduce the resource of TestReplicationBase Key: HBASE-24948 URL: https://issues.apache.org/jira/browse/HBASE-24948 Project: HBase Issue Type: Sub-task Reporter: Guanghao Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24052) Add debug+fix to TestMasterShutdown
[ https://issues.apache.org/jira/browse/HBASE-24052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-24052. Fix Version/s: 2.2.6 Resolution: Fixed Pushed to branch-2.2. > Add debug+fix to TestMasterShutdown > --- > > Key: HBASE-24052 > URL: https://issues.apache.org/jira/browse/HBASE-24052 > Project: HBase > Issue Type: Sub-task >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Trivial > Fix For: 3.0.0-alpha-1, 2.2.6, 2.3.0 > > Attachments: > 0001-HBASE-24052-Add-debug-to-TestMasterShutdown.addendum.patch, > 0001-HBASE-24052-Add-debug-to-TestMasterShutdown.addendum2.patch, > 0001-HBASE-24052-Add-debug-to-TestMasterShutdown.patch > > > Temporarily add debug to TestMasterShutdown overnight to learn more about a > test failure not reproducible locally. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (HBASE-24052) Add debug+fix to TestMasterShutdown
[ https://issues.apache.org/jira/browse/HBASE-24052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang reopened HBASE-24052: Reopen for cherry-pick to branch-2.2. > Add debug+fix to TestMasterShutdown > --- > > Key: HBASE-24052 > URL: https://issues.apache.org/jira/browse/HBASE-24052 > Project: HBase > Issue Type: Sub-task >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Trivial > Fix For: 3.0.0-alpha-1, 2.3.0 > > Attachments: > 0001-HBASE-24052-Add-debug-to-TestMasterShutdown.addendum.patch, > 0001-HBASE-24052-Add-debug-to-TestMasterShutdown.addendum2.patch, > 0001-HBASE-24052-Add-debug-to-TestMasterShutdown.patch > > > Temporarily add debug to TestMasterShutdown overnight to learn more about a > test failure not reproducible locally. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24946) Remove the metrics assert in TestClusterRestartFailover
Guanghao Zhang created HBASE-24946: -- Summary: Remove the metrics assert in TestClusterRestartFailover Key: HBASE-24946 URL: https://issues.apache.org/jira/browse/HBASE-24946 Project: HBase Issue Type: Sub-task Reporter: Guanghao Zhang MetricsMasterSource masterSource = UTIL.getHBaseCluster().getMaster().getMasterMetrics() .getMetricsSource(); metricsHelper.assertCounter(MetricsMasterSource.SERVER_CRASH_METRIC_PREFIX+"SubmittedCount", 4, masterSource); Introduced by HBASE-24199. But flaky now as this unit test will restart all clusters. Meanwhile, this metric already tested by TestMasterMetrics. I plan to remove this assert for branch-2.2. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24871) Replication may loss data when refresh recovered replication sources
[
https://issues.apache.org/jira/browse/HBASE-24871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang resolved HBASE-24871.
Fix Version/s: 2.3.2
2.2.6
3.0.0-alpha-1
Resolution: Fixed
> Replication may loss data when refresh recovered replication sources
>
>
> Key: HBASE-24871
> URL: https://issues.apache.org/jira/browse/HBASE-24871
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 3.0.0-alpha-1, 2.3.0, 2.2.5
>Reporter: Guanghao Zhang
>Assignee: Sun Xin
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.2.6, 2.3.2
>
>
> See code in
> [https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java#L519]
>
> The WAL enqueued to wrong replication source.
> {code:java}
> for (String queueId : previousQueueIds) {
> ReplicationSourceInterface replicationSource = createSource(queueId,
> peer);
> this.oldsources.add(replicationSource);
> this.queueStorage.getWALsInQueue(this.server.getServerName(), queueId)
> .forEach(wal -> src.enqueueLog(new Path(wal)));
> toStartup.add(replicationSource);
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-24906) Enlarge the wait time in TestReplicationEndpoint/TestMetaWithReplicasBasic
[ https://issues.apache.org/jira/browse/HBASE-24906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-24906. Fix Version/s: 2.2.6 Resolution: Fixed > Enlarge the wait time in TestReplicationEndpoint/TestMetaWithReplicasBasic > -- > > Key: HBASE-24906 > URL: https://issues.apache.org/jira/browse/HBASE-24906 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 2.2.6 > > > Failed many times. But the failed reason are different. The replicated > entries number are different. So it means the replication is work and it need > more time to replicate all 2500 entries. > h3. Error Message > Waiting timed out after [30,000] msec Failed to replicate all edits, expected > = 2500 replicated = 2499 > > h3. Error Message > Waiting timed out after [30,000] msec Failed to replicate all edits, expected > = 2500 replicated = 2481 > > h3. Error Message > Waiting timed out after [30,000] msec Failed to replicate all edits, expected > = 2500 replicated = 2491 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24904) Speed up some unit tests
[ https://issues.apache.org/jira/browse/HBASE-24904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-24904. Fix Version/s: 2.2.6 Resolution: Fixed > Speed up some unit tests > > > Key: HBASE-24904 > URL: https://issues.apache.org/jira/browse/HBASE-24904 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 2.2.6 > > > See > [https://ci-hadoop.apache.org/job/HBase/job/HBase-Flaky-Tests/job/branch-2.2/42/testReport/org.apache.hadoop.hbase.client/TestAsyncTableAdminApi/] > [https://ci-hadoop.apache.org/job/HBase/job/HBase-Flaky-Tests/job/branch-2.2/61/testReport/junit/org.apache.hadoop.hbase.client/TestSnapshotTemporaryDirectoryWithRegionReplicas//] > > These ut are flaky because they take too much time which more than 780 > seconds. > > Split TestAsyncTableAdminApi/TestAdminShell/TestLoadIncrementalHFiles > > Reduce region numbers in > TestSnapshotTemporaryDirectoryWithRegionReplicas/TestRegionReplicaFailover/TestSCP* -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24912) Enlarge MemstoreFlusherChore/CompactionChecker period for unit test
[ https://issues.apache.org/jira/browse/HBASE-24912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-24912. Fix Version/s: 2.3.2 2.2.6 2.4.0 3.0.0-alpha-1 Resolution: Fixed Pushed to branch-2.2+. Thanks [~stack] for reviewing. > Enlarge MemstoreFlusherChore/CompactionChecker period for unit test > --- > > Key: HBASE-24912 > URL: https://issues.apache.org/jira/browse/HBASE-24912 > Project: HBase > Issue Type: Improvement >Reporter: Guanghao Zhang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0, 2.2.6, 2.3.2 > > > Too many debug logs when run unit test now. > > 2020-08-19 01:20:59,899 DEBUG [regionserver/asf909:0.Chore.1] > hbase.ScheduledChore(192): CompactionChecker execution time: 0 ms. > 2020-08-19 01:20:59,899 DEBUG [regionserver/asf909:0.Chore.1] > hbase.ScheduledChore(192): MemstoreFlusherChore execution time: 0 ms. > 2020-08-19 01:20:59,900 DEBUG [regionserver/asf909:0.Chore.1] > hbase.ScheduledChore(192): CompactionChecker execution time: 0 ms. > 2020-08-19 01:20:59,900 DEBUG [regionserver/asf909:0.Chore.1] > hbase.ScheduledChore(192): MemstoreFlusherChore execution time: 0 ms. > 2020-08-19 01:20:59,905 DEBUG [regionserver/asf909:0.Chore.1] > hbase.ScheduledChore(192): MemstoreFlusherChore execution time: 0 ms. > 2020-08-19 01:20:59,905 DEBUG [regionserver/asf909:0.Chore.1] > hbase.ScheduledChore(192): CompactionChecker execution time: 0 ms. > 2020-08-19 01:21:00,001 DEBUG [regionserver/asf909:0.Chore.1] > hbase.ScheduledChore(192): CompactionChecker execution time: 0 ms. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24895) Speed up TestFromClientSide3 by reduce the table regions number
[ https://issues.apache.org/jira/browse/HBASE-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-24895. Fix Version/s: 2.2.6 Resolution: Fixed > Speed up TestFromClientSide3 by reduce the table regions number > --- > > Key: HBASE-24895 > URL: https://issues.apache.org/jira/browse/HBASE-24895 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 2.2.6 > > > [https://ci-hadoop.apache.org/job/HBase/job/HBase-Flaky-Tests/job/branch-2.2/52/testReport/junit/org.apache.hadoop.hbase.client/TestFromClientSide3//] > > |[testHTableExistsMethodMultipleRegionsMultipleGets|https://ci-hadoop.apache.org/job/HBase/job/HBase-Flaky-Tests/job/branch-2.2/52/testReport/junit/org.apache.hadoop.hbase.client/TestFromClientSide3//testHTableExistsMethodMultipleRegionsMultipleGets]|2 > min 58 sec|Regression| > |[testHTableExistsMethodMultipleRegionsSingleGet|https://ci-hadoop.apache.org/job/HBase/job/HBase-Flaky-Tests/job/branch-2.2/52/testReport/junit/org.apache.hadoop.hbase.client/TestFromClientSide3//testHTableExistsMethodMultipleRegionsSingleGet]|4 > min 20 sec|Passed| > > It take too many time and timeout. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24912) Enalrge MemstoreFlusherChore/CompactionChecker period for unit test
Guanghao Zhang created HBASE-24912: -- Summary: Enalrge MemstoreFlusherChore/CompactionChecker period for unit test Key: HBASE-24912 URL: https://issues.apache.org/jira/browse/HBASE-24912 Project: HBase Issue Type: Improvement Reporter: Guanghao Zhang Too many debug logs when run unit test now. 2020-08-19 01:20:59,899 DEBUG [regionserver/asf909:0.Chore.1] hbase.ScheduledChore(192): CompactionChecker execution time: 0 ms. 2020-08-19 01:20:59,899 DEBUG [regionserver/asf909:0.Chore.1] hbase.ScheduledChore(192): MemstoreFlusherChore execution time: 0 ms. 2020-08-19 01:20:59,900 DEBUG [regionserver/asf909:0.Chore.1] hbase.ScheduledChore(192): CompactionChecker execution time: 0 ms. 2020-08-19 01:20:59,900 DEBUG [regionserver/asf909:0.Chore.1] hbase.ScheduledChore(192): MemstoreFlusherChore execution time: 0 ms. 2020-08-19 01:20:59,905 DEBUG [regionserver/asf909:0.Chore.1] hbase.ScheduledChore(192): MemstoreFlusherChore execution time: 0 ms. 2020-08-19 01:20:59,905 DEBUG [regionserver/asf909:0.Chore.1] hbase.ScheduledChore(192): CompactionChecker execution time: 0 ms. 2020-08-19 01:21:00,001 DEBUG [regionserver/asf909:0.Chore.1] hbase.ScheduledChore(192): CompactionChecker execution time: 0 ms. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24907) Turn off the balancer when test region admin api
[ https://issues.apache.org/jira/browse/HBASE-24907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-24907. Fix Version/s: 2.2.6 Resolution: Fixed > Turn off the balancer when test region admin api > > > Key: HBASE-24907 > URL: https://issues.apache.org/jira/browse/HBASE-24907 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Priority: Major > Fix For: 2.2.6 > > > For region admin api, we will test move/split/merge/assign/unassign and test > the region location right or not. But the balancer may move region to other > places and break the UT. So turn off the balancer for TestAsyncRegionAdminApi. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24907) Turn off the balancer when test region admin api
Guanghao Zhang created HBASE-24907: -- Summary: Turn off the balancer when test region admin api Key: HBASE-24907 URL: https://issues.apache.org/jira/browse/HBASE-24907 Project: HBase Issue Type: Sub-task Reporter: Guanghao Zhang For region admin api, we will test move/split/merge/assign/unassign and test the region location right or not. But the balancer may move region to other places and break the UT. So turn off the balancer for TestAsyncRegionAdminApi. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24906) Enalrge the wait time in TestReplicationEndpoint#testInterClusterReplication
Guanghao Zhang created HBASE-24906: -- Summary: Enalrge the wait time in TestReplicationEndpoint#testInterClusterReplication Key: HBASE-24906 URL: https://issues.apache.org/jira/browse/HBASE-24906 Project: HBase Issue Type: Sub-task Reporter: Guanghao Zhang Failed many times. But the failed reason are different. The replicated entries number are different. So it means the replication is work and it need more time to replicate all 2500 entries. h3. Error Message Waiting timed out after [30,000] msec Failed to replicate all edits, expected = 2500 replicated = 2499 h3. Error Message Waiting timed out after [30,000] msec Failed to replicate all edits, expected = 2500 replicated = 2481 h3. Error Message Waiting timed out after [30,000] msec Failed to replicate all edits, expected = 2500 replicated = 2491 -- This message was sent by Atlassian Jira (v8.3.4#803005)
