svn commit: r56717 - /dev/hbase/2.4.14RC0/
Author: huaxiangsun Date: Wed Sep 7 17:33:30 2022 New Revision: 56717 Log: Remove old 2.4.14 release candidate Removed: dev/hbase/2.4.14RC0/
[hbase] branch master updated: HBASE-27345 Add 2.4.14 to the downloads page (#4746)
This is an automated email from the ASF dual-hosted git repository. huaxiangsun pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hbase.git The following commit(s) were added to refs/heads/master by this push: new 61f02fc0117 HBASE-27345 Add 2.4.14 to the downloads page (#4746) 61f02fc0117 is described below commit 61f02fc011778035f3f992fd8dd22ce01fae40ec Author: huaxiangsun AuthorDate: Mon Aug 29 15:16:16 2022 -0700 HBASE-27345 Add 2.4.14 to the downloads page (#4746) Signed-off-by: stack --- src/site/xdoc/downloads.xml | 16 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/src/site/xdoc/downloads.xml b/src/site/xdoc/downloads.xml index 31c42e31e91..8137fe7e552 100644 --- a/src/site/xdoc/downloads.xml +++ b/src/site/xdoc/downloads.xml @@ -70,24 +70,24 @@ under the License. -2.4.13 +2.4.14 -2022/07/01 +2022/08/29 -https://downloads.apache.org/hbase/2.4.13/api_compare_2.4.12_to_2.4.13RC1.html;>2.4.13 vs 2.4.12 +https://downloads.apache.org/hbase/2.4.14/api_compare_2.4.13_to_2.4.14RC1.html;>2.4.14 vs 2.4.13 -https://downloads.apache.org/hbase/2.4.13/CHANGES.md;>Changes +https://downloads.apache.org/hbase/2.4.14/CHANGES.md;>Changes -https://downloads.apache.org/hbase/2.4.13/RELEASENOTES.md;>Release Notes +https://downloads.apache.org/hbase/2.4.14/RELEASENOTES.md;>Release Notes -https://www.apache.org/dyn/closer.lua/hbase/2.4.13/hbase-2.4.13-src.tar.gz;>src (https://downloads.apache.org/hbase/2.4.13/hbase-2.4.13-src.tar.gz.sha512;>sha512 https://downloads.apache.org/hbase/2.4.13/hbase-2.4.13-src.tar.gz.asc;>asc) -https://www.apache.org/dyn/closer.lua/hbase/2.4.13/hbase-2.4.13-bin.tar.gz;>bin (https://downloads.apache.org/hbase/2.4.13/hbase-2.4.13-bin.tar.gz.sha512;>sha512 https://downloads.apache.org/hbase/2.4.13/hbase-2.4.13-bin.tar.gz.asc;>asc) -https://www.apache.org/dyn/closer.lua/hbase/2.4.13/hbase-2.4.13-client-bin.tar.gz;>client-bin (https://downloads.apache.org/hbase/2.4.13/hbase-2.4.13-client-bin.tar.gz.sha512;>sha512 https://downloads.apache.org/hbase/2.4.13/hbase-2.4.13-client-bin.tar.gz.asc;>asc) +https://www.apache.org/dyn/closer.lua/hbase/2.4.14/hbase-2.4.14-src.tar.gz;>src (https://downloads.apache.org/hbase/2.4.14/hbase-2.4.14-src.tar.gz.sha512;>sha512 https://downloads.apache.org/hbase/2.4.14/hbase-2.4.14-src.tar.gz.asc;>asc) +https://www.apache.org/dyn/closer.lua/hbase/2.4.14/hbase-2.4.14-bin.tar.gz;>bin (https://downloads.apache.org/hbase/2.4.14/hbase-2.4.14-bin.tar.gz.sha512;>sha512 https://downloads.apache.org/hbase/2.4.14/hbase-2.4.14-bin.tar.gz.asc;>asc) +https://www.apache.org/dyn/closer.lua/hbase/2.4.14/hbase-2.4.14-client-bin.tar.gz;>client-bin (https://downloads.apache.org/hbase/2.4.14/hbase-2.4.14-client-bin.tar.gz.sha512;>sha512 https://downloads.apache.org/hbase/2.4.14/hbase-2.4.14-client-bin.tar.gz.asc;>asc) stable release
svn commit: r56537 - /release/hbase/2.4.13/
Author: huaxiangsun Date: Mon Aug 29 20:35:41 2022 New Revision: 56537 Log: Drop 2.4.13 release Removed: release/hbase/2.4.13/
[hbase] annotated tag rel/2.4.14 updated (2e7d75a8920 -> c0642a8bd63)
This is an automated email from the ASF dual-hosted git repository. huaxiangsun pushed a change to annotated tag rel/2.4.14 in repository https://gitbox.apache.org/repos/asf/hbase.git *** WARNING: tag rel/2.4.14 was modified! *** from 2e7d75a8920 (commit) to c0642a8bd63 (tag) tagging 811d188b9b0b54764be5855e7fd876d57fd5e90b (tag) length 989 bytes by Huaxiang Sun on Mon Aug 29 13:28:28 2022 -0700 - Log - Tag 2.4.14 -BEGIN PGP SIGNATURE- iQIzBAABCAAdFiEEZQ5ukLH6mbYnhA++I6kA1BF8g14FAmMNIWwACgkQI6kA1BF8 g14Drw//cXLvQOTGadJ7m0apzWvIHApt7lrY0zkKJyYjh2B+YMUs2Z0cqpZY+HL+ LdctLsIuUk2s3s0MK2fQl9BJLgmstbC53y6X87GxgSol5xSBLRZ96xJ3XiLNBmos s/Uf7dHeElBOK+US33/G49SU8ZL8D3Pe6LEL1CLDaQRPoKL00daKpD5a54PfpOyk 9Yu8LDAEKJnL/2I0KUWsYMz2sU2CI9Sj2NZUQha8yGYlOWgoj/NiMweduBjdz0DZ +sJ4Kr2PY54RMbhFcW2XbiIDXhMtzRCxKkpgn16IXivvALGzQfaZK4gSlMl1bhaC vuRhO6fOnOxAFHqcgulFlcsNu6O0C02AHQf67cN+Os4/Gl6+BZxokbSc5ygKaINr V9v9SAaCVeHE3n9Dwxn9NpWmlWRUZCdUN4ZNLsiXv/Z07GEDKx+F4+rXnpHlxYwy dGufGL5DiDE3u7ml8wTF4t8um+rxgwXhq9OynSBAO+5RRpyJXG4CrZavuj+aeDa3 26Z1mZYg/UdLN/Rv3otDqi198xwesDMvbDbiWJ9TzfhfkIH3WlyJYzjg37kYIPL8 FMWA7eydH3LSzjlCqM+GnmQoYYsfnYFkIM8B60B6g2r/IdCfwiO9NSJdo/WrHr1q JQ/BCVtmdKiBbHikdMuybsnmP39m35Ugkd1IOnP5WEaFEouhSVA= =fjFw -END PGP SIGNATURE- --- No new revisions were added by this update. Summary of changes:
svn commit: r56536 - /dev/hbase/2.4.14RC1/ /release/hbase/2.4.14/
Author: huaxiangsun Date: Mon Aug 29 19:18:10 2022 New Revision: 56536 Log: publish HBase 2.4.14 Added: release/hbase/2.4.14/ - copied from r56535, dev/hbase/2.4.14RC1/ Removed: dev/hbase/2.4.14RC1/
svn commit: r56475 [3/4] - /dev/hbase/2.4.14RC1/
Added: dev/hbase/2.4.14RC1/RELEASENOTES.md == --- dev/hbase/2.4.14RC1/RELEASENOTES.md (added) +++ dev/hbase/2.4.14RC1/RELEASENOTES.md Wed Aug 24 02:13:21 2022 @@ -0,0 +1,20159 @@ +# RELEASENOTES + + +# HBASE 2.4.14 Release Notes + +These release notes cover new developer and user-facing incompatibilities, important issues, features, and major improvements. + + +--- + +* [HBASE-27089](https://issues.apache.org/jira/browse/HBASE-27089) | *Minor* | **Add âcommons.crypto.stream.buffer.sizeâ configuration** + +Add a 'commons.crypto.stream.buffer.size' config for setting the buffer size when doing AES crypto for RPC. + + +--- + +* [HBASE-27281](https://issues.apache.org/jira/browse/HBASE-27281) | *Critical* | **Add default implementation for Connection$getClusterId** + +Adds a default null implementation for Connection$getClusterId. Downstream applications should implement this method. + + +--- + +* [HBASE-27251](https://issues.apache.org/jira/browse/HBASE-27251) | *Critical* | **Rolling back from 2.5.0-SNAPSHOT to 2.4.13 fails due to \`File does not exist: /hbase/MasterData/data/master/store/.initialized/.regioninfo\`** + +Before upgrading HBase cluster to 2.5.0, it is strongly recommended to upgrade the cluster to HBase-2.4.14 or later first. This is to guarantee a smooth rollback in case. Otherwise, it may run into HBASE-27251. + + +--- + +* [HBASE-27225](https://issues.apache.org/jira/browse/HBASE-27225) | *Major* | **Add BucketAllocator bucket size statistic logging** + +When bucket cache is enabled, setting loglevel to DEBUG for the org.apache.hadoop.hbase.io.hfile.bucket.BucketAllocator logger will result in a periodic summary of bucket usage for the bucket cache. This log output can give insight into block distribution of the cluster, where bucket cache memory is being allocated, and how much waste/fragmentation there is in the bucket cache. See Javadoc in BucketAllocate for more explanation of the logging. + + +--- + +* [HBASE-27204](https://issues.apache.org/jira/browse/HBASE-27204) | *Critical* | **BlockingRpcClient will hang for 20 seconds when SASL is enabled after finishing negotiation** + +When Kerberos authentication succeeds, on the server side, after receiving the final SASL token from the client, we simply wait for the client to continue by sending the connection header. After HBASE-24579, on the client side, an additional readStatus() was added, which assumed that after negotiation has completed a status code will be sent. However when authentication has succeeded the server will not send one. As a result the client would hang and only throw an exception when the configured read timeout is reached, which is 20 seconds by default. This was especially noticeable when using BlockingRpcClient as the client implementation. HBASE-24579 was reverted to correct this issue. + + +--- + +* [HBASE-27219](https://issues.apache.org/jira/browse/HBASE-27219) | *Minor* | **Change JONI encoding in RegexStringComparator** + +In RegexStringComparator an infinite loop can occur if an invalid UTF8 is encountered. We now use joni's NonStrictUTF8Encoding instead of UTF8Encoding to avoid the issue. + + +--- + +* [HBASE-27048](https://issues.apache.org/jira/browse/HBASE-27048) | *Major* | **Server side scanner time limit should account for time in queue** + +Server will now account for queue time when determining how long a scanner can run before heartbeat should be returned. This should help avoid timeouts when server is overloaded. + + +--- + +* [HBASE-27060](https://issues.apache.org/jira/browse/HBASE-27060) | *Major* | **Allow sharing connections between AggregationClient instances** + +Added a new no-arg constructor for AggregationClient which allows lightweight usage of just the methods accepting a Table. Also added a new constructor which takes a Connection, allowing the user to pass their own externally managed connection which will be used when methods accepting a TableName are used. + + +--- + +* [HBASE-26790](https://issues.apache.org/jira/browse/HBASE-26790) | *Major* | **getAllRegionLocations can cache locations with null hostname** + +getAllRegionLocations will no longer add locations to the MetaCache which have no server name. Region locations without a server name can occur for brief periods during region splits and merges, but the MetaCache relies on every cached location having a server name. + + +--- + +* [HBASE-26945](https://issues.apache.org/jira/browse/HBASE-26945) | *Minor* | **Quotas causes too much load on meta for large clusters** + +The quotas chore no longer scans meta to get information about region counts. Instead it uses the Admin API, which reads in-memory state in the HMaster and should scale well for large clusters. + + + +# HBASE 2.4.13 Release Notes + +These release notes cover new developer and user-facing incompatibilities, important issues, features, and
svn commit: r56475 [4/4] - /dev/hbase/2.4.14RC1/
Added: dev/hbase/2.4.14RC1/api_compare_2.4.13_to_2.4.14RC1.html
==
--- dev/hbase/2.4.14RC1/api_compare_2.4.13_to_2.4.14RC1.html (added)
+++ dev/hbase/2.4.14RC1/api_compare_2.4.13_to_2.4.14RC1.html Wed Aug 24
02:13:21 2022
@@ -0,0 +1,541 @@
+
+
+http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd;>
+http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
+
+
+
+
+
+hbase: rel/2.4.13 to 2.4.14RC1 compatibility report
+
+body {
+font-family:Arial, sans-serif;
+background-color:White;
+color:Black;
+}
+hr {
+color:Black;
+background-color:Black;
+height:1px;
+border:0;
+}
+h1 {
+margin-bottom:0px;
+padding-bottom:0px;
+font-size:1.625em;
+}
+h2 {
+margin-bottom:0px;
+padding-bottom:0px;
+font-size:1.25em;
+white-space:nowrap;
+}
+div.symbols {
+color:#003E69;
+}
+div.symbols i {
+color:Brown;
+}
+span.section {
+font-weight:bold;
+cursor:pointer;
+color:#003E69;
+white-space:nowrap;
+margin-left:0.3125em;
+}
+span:hover.section {
+color:#336699;
+}
+span.sect_aff {
+cursor:pointer;
+padding-left:1.55em;
+font-size:0.875em;
+color:#cc3300;
+}
+span.ext {
+font-weight:normal;
+}
+span.jar {
+color:#cc3300;
+font-size:0.875em;
+font-weight:bold;
+}
+div.jar_list {
+padding-left:0.4em;
+font-size:0.94em;
+}
+span.pkg_t {
+color:#408080;
+font-size:0.875em;
+}
+span.pkg {
+color:#408080;
+font-size:0.875em;
+font-weight:bold;
+}
+span.cname {
+color:Green;
+font-size:0.875em;
+font-weight:bold;
+}
+span.iname_b {
+font-weight:bold;
+}
+span.iname_a {
+color:#33;
+font-weight:bold;
+font-size:0.94em;
+}
+span.sym_p {
+font-weight:normal;
+white-space:normal;
+}
+span.sym_pd {
+white-space:normal;
+}
+span.sym_p span, span.sym_pd span {
+white-space:nowrap;
+}
+span.attr {
+color:Black;
+font-weight:normal;
+}
+span.deprecated {
+color:Red;
+font-weight:bold;
+font-family:Monaco, monospace;
+}
+div.affect {
+padding-left:1em;
+padding-bottom:10px;
+font-size:0.87em;
+font-style:italic;
+line-height:0.9em;
+}
+div.affected {
+padding-left:2em;
+padding-top:10px;
+}
+table.ptable {
+border-collapse:collapse;
+border:1px outset black;
+margin-left:0.95em;
+margin-top:3px;
+margin-bottom:3px;
+width:56.25em;
+}
+table.ptable td {
+border:1px solid Gray;
+padding:3px;
+font-size:0.875em;
+text-align:left;
+vertical-align:top;
+max-width:28em;
+word-wrap:break-word;
+}
+table.ptable th {
+background-color:#ee;
+font-weight:bold;
+color:#33;
+font-family:Verdana, Arial;
+font-size:0.875em;
+border:1px solid Gray;
+text-align:center;
+vertical-align:top;
+white-space:nowrap;
+padding:3px;
+}
+table.summary {
+border-collapse:collapse;
+border:1px outset black;
+}
+table.summary th {
+background-color:#ee;
+font-weight:normal;
+text-align:left;
+font-size:0.94em;
+white-space:nowrap;
+border:1px inset Gray;
+padding:3px;
+}
+table.summary td {
+text-align:right;
+white-space:nowrap;
+border:1px inset Gray;
+padding:3px 5px 3px 10px;
+}
+span.mngl {
+padding-left:1em;
+font-size:0.875em;
+cursor:text;
+color:#44;
+font-weight:bold;
+}
+span.pleft {
+padding-left:2.5em;
+}
+span.color_p {
+font-style:italic;
+color:Brown;
+}
+span.param {
+font-style:italic;
+}
+span.focus_p {
+font-style:italic;
+background-color:#DCDCDC;
+}
+span.ttype {
+font-weight:normal;
+}
+span.nowrap {
+white-space:nowrap;
+}
+span.value {
+white-space:nowrap;
+font-weight:bold;
+}
+.passed {
+background-color:#CCFFCC;
+font-weight:normal;
+}
+.warning {
+background-color:#F4F4AF;
+font-weight:normal;
+}
+.failed {
+background-color:#FF;
+font-weight:normal;
+}
+.new {
+background-color:#C6DEFF;
+font-weight:normal;
+}
+
+.compatible {
+background-color:#CCFFCC;
+font-weight:normal;
+}
+.almost_compatible {
+background-color:#FFDAA3;
+font-weight:normal;
+}
+.incompatible {
+background-color:#FF;
+font-weight:normal;
+}
+.gray {
+background-color:#DCDCDC;
+font-weight:normal;
+}
+
+.top_ref {
+font-size:0.69em;
+}
+.footer {
+font-size:0.8125em;
+}
+.tabset {
+float:left;
+}
+a.tab {
+border:1px solid Black;
+float:left;
+margin:0px 5px -1px 0px;
+padding:3px 5px 3px 5px;
+position:relative;
+font-size:0.875em;
+background-color:#DDD;
+text-decoration:none;
+color:Black;
+}
+a.disabled:hover
+{
+color:Black;
+background:#EEE;
+}
+a.active:hover
+{
+color:Black;
+background:White;
+}
+a.active {
+border-bottom-color:White;
+background-color:White;
+}
+div.tab {
+
svn commit: r56475 [2/4] - /dev/hbase/2.4.14RC1/
Added: dev/hbase/2.4.14RC1/CHANGES.md == --- dev/hbase/2.4.14RC1/CHANGES.md (added) +++ dev/hbase/2.4.14RC1/CHANGES.md Wed Aug 24 02:13:21 2022 @@ -0,0 +1,1382 @@ +# HBASE Changelog + + +## Release 2.4.14 - Unreleased (as of 2022-08-23) + + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | +|: |: | :--- |: | +| [HBASE-27089](https://issues.apache.org/jira/browse/HBASE-27089) | Add âcommons.crypto.stream.buffer.sizeâ configuration | Minor | io | +| [HBASE-27268](https://issues.apache.org/jira/browse/HBASE-27268) | In trace log mode, the client does not print callId/startTime and the server does not print receiveTime | Minor | logging | +| [HBASE-27296](https://issues.apache.org/jira/browse/HBASE-27296) | Some Cell's implementation of toString() such as IndividualBytesFieldCell prints out value and tags which is too verbose | Minor | logging | +| [HBASE-27273](https://issues.apache.org/jira/browse/HBASE-27273) | Should stop autoRead and skip all the bytes when rpc request too big | Major | IPC/RPC | +| [HBASE-27257](https://issues.apache.org/jira/browse/HBASE-27257) | Remove unnecessary usage of CachedBlocksByFile from RS UI | Major | . | +| [HBASE-27225](https://issues.apache.org/jira/browse/HBASE-27225) | Add BucketAllocator bucket size statistic logging | Major | . | +| [HBASE-27208](https://issues.apache.org/jira/browse/HBASE-27208) | Use spotless to purge the missing summary warnings from error prone | Major | pom | +| [HBASE-27048](https://issues.apache.org/jira/browse/HBASE-27048) | Server side scanner time limit should account for time in queue | Major | . | +| [HBASE-27188](https://issues.apache.org/jira/browse/HBASE-27188) | Report maxStoreFileCount in jmx | Minor | . | +| [HBASE-27186](https://issues.apache.org/jira/browse/HBASE-27186) | Report block cache size metrics separately for L1 and L2 | Minor | . | +| [HBASE-26218](https://issues.apache.org/jira/browse/HBASE-26218) | Better logging in CanaryTool | Minor | canary | +| [HBASE-27060](https://issues.apache.org/jira/browse/HBASE-27060) | Allow sharing connections between AggregationClient instances | Major | . | +| [HBASE-27146](https://issues.apache.org/jira/browse/HBASE-27146) | Avoid CellUtil.cloneRow in MetaCellComparator | Major | meta, Offheaping, Performance | +| [HBASE-26945](https://issues.apache.org/jira/browse/HBASE-26945) | Quotas causes too much load on meta for large clusters | Minor | . | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | +|: |: | :--- |: | +| [HBASE-27292](https://issues.apache.org/jira/browse/HBASE-27292) | Fix build failure against Hadoop 3.3.4 due to added dependency on okhttp | Major | build, hadoop3, pom | +| [HBASE-27244](https://issues.apache.org/jira/browse/HBASE-27244) | bin/hbase still use slf4j-log4j while reload4j in place | Major | shell | +| [HBASE-27275](https://issues.apache.org/jira/browse/HBASE-27275) | graceful\_stop.sh unable to restore the balance state | Blocker | regionserver | +| [HBASE-27282](https://issues.apache.org/jira/browse/HBASE-27282) | CME in AuthManager causes region server crash | Major | acl | +| [HBASE-26775](https://issues.apache.org/jira/browse/HBASE-26775) | TestProcedureSchedulerConcurrency fails in pre commit | Major | proc-v2, test | +| [HBASE-27269](https://issues.apache.org/jira/browse/HBASE-27269) | The implementation of TestReplicationStatus.waitOnMetricsReport is incorrect | Major | Replication, test | +| [HBASE-27271](https://issues.apache.org/jira/browse/HBASE-27271) | BufferCallBeforeInitHandler should ignore the flush request | Major | IPC/RPC | +| [HBASE-27251](https://issues.apache.org/jira/browse/HBASE-27251) | Rolling back from 2.5.0-SNAPSHOT to 2.4.13 fails due to \`File does not exist: /hbase/MasterData/data/master/store/.initialized/.regioninfo\` | Critical | master | +| [HBASE-27087](https://issues.apache.org/jira/browse/HBASE-27087) | TestQuotaThrottle times out | Major | test | +| [HBASE-27239](https://issues.apache.org/jira/browse/HBASE-27239) | Upgrade reload4j due to XXE vulnerability | Major | . | +| [HBASE-27204](https://issues.apache.org/jira/browse/HBASE-27204) | BlockingRpcClient will hang for 20 seconds when SASL is enabled after finishing negotiation | Critical | rpc, sasl, security | +| [HBASE-27219](https://issues.apache.org/jira/browse/HBASE-27219) | Change JONI encoding in RegexStringComparator | Minor | Filters | +| [HBASE-27205](https://issues.apache.org/jira/browse/HBASE-27205) | Fix tests that rely on EnvironmentEdgeManager in branch-2.4 | Minor | . | +| [HBASE-27211](https://issues.apache.org/jira/browse/HBASE-27211) | Data race in MonitoredTaskImpl could cause split wal failure | Critical | monitoring, wal | +| [HBASE-27053](https://issues.apache.org/jira/browse/HBASE-27053) | IOException during caching of uncompressed block to
svn commit: r56475 [1/4] - /dev/hbase/2.4.14RC1/
Author: huaxiangsun Date: Wed Aug 24 02:13:21 2022 New Revision: 56475 Log: Apache hbase 2.4.14RC1 Added: dev/hbase/2.4.14RC1/ dev/hbase/2.4.14RC1/CHANGES.md dev/hbase/2.4.14RC1/RELEASENOTES.md dev/hbase/2.4.14RC1/api_compare_2.4.13_to_2.4.14RC1.html dev/hbase/2.4.14RC1/hbase-2.4.14-bin.tar.gz (with props) dev/hbase/2.4.14RC1/hbase-2.4.14-bin.tar.gz.asc dev/hbase/2.4.14RC1/hbase-2.4.14-bin.tar.gz.sha512 dev/hbase/2.4.14RC1/hbase-2.4.14-client-bin.tar.gz (with props) dev/hbase/2.4.14RC1/hbase-2.4.14-client-bin.tar.gz.asc dev/hbase/2.4.14RC1/hbase-2.4.14-client-bin.tar.gz.sha512 dev/hbase/2.4.14RC1/hbase-2.4.14-src.tar.gz (with props) dev/hbase/2.4.14RC1/hbase-2.4.14-src.tar.gz.asc dev/hbase/2.4.14RC1/hbase-2.4.14-src.tar.gz.sha512
[hbase] 01/01: Preparing development version 2.4.15-SNAPSHOT
This is an automated email from the ASF dual-hosted git repository. huaxiangsun pushed a commit to branch branch-2.4 in repository https://gitbox.apache.org/repos/asf/hbase.git commit bcc725c56ce23a436f18a980d227986e096792f1 Author: Huaxiang Sun AuthorDate: Tue Aug 23 23:18:32 2022 + Preparing development version 2.4.15-SNAPSHOT Signed-off-by: Huaxiang Sun --- hbase-annotations/pom.xml | 2 +- hbase-archetypes/hbase-archetype-builder/pom.xml | 2 +- hbase-archetypes/hbase-client-project/pom.xml | 2 +- hbase-archetypes/hbase-shaded-client-project/pom.xml | 2 +- hbase-archetypes/pom.xml | 2 +- hbase-assembly/pom.xml | 2 +- hbase-asyncfs/pom.xml | 2 +- hbase-build-configuration/pom.xml | 2 +- hbase-checkstyle/pom.xml | 4 ++-- hbase-client/pom.xml | 2 +- hbase-common/pom.xml | 2 +- hbase-endpoint/pom.xml | 2 +- hbase-examples/pom.xml | 2 +- hbase-external-blockcache/pom.xml | 2 +- hbase-hadoop-compat/pom.xml| 2 +- hbase-hadoop2-compat/pom.xml | 2 +- hbase-hbtop/pom.xml| 2 +- hbase-http/pom.xml | 2 +- hbase-it/pom.xml | 2 +- hbase-logging/pom.xml | 2 +- hbase-mapreduce/pom.xml| 2 +- hbase-metrics-api/pom.xml | 2 +- hbase-metrics/pom.xml | 2 +- hbase-procedure/pom.xml| 2 +- hbase-protocol-shaded/pom.xml | 2 +- hbase-protocol/pom.xml | 2 +- hbase-replication/pom.xml | 2 +- hbase-resource-bundle/pom.xml | 2 +- hbase-rest/pom.xml | 2 +- hbase-rsgroup/pom.xml | 2 +- hbase-server/pom.xml | 2 +- hbase-shaded/hbase-shaded-check-invariants/pom.xml | 2 +- hbase-shaded/hbase-shaded-client-byo-hadoop/pom.xml| 2 +- hbase-shaded/hbase-shaded-client/pom.xml | 2 +- hbase-shaded/hbase-shaded-mapreduce/pom.xml| 2 +- hbase-shaded/hbase-shaded-testing-util-tester/pom.xml | 2 +- hbase-shaded/hbase-shaded-testing-util/pom.xml | 2 +- hbase-shaded/hbase-shaded-with-hadoop-check-invariants/pom.xml | 2 +- hbase-shaded/pom.xml | 2 +- hbase-shell/pom.xml| 2 +- hbase-testing-util/pom.xml | 2 +- hbase-thrift/pom.xml | 2 +- hbase-zookeeper/pom.xml| 2 +- pom.xml| 2 +- 44 files changed, 45 insertions(+), 45 deletions(-) diff --git a/hbase-annotations/pom.xml b/hbase-annotations/pom.xml index 37ca10415b9..bb67a145282 100644 --- a/hbase-annotations/pom.xml +++ b/hbase-annotations/pom.xml @@ -23,7 +23,7 @@ org.apache.hbase hbase -2.4.14 +2.4.15-SNAPSHOT .. diff --git a/hbase-archetypes/hbase-archetype-builder/pom.xml b/hbase-archetypes/hbase-archetype-builder/pom.xml index b67dbb29109..777d45c2787 100644 --- a/hbase-archetypes/hbase-archetype-builder/pom.xml +++ b/hbase-archetypes/hbase-archetype-builder/pom.xml @@ -24,7 +24,7 @@ org.apache.hbase hbase-archetypes -2.4.14 +2.4.15-SNAPSHOT .. diff --git a/hbase-archetypes/hbase-client-project/pom.xml b/hbase-archetypes/hbase-client-project/pom.xml index 9cec3a4934d..b15e57b384d 100644 --- a/hbase-archetypes/hbase-client-project/pom.xml +++ b/hbase-archetypes/hbase-client-project/pom.xml @@ -23,7 +23,7 @@ org.apache.hbase hbase-archetypes -2.4.14 +2.4.15-SNAPSHOT .. hbase-client-project diff --git a/hbase-archetypes/hbase-shaded-client-project/pom.xml b/hbase-archetypes/hbase-shaded-client-project/pom.xml index fb5e3f945ca..c13da88bffe 100644 --- a/hbase-archetypes/hbase-shaded-client-project/pom.xml +++ b/hbase-archetypes/hbase-shaded-client-project/pom.xml @@ -23,7 +23,7 @@ org.apache.hbase hbase-archetypes -2.4.14 +2.4.15-SNAPSHOT .. hbase
[hbase] branch branch-2.4 updated (0bcbc02b5b4 -> bcc725c56ce)
This is an automated email from the ASF dual-hosted git repository. huaxiangsun pushed a change to branch branch-2.4 in repository https://gitbox.apache.org/repos/asf/hbase.git from 0bcbc02b5b4 Preparing development version 2.4.15-SNAPSHOT add 2e7d75a8920 Preparing hbase release 2.4.14RC1; tagging and updates to CHANGES.md and RELEASENOTES.md new bcc725c56ce Preparing development version 2.4.15-SNAPSHOT The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.md | 11 ++- RELEASENOTES.md | 14 -- 2 files changed, 2 insertions(+), 23 deletions(-)
[hbase] annotated tag 2.4.14RC1 created (now 811d188b9b0)
This is an automated email from the ASF dual-hosted git repository. huaxiangsun pushed a change to annotated tag 2.4.14RC1 in repository https://gitbox.apache.org/repos/asf/hbase.git at 811d188b9b0 (tag) tagging 2e7d75a89271a7479b2f668c4db7a241be3f (commit) replaces 2.4.14RC0 by Huaxiang Sun on Tue Aug 23 23:18:18 2022 + - Log - Via create-release -BEGIN PGP SIGNATURE- iQIzBAABCgAdFiEEZQ5ukLH6mbYnhA++I6kA1BF8g14FAmMFYDoACgkQI6kA1BF8 g15VHBAAmLrwb+dfgXoKr+woQjPEj9hcCVn1UANJyw8dk7qXo6Yb1vft6ZFL900O nozVNhYspTyETU1Ir8jQM8W0cXBypxMjk0rlix5myLMyN3J577EWAa8PUiH/CND1 8iuXGY9a7ssAx6eoyGVQ7nkCqHMe6gAZGPLlOnrgX3A271lo4TFjsKfbcGsvvI8D sbxFzISeCLmTnrnfvwOJFydWjvT/FvPJxtY7N/xcvy9+gK/orm2qO0LcSJEoqNfP e2M/TbyNSmEZcb/V7hocH9xjkSdmQ8KL23F8NoUP8OGJZZJOjZU90zhcMW7g9cZL 7DbrWtVQOHMNx5o1LNgzM8PG//HLnKfZ9KpLcH+fXTJsLJcdm3JoOQpqIXOs+boN aeeKPzl1R7hDVKm0JgJFk9giNf89y5Bc+qacrxlRiw4m0WpNSxRq/JQ+vCSuyh8D XmdiUECCFWSjhDVf1yr5ZxfBHhVTSpE6BqjmgtHM0UCim1JLVA14AfvGYRxOGpww mucHd2l2uE7lOskHlGuTf/X19kZDrVmG9ugoL7OPWaKDicQ4ue4N8hAYKidjZQQY oeyOT4QbHqVxZ4/FesEyMFkYnD5YrjZKHdHAWEJbX/NKeCbT2cqLXmBgAdDJW+0x SNd3DPvmd4gHjkr2coNPD034cG9+e43KOXCS9cORltqdId8eYqE= =24KL -END PGP SIGNATURE- --- This annotated tag includes the following new commits: new 2e7d75a8920 Preparing hbase release 2.4.14RC1; tagging and updates to CHANGES.md and RELEASENOTES.md The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[hbase] 01/01: Preparing hbase release 2.4.14RC1; tagging and updates to CHANGES.md and RELEASENOTES.md
This is an automated email from the ASF dual-hosted git repository. huaxiangsun pushed a commit to annotated tag 2.4.14RC1 in repository https://gitbox.apache.org/repos/asf/hbase.git commit 2e7d75a89271a7479b2f668c4db7a241be3f Author: Huaxiang Sun AuthorDate: Tue Aug 23 23:18:18 2022 + Preparing hbase release 2.4.14RC1; tagging and updates to CHANGES.md and RELEASENOTES.md Signed-off-by: Huaxiang Sun --- CHANGES.md | 11 ++- RELEASENOTES.md| 14 -- hbase-annotations/pom.xml | 2 +- hbase-archetypes/hbase-archetype-builder/pom.xml | 2 +- hbase-archetypes/hbase-client-project/pom.xml | 2 +- hbase-archetypes/hbase-shaded-client-project/pom.xml | 2 +- hbase-archetypes/pom.xml | 2 +- hbase-assembly/pom.xml | 2 +- hbase-asyncfs/pom.xml | 2 +- hbase-build-configuration/pom.xml | 2 +- hbase-checkstyle/pom.xml | 4 ++-- hbase-client/pom.xml | 2 +- hbase-common/pom.xml | 2 +- hbase-endpoint/pom.xml | 2 +- hbase-examples/pom.xml | 2 +- hbase-external-blockcache/pom.xml | 2 +- hbase-hadoop-compat/pom.xml| 2 +- hbase-hadoop2-compat/pom.xml | 2 +- hbase-hbtop/pom.xml| 2 +- hbase-http/pom.xml | 2 +- hbase-it/pom.xml | 2 +- hbase-logging/pom.xml | 2 +- hbase-mapreduce/pom.xml| 2 +- hbase-metrics-api/pom.xml | 2 +- hbase-metrics/pom.xml | 2 +- hbase-procedure/pom.xml| 2 +- hbase-protocol-shaded/pom.xml | 2 +- hbase-protocol/pom.xml | 2 +- hbase-replication/pom.xml | 2 +- hbase-resource-bundle/pom.xml | 2 +- hbase-rest/pom.xml | 2 +- hbase-rsgroup/pom.xml | 2 +- hbase-server/pom.xml | 2 +- hbase-shaded/hbase-shaded-check-invariants/pom.xml | 2 +- hbase-shaded/hbase-shaded-client-byo-hadoop/pom.xml| 2 +- hbase-shaded/hbase-shaded-client/pom.xml | 2 +- hbase-shaded/hbase-shaded-mapreduce/pom.xml| 2 +- hbase-shaded/hbase-shaded-testing-util-tester/pom.xml | 2 +- hbase-shaded/hbase-shaded-testing-util/pom.xml | 2 +- .../hbase-shaded-with-hadoop-check-invariants/pom.xml | 2 +- hbase-shaded/pom.xml | 2 +- hbase-shell/pom.xml| 2 +- hbase-testing-util/pom.xml | 2 +- hbase-thrift/pom.xml | 2 +- hbase-zookeeper/pom.xml| 2 +- pom.xml| 2 +- 46 files changed, 47 insertions(+), 68 deletions(-) diff --git a/CHANGES.md b/CHANGES.md index 406d296d2d7..09ee748405b 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -20,17 +20,10 @@ # Be careful doing manual edits in this file. Do not change format # of release header or remove the below marker. This file is generated. # DO NOT REMOVE THIS MARKER; FOR INTERPOLATING CHANGES!--> -## Release 2.4.14 - Unreleased (as of 2022-08-22) +## Release 2.4.14 - Unreleased (as of 2022-08-23) -### NEW FEATURES: - -| JIRA | Summary | Priority | Component | -|: |: | :--- |: | -| [HBASE-27129](https://issues.apache.org/jira/browse/HBASE-27129) | Add a config that allows us to configure region-level storage policies | Major | regionserver | - - ### IMPROVEMENTS: | JIRA | Summary | Priority | Component | @@ -49,7 +42,6 @@ | [HBASE-27060](https://issues.apache.org/jira/browse/HBASE-27060) | Allow sharing connections between AggregationClient instances | Major | . | | [HBASE-27146](https://issues.apache.org/jira/browse/HBASE-27146) | Avoid CellUtil.cloneRow in MetaCellComparator | Major | meta, Offheaping, Performance | | [HBASE-26945](https://issues.apache.org/jira/browse/HBASE-26945) | Quotas causes too much load on meta for large clusters | Minor | . | -| [HBASE-27125](https://issues.apache.org/jira/browse/HBASE-27125) | The batch size of cleaning expired mob fi
svn commit: r56456 [3/4] - /dev/hbase/2.4.14RC0/
Added: dev/hbase/2.4.14RC0/RELEASENOTES.md == --- dev/hbase/2.4.14RC0/RELEASENOTES.md (added) +++ dev/hbase/2.4.14RC0/RELEASENOTES.md Mon Aug 22 21:59:28 2022 @@ -0,0 +1,20173 @@ +# RELEASENOTES + + +# HBASE 2.4.14 Release Notes + +These release notes cover new developer and user-facing incompatibilities, important issues, features, and major improvements. + + +--- + +* [HBASE-27129](https://issues.apache.org/jira/browse/HBASE-27129) | *Major* | **Add a config that allows us to configure region-level storage policies** + +Add a 'hbase.hregion.block.storage.policy' so you can config storage policy at region level. This is useful when you want to control the storage policy for the directories other than CF directories, such as .splits, .recovered.edits, etc. + + +--- + +* [HBASE-27089](https://issues.apache.org/jira/browse/HBASE-27089) | *Minor* | **Add âcommons.crypto.stream.buffer.sizeâ configuration** + +Add a 'commons.crypto.stream.buffer.size' config for setting the buffer size when doing AES crypto for RPC. + + +--- + +* [HBASE-27281](https://issues.apache.org/jira/browse/HBASE-27281) | *Critical* | **Add default implementation for Connection$getClusterId** + +Adds a default null implementation for Connection$getClusterId. Downstream applications should implement this method. + + +--- + +* [HBASE-27251](https://issues.apache.org/jira/browse/HBASE-27251) | *Critical* | **Rolling back from 2.5.0-SNAPSHOT to 2.4.13 fails due to \`File does not exist: /hbase/MasterData/data/master/store/.initialized/.regioninfo\`** + +Before upgrading HBase cluster to 2.5.0, it is strongly recommended to upgrade the cluster to HBase-2.4.14 or later first. This is to guarantee a smooth rollback in case. Otherwise, it may run into HBASE-27251. + + +--- + +* [HBASE-27225](https://issues.apache.org/jira/browse/HBASE-27225) | *Major* | **Add BucketAllocator bucket size statistic logging** + +When bucket cache is enabled, setting loglevel to DEBUG for the org.apache.hadoop.hbase.io.hfile.bucket.BucketAllocator logger will result in a periodic summary of bucket usage for the bucket cache. This log output can give insight into block distribution of the cluster, where bucket cache memory is being allocated, and how much waste/fragmentation there is in the bucket cache. See Javadoc in BucketAllocate for more explanation of the logging. + + +--- + +* [HBASE-27204](https://issues.apache.org/jira/browse/HBASE-27204) | *Critical* | **BlockingRpcClient will hang for 20 seconds when SASL is enabled after finishing negotiation** + +When Kerberos authentication succeeds, on the server side, after receiving the final SASL token from the client, we simply wait for the client to continue by sending the connection header. After HBASE-24579, on the client side, an additional readStatus() was added, which assumed that after negotiation has completed a status code will be sent. However when authentication has succeeded the server will not send one. As a result the client would hang and only throw an exception when the configured read timeout is reached, which is 20 seconds by default. This was especially noticeable when using BlockingRpcClient as the client implementation. HBASE-24579 was reverted to correct this issue. + + +--- + +* [HBASE-27219](https://issues.apache.org/jira/browse/HBASE-27219) | *Minor* | **Change JONI encoding in RegexStringComparator** + +In RegexStringComparator an infinite loop can occur if an invalid UTF8 is encountered. We now use joni's NonStrictUTF8Encoding instead of UTF8Encoding to avoid the issue. + + +--- + +* [HBASE-27048](https://issues.apache.org/jira/browse/HBASE-27048) | *Major* | **Server side scanner time limit should account for time in queue** + +Server will now account for queue time when determining how long a scanner can run before heartbeat should be returned. This should help avoid timeouts when server is overloaded. + + +--- + +* [HBASE-27060](https://issues.apache.org/jira/browse/HBASE-27060) | *Major* | **Allow sharing connections between AggregationClient instances** + +Added a new no-arg constructor for AggregationClient which allows lightweight usage of just the methods accepting a Table. Also added a new constructor which takes a Connection, allowing the user to pass their own externally managed connection which will be used when methods accepting a TableName are used. + + +--- + +* [HBASE-26790](https://issues.apache.org/jira/browse/HBASE-26790) | *Major* | **getAllRegionLocations can cache locations with null hostname** + +getAllRegionLocations will no longer add locations to the MetaCache which have no server name. Region locations without a server name can occur for brief periods during region splits and merges, but the MetaCache relies on every cached location having a server name. + + +--- + +* [HBASE-26945](https://issues.apache.org/jira/browse/HBASE-26945) |
svn commit: r56456 [1/4] - /dev/hbase/2.4.14RC0/
Author: huaxiangsun Date: Mon Aug 22 21:59:28 2022 New Revision: 56456 Log: Apache hbase 2.4.14RC0 Added: dev/hbase/2.4.14RC0/ dev/hbase/2.4.14RC0/CHANGES.md dev/hbase/2.4.14RC0/RELEASENOTES.md dev/hbase/2.4.14RC0/api_compare_2.4.13_to_2.4.14RC0.html dev/hbase/2.4.14RC0/hbase-2.4.14-bin.tar.gz (with props) dev/hbase/2.4.14RC0/hbase-2.4.14-bin.tar.gz.asc dev/hbase/2.4.14RC0/hbase-2.4.14-bin.tar.gz.sha512 dev/hbase/2.4.14RC0/hbase-2.4.14-client-bin.tar.gz (with props) dev/hbase/2.4.14RC0/hbase-2.4.14-client-bin.tar.gz.asc dev/hbase/2.4.14RC0/hbase-2.4.14-client-bin.tar.gz.sha512 dev/hbase/2.4.14RC0/hbase-2.4.14-src.tar.gz (with props) dev/hbase/2.4.14RC0/hbase-2.4.14-src.tar.gz.asc dev/hbase/2.4.14RC0/hbase-2.4.14-src.tar.gz.sha512
svn commit: r56456 [2/4] - /dev/hbase/2.4.14RC0/
Added: dev/hbase/2.4.14RC0/CHANGES.md == --- dev/hbase/2.4.14RC0/CHANGES.md (added) +++ dev/hbase/2.4.14RC0/CHANGES.md Mon Aug 22 21:59:28 2022 @@ -0,0 +1,1389 @@ +# HBASE Changelog + + +## Release 2.4.14 - Unreleased (as of 2022-08-22) + + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | +|: |: | :--- |: | +| [HBASE-27129](https://issues.apache.org/jira/browse/HBASE-27129) | Add a config that allows us to configure region-level storage policies | Major | regionserver | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | +|: |: | :--- |: | +| [HBASE-27089](https://issues.apache.org/jira/browse/HBASE-27089) | Add âcommons.crypto.stream.buffer.sizeâ configuration | Minor | io | +| [HBASE-27268](https://issues.apache.org/jira/browse/HBASE-27268) | In trace log mode, the client does not print callId/startTime and the server does not print receiveTime | Minor | logging | +| [HBASE-27296](https://issues.apache.org/jira/browse/HBASE-27296) | Some Cell's implementation of toString() such as IndividualBytesFieldCell prints out value and tags which is too verbose | Minor | logging | +| [HBASE-27273](https://issues.apache.org/jira/browse/HBASE-27273) | Should stop autoRead and skip all the bytes when rpc request too big | Major | IPC/RPC | +| [HBASE-27257](https://issues.apache.org/jira/browse/HBASE-27257) | Remove unnecessary usage of CachedBlocksByFile from RS UI | Major | . | +| [HBASE-27225](https://issues.apache.org/jira/browse/HBASE-27225) | Add BucketAllocator bucket size statistic logging | Major | . | +| [HBASE-27208](https://issues.apache.org/jira/browse/HBASE-27208) | Use spotless to purge the missing summary warnings from error prone | Major | pom | +| [HBASE-27048](https://issues.apache.org/jira/browse/HBASE-27048) | Server side scanner time limit should account for time in queue | Major | . | +| [HBASE-27188](https://issues.apache.org/jira/browse/HBASE-27188) | Report maxStoreFileCount in jmx | Minor | . | +| [HBASE-27186](https://issues.apache.org/jira/browse/HBASE-27186) | Report block cache size metrics separately for L1 and L2 | Minor | . | +| [HBASE-26218](https://issues.apache.org/jira/browse/HBASE-26218) | Better logging in CanaryTool | Minor | canary | +| [HBASE-27060](https://issues.apache.org/jira/browse/HBASE-27060) | Allow sharing connections between AggregationClient instances | Major | . | +| [HBASE-27146](https://issues.apache.org/jira/browse/HBASE-27146) | Avoid CellUtil.cloneRow in MetaCellComparator | Major | meta, Offheaping, Performance | +| [HBASE-26945](https://issues.apache.org/jira/browse/HBASE-26945) | Quotas causes too much load on meta for large clusters | Minor | . | +| [HBASE-27125](https://issues.apache.org/jira/browse/HBASE-27125) | The batch size of cleaning expired mob files should have an upper bound | Minor | mob | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | +|: |: | :--- |: | +| [HBASE-27292](https://issues.apache.org/jira/browse/HBASE-27292) | Fix build failure against Hadoop 3.3.4 due to added dependency on okhttp | Major | build, hadoop3, pom | +| [HBASE-27244](https://issues.apache.org/jira/browse/HBASE-27244) | bin/hbase still use slf4j-log4j while reload4j in place | Major | shell | +| [HBASE-27275](https://issues.apache.org/jira/browse/HBASE-27275) | graceful\_stop.sh unable to restore the balance state | Blocker | regionserver | +| [HBASE-27282](https://issues.apache.org/jira/browse/HBASE-27282) | CME in AuthManager causes region server crash | Major | acl | +| [HBASE-26775](https://issues.apache.org/jira/browse/HBASE-26775) | TestProcedureSchedulerConcurrency fails in pre commit | Major | proc-v2, test | +| [HBASE-27269](https://issues.apache.org/jira/browse/HBASE-27269) | The implementation of TestReplicationStatus.waitOnMetricsReport is incorrect | Major | Replication, test | +| [HBASE-27271](https://issues.apache.org/jira/browse/HBASE-27271) | BufferCallBeforeInitHandler should ignore the flush request | Major | IPC/RPC | +| [HBASE-27251](https://issues.apache.org/jira/browse/HBASE-27251) | Rolling back from 2.5.0-SNAPSHOT to 2.4.13 fails due to \`File does not exist: /hbase/MasterData/data/master/store/.initialized/.regioninfo\` | Critical | master | +| [HBASE-27087](https://issues.apache.org/jira/browse/HBASE-27087) | TestQuotaThrottle times out | Major | test | +| [HBASE-27239](https://issues.apache.org/jira/browse/HBASE-27239) | Upgrade reload4j due to XXE vulnerability | Major | . | +| [HBASE-27204](https://issues.apache.org/jira/browse/HBASE-27204) | BlockingRpcClient will hang for 20 seconds when SASL is enabled after finishing negotiation | Critical | rpc, sasl, security | +| [HBASE-27219](https://issues.apache.org/jira/browse/HBASE-27219) | Change JONI encoding in RegexStringComparator | Minor | Filters | +|
svn commit: r56456 [4/4] - /dev/hbase/2.4.14RC0/
Added: dev/hbase/2.4.14RC0/api_compare_2.4.13_to_2.4.14RC0.html
==
--- dev/hbase/2.4.14RC0/api_compare_2.4.13_to_2.4.14RC0.html (added)
+++ dev/hbase/2.4.14RC0/api_compare_2.4.13_to_2.4.14RC0.html Mon Aug 22
21:59:28 2022
@@ -0,0 +1,541 @@
+
+
+http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd;>
+http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
+
+
+
+
+
+hbase: rel/2.4.13 to 2.4.14RC0 compatibility report
+
+body {
+font-family:Arial, sans-serif;
+background-color:White;
+color:Black;
+}
+hr {
+color:Black;
+background-color:Black;
+height:1px;
+border:0;
+}
+h1 {
+margin-bottom:0px;
+padding-bottom:0px;
+font-size:1.625em;
+}
+h2 {
+margin-bottom:0px;
+padding-bottom:0px;
+font-size:1.25em;
+white-space:nowrap;
+}
+div.symbols {
+color:#003E69;
+}
+div.symbols i {
+color:Brown;
+}
+span.section {
+font-weight:bold;
+cursor:pointer;
+color:#003E69;
+white-space:nowrap;
+margin-left:0.3125em;
+}
+span:hover.section {
+color:#336699;
+}
+span.sect_aff {
+cursor:pointer;
+padding-left:1.55em;
+font-size:0.875em;
+color:#cc3300;
+}
+span.ext {
+font-weight:normal;
+}
+span.jar {
+color:#cc3300;
+font-size:0.875em;
+font-weight:bold;
+}
+div.jar_list {
+padding-left:0.4em;
+font-size:0.94em;
+}
+span.pkg_t {
+color:#408080;
+font-size:0.875em;
+}
+span.pkg {
+color:#408080;
+font-size:0.875em;
+font-weight:bold;
+}
+span.cname {
+color:Green;
+font-size:0.875em;
+font-weight:bold;
+}
+span.iname_b {
+font-weight:bold;
+}
+span.iname_a {
+color:#33;
+font-weight:bold;
+font-size:0.94em;
+}
+span.sym_p {
+font-weight:normal;
+white-space:normal;
+}
+span.sym_pd {
+white-space:normal;
+}
+span.sym_p span, span.sym_pd span {
+white-space:nowrap;
+}
+span.attr {
+color:Black;
+font-weight:normal;
+}
+span.deprecated {
+color:Red;
+font-weight:bold;
+font-family:Monaco, monospace;
+}
+div.affect {
+padding-left:1em;
+padding-bottom:10px;
+font-size:0.87em;
+font-style:italic;
+line-height:0.9em;
+}
+div.affected {
+padding-left:2em;
+padding-top:10px;
+}
+table.ptable {
+border-collapse:collapse;
+border:1px outset black;
+margin-left:0.95em;
+margin-top:3px;
+margin-bottom:3px;
+width:56.25em;
+}
+table.ptable td {
+border:1px solid Gray;
+padding:3px;
+font-size:0.875em;
+text-align:left;
+vertical-align:top;
+max-width:28em;
+word-wrap:break-word;
+}
+table.ptable th {
+background-color:#ee;
+font-weight:bold;
+color:#33;
+font-family:Verdana, Arial;
+font-size:0.875em;
+border:1px solid Gray;
+text-align:center;
+vertical-align:top;
+white-space:nowrap;
+padding:3px;
+}
+table.summary {
+border-collapse:collapse;
+border:1px outset black;
+}
+table.summary th {
+background-color:#ee;
+font-weight:normal;
+text-align:left;
+font-size:0.94em;
+white-space:nowrap;
+border:1px inset Gray;
+padding:3px;
+}
+table.summary td {
+text-align:right;
+white-space:nowrap;
+border:1px inset Gray;
+padding:3px 5px 3px 10px;
+}
+span.mngl {
+padding-left:1em;
+font-size:0.875em;
+cursor:text;
+color:#44;
+font-weight:bold;
+}
+span.pleft {
+padding-left:2.5em;
+}
+span.color_p {
+font-style:italic;
+color:Brown;
+}
+span.param {
+font-style:italic;
+}
+span.focus_p {
+font-style:italic;
+background-color:#DCDCDC;
+}
+span.ttype {
+font-weight:normal;
+}
+span.nowrap {
+white-space:nowrap;
+}
+span.value {
+white-space:nowrap;
+font-weight:bold;
+}
+.passed {
+background-color:#CCFFCC;
+font-weight:normal;
+}
+.warning {
+background-color:#F4F4AF;
+font-weight:normal;
+}
+.failed {
+background-color:#FF;
+font-weight:normal;
+}
+.new {
+background-color:#C6DEFF;
+font-weight:normal;
+}
+
+.compatible {
+background-color:#CCFFCC;
+font-weight:normal;
+}
+.almost_compatible {
+background-color:#FFDAA3;
+font-weight:normal;
+}
+.incompatible {
+background-color:#FF;
+font-weight:normal;
+}
+.gray {
+background-color:#DCDCDC;
+font-weight:normal;
+}
+
+.top_ref {
+font-size:0.69em;
+}
+.footer {
+font-size:0.8125em;
+}
+.tabset {
+float:left;
+}
+a.tab {
+border:1px solid Black;
+float:left;
+margin:0px 5px -1px 0px;
+padding:3px 5px 3px 5px;
+position:relative;
+font-size:0.875em;
+background-color:#DDD;
+text-decoration:none;
+color:Black;
+}
+a.disabled:hover
+{
+color:Black;
+background:#EEE;
+}
+a.active:hover
+{
+color:Black;
+background:White;
+}
+a.active {
+border-bottom-color:White;
+background-color:White;
+}
+div.tab {
+
[hbase] branch branch-2.4 updated (999cc74bd59 -> 0bcbc02b5b4)
This is an automated email from the ASF dual-hosted git repository. huaxiangsun pushed a change to branch branch-2.4 in repository https://gitbox.apache.org/repos/asf/hbase.git from 999cc74bd59 HBASE-27089 Add “commons.crypto.stream.buffer.size” configuration (#4491) add f507fb71c67 Preparing hbase release 2.4.14RC0; tagging and updates to CHANGES.md and RELEASENOTES.md new 0bcbc02b5b4 Preparing development version 2.4.15-SNAPSHOT The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.md | 90 ++ RELEASENOTES.md| 90 ++ hbase-annotations/pom.xml | 2 +- hbase-archetypes/hbase-archetype-builder/pom.xml | 2 +- hbase-archetypes/hbase-client-project/pom.xml | 2 +- .../hbase-shaded-client-project/pom.xml| 2 +- hbase-archetypes/pom.xml | 2 +- hbase-assembly/pom.xml | 2 +- hbase-asyncfs/pom.xml | 2 +- hbase-build-configuration/pom.xml | 2 +- hbase-checkstyle/pom.xml | 4 +- hbase-client/pom.xml | 2 +- hbase-common/pom.xml | 2 +- hbase-endpoint/pom.xml | 2 +- hbase-examples/pom.xml | 2 +- hbase-external-blockcache/pom.xml | 2 +- hbase-hadoop-compat/pom.xml| 2 +- hbase-hadoop2-compat/pom.xml | 2 +- hbase-hbtop/pom.xml| 2 +- hbase-http/pom.xml | 2 +- hbase-it/pom.xml | 2 +- hbase-logging/pom.xml | 2 +- hbase-mapreduce/pom.xml| 2 +- hbase-metrics-api/pom.xml | 2 +- hbase-metrics/pom.xml | 2 +- hbase-procedure/pom.xml| 2 +- hbase-protocol-shaded/pom.xml | 2 +- hbase-protocol/pom.xml | 2 +- hbase-replication/pom.xml | 2 +- hbase-resource-bundle/pom.xml | 2 +- hbase-rest/pom.xml | 2 +- hbase-rsgroup/pom.xml | 2 +- hbase-server/pom.xml | 2 +- hbase-shaded/hbase-shaded-check-invariants/pom.xml | 2 +- .../hbase-shaded-client-byo-hadoop/pom.xml | 2 +- hbase-shaded/hbase-shaded-client/pom.xml | 2 +- hbase-shaded/hbase-shaded-mapreduce/pom.xml| 2 +- .../hbase-shaded-testing-util-tester/pom.xml | 2 +- hbase-shaded/hbase-shaded-testing-util/pom.xml | 2 +- .../pom.xml| 2 +- hbase-shaded/pom.xml | 2 +- hbase-shell/pom.xml| 2 +- hbase-testing-util/pom.xml | 2 +- hbase-thrift/pom.xml | 2 +- hbase-zookeeper/pom.xml| 2 +- pom.xml| 2 +- 46 files changed, 225 insertions(+), 45 deletions(-)
[hbase] 01/01: Preparing development version 2.4.15-SNAPSHOT
This is an automated email from the ASF dual-hosted git repository. huaxiangsun pushed a commit to branch branch-2.4 in repository https://gitbox.apache.org/repos/asf/hbase.git commit 0bcbc02b5b46ec94bcc109e09844a6c9bcfc0fb0 Author: Huaxiang Sun AuthorDate: Mon Aug 22 19:38:35 2022 + Preparing development version 2.4.15-SNAPSHOT Signed-off-by: Huaxiang Sun --- hbase-annotations/pom.xml | 2 +- hbase-archetypes/hbase-archetype-builder/pom.xml | 2 +- hbase-archetypes/hbase-client-project/pom.xml | 2 +- hbase-archetypes/hbase-shaded-client-project/pom.xml | 2 +- hbase-archetypes/pom.xml | 2 +- hbase-assembly/pom.xml | 2 +- hbase-asyncfs/pom.xml | 2 +- hbase-build-configuration/pom.xml | 2 +- hbase-checkstyle/pom.xml | 4 ++-- hbase-client/pom.xml | 2 +- hbase-common/pom.xml | 2 +- hbase-endpoint/pom.xml | 2 +- hbase-examples/pom.xml | 2 +- hbase-external-blockcache/pom.xml | 2 +- hbase-hadoop-compat/pom.xml| 2 +- hbase-hadoop2-compat/pom.xml | 2 +- hbase-hbtop/pom.xml| 2 +- hbase-http/pom.xml | 2 +- hbase-it/pom.xml | 2 +- hbase-logging/pom.xml | 2 +- hbase-mapreduce/pom.xml| 2 +- hbase-metrics-api/pom.xml | 2 +- hbase-metrics/pom.xml | 2 +- hbase-procedure/pom.xml| 2 +- hbase-protocol-shaded/pom.xml | 2 +- hbase-protocol/pom.xml | 2 +- hbase-replication/pom.xml | 2 +- hbase-resource-bundle/pom.xml | 2 +- hbase-rest/pom.xml | 2 +- hbase-rsgroup/pom.xml | 2 +- hbase-server/pom.xml | 2 +- hbase-shaded/hbase-shaded-check-invariants/pom.xml | 2 +- hbase-shaded/hbase-shaded-client-byo-hadoop/pom.xml| 2 +- hbase-shaded/hbase-shaded-client/pom.xml | 2 +- hbase-shaded/hbase-shaded-mapreduce/pom.xml| 2 +- hbase-shaded/hbase-shaded-testing-util-tester/pom.xml | 2 +- hbase-shaded/hbase-shaded-testing-util/pom.xml | 2 +- hbase-shaded/hbase-shaded-with-hadoop-check-invariants/pom.xml | 2 +- hbase-shaded/pom.xml | 2 +- hbase-shell/pom.xml| 2 +- hbase-testing-util/pom.xml | 2 +- hbase-thrift/pom.xml | 2 +- hbase-zookeeper/pom.xml| 2 +- pom.xml| 2 +- 44 files changed, 45 insertions(+), 45 deletions(-) diff --git a/hbase-annotations/pom.xml b/hbase-annotations/pom.xml index 37ca10415b9..bb67a145282 100644 --- a/hbase-annotations/pom.xml +++ b/hbase-annotations/pom.xml @@ -23,7 +23,7 @@ org.apache.hbase hbase -2.4.14 +2.4.15-SNAPSHOT .. diff --git a/hbase-archetypes/hbase-archetype-builder/pom.xml b/hbase-archetypes/hbase-archetype-builder/pom.xml index b67dbb29109..777d45c2787 100644 --- a/hbase-archetypes/hbase-archetype-builder/pom.xml +++ b/hbase-archetypes/hbase-archetype-builder/pom.xml @@ -24,7 +24,7 @@ org.apache.hbase hbase-archetypes -2.4.14 +2.4.15-SNAPSHOT .. diff --git a/hbase-archetypes/hbase-client-project/pom.xml b/hbase-archetypes/hbase-client-project/pom.xml index 9cec3a4934d..b15e57b384d 100644 --- a/hbase-archetypes/hbase-client-project/pom.xml +++ b/hbase-archetypes/hbase-client-project/pom.xml @@ -23,7 +23,7 @@ org.apache.hbase hbase-archetypes -2.4.14 +2.4.15-SNAPSHOT .. hbase-client-project diff --git a/hbase-archetypes/hbase-shaded-client-project/pom.xml b/hbase-archetypes/hbase-shaded-client-project/pom.xml index fb5e3f945ca..c13da88bffe 100644 --- a/hbase-archetypes/hbase-shaded-client-project/pom.xml +++ b/hbase-archetypes/hbase-shaded-client-project/pom.xml @@ -23,7 +23,7 @@ org.apache.hbase hbase-archetypes -2.4.14 +2.4.15-SNAPSHOT .. hbase
[hbase] 01/01: Preparing hbase release 2.4.14RC0; tagging and updates to CHANGES.md and RELEASENOTES.md
This is an automated email from the ASF dual-hosted git repository. huaxiangsun pushed a commit to annotated tag 2.4.14RC0 in repository https://gitbox.apache.org/repos/asf/hbase.git commit f507fb71c6724ed44eb4ac25bbdd3e35b2441a1a Author: Huaxiang Sun AuthorDate: Mon Aug 22 19:38:24 2022 + Preparing hbase release 2.4.14RC0; tagging and updates to CHANGES.md and RELEASENOTES.md Signed-off-by: Huaxiang Sun --- CHANGES.md | 90 ++ RELEASENOTES.md| 90 ++ hbase-annotations/pom.xml | 2 +- hbase-archetypes/hbase-archetype-builder/pom.xml | 2 +- hbase-archetypes/hbase-client-project/pom.xml | 2 +- .../hbase-shaded-client-project/pom.xml| 2 +- hbase-archetypes/pom.xml | 2 +- hbase-assembly/pom.xml | 2 +- hbase-asyncfs/pom.xml | 2 +- hbase-build-configuration/pom.xml | 2 +- hbase-checkstyle/pom.xml | 4 +- hbase-client/pom.xml | 2 +- hbase-common/pom.xml | 2 +- hbase-endpoint/pom.xml | 2 +- hbase-examples/pom.xml | 2 +- hbase-external-blockcache/pom.xml | 2 +- hbase-hadoop-compat/pom.xml| 2 +- hbase-hadoop2-compat/pom.xml | 2 +- hbase-hbtop/pom.xml| 2 +- hbase-http/pom.xml | 2 +- hbase-it/pom.xml | 2 +- hbase-logging/pom.xml | 2 +- hbase-mapreduce/pom.xml| 2 +- hbase-metrics-api/pom.xml | 2 +- hbase-metrics/pom.xml | 2 +- hbase-procedure/pom.xml| 2 +- hbase-protocol-shaded/pom.xml | 2 +- hbase-protocol/pom.xml | 2 +- hbase-replication/pom.xml | 2 +- hbase-resource-bundle/pom.xml | 2 +- hbase-rest/pom.xml | 2 +- hbase-rsgroup/pom.xml | 2 +- hbase-server/pom.xml | 2 +- hbase-shaded/hbase-shaded-check-invariants/pom.xml | 2 +- .../hbase-shaded-client-byo-hadoop/pom.xml | 2 +- hbase-shaded/hbase-shaded-client/pom.xml | 2 +- hbase-shaded/hbase-shaded-mapreduce/pom.xml| 2 +- .../hbase-shaded-testing-util-tester/pom.xml | 2 +- hbase-shaded/hbase-shaded-testing-util/pom.xml | 2 +- .../pom.xml| 2 +- hbase-shaded/pom.xml | 2 +- hbase-shell/pom.xml| 2 +- hbase-testing-util/pom.xml | 2 +- hbase-thrift/pom.xml | 2 +- hbase-zookeeper/pom.xml| 2 +- pom.xml| 2 +- 46 files changed, 225 insertions(+), 45 deletions(-) diff --git a/CHANGES.md b/CHANGES.md index 2e77dc51d99..406d296d2d7 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -20,6 +20,96 @@ # Be careful doing manual edits in this file. Do not change format # of release header or remove the below marker. This file is generated. # DO NOT REMOVE THIS MARKER; FOR INTERPOLATING CHANGES!--> +## Release 2.4.14 - Unreleased (as of 2022-08-22) + + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | +|: |: | :--- |: | +| [HBASE-27129](https://issues.apache.org/jira/browse/HBASE-27129) | Add a config that allows us to configure region-level storage policies | Major | regionserver | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | +|: |: | :--- |: | +| [HBASE-27089](https://issues.apache.org/jira/browse/HBASE-27089) | Add “commons.crypto.stream.buffer.size” configuration | Minor | io | +| [HBASE-27268](https://issues.apache.org/jira/browse/HBASE-27268) | In trace log mode, the client does not print callId/startTime and the server does not print receiveTime | Minor | logging | +| [HBASE-27296](https://issues.apache.org/jira/browse/HBASE-27296) | Some Cell's implementation of toString() such as IndividualBytesFieldCell prints out value and tags which is too verbose | Minor | logging | +| [HBASE-27273](https://issues.apache.org/jira/browse/HBASE-27273) | Should stop autoRead and skip all the bytes when rpc request too big | Major | IPC/RPC | +| [HBASE-27257](https://issues.apache.org/jira/browse/HBASE-27257) | Remove unnecessary usage of CachedBlocksByFile from RS UI | Major | . | +| [HBASE-27225](https://issues.apache.org/jira/browse/HBASE-27225) | Add BucketAllocator buc
[hbase] annotated tag 2.4.14RC0 created (now 417d0732b48)
This is an automated email from the ASF dual-hosted git repository. huaxiangsun pushed a change to annotated tag 2.4.14RC0 in repository https://gitbox.apache.org/repos/asf/hbase.git at 417d0732b48 (tag) tagging f507fb71c6724ed44eb4ac25bbdd3e35b2441a1a (commit) replaces rel/2.4.13 by Huaxiang Sun on Mon Aug 22 19:38:24 2022 + - Log - Via create-release -BEGIN PGP SIGNATURE- iQIzBAABCgAdFiEEZQ5ukLH6mbYnhA++I6kA1BF8g14FAmMD2zAACgkQI6kA1BF8 g16mpRAAqMSBelk+htHx6ZOv0YrXkOnI/g6J6Irsos90shNn8UdFcAZ2Qq06Rlxg hOf2hWwfN663A3Fl5g4td9rlt07H6AnZu2HzHWeNgADex/nAZ1A31wu7dCOVqsek dMCsQTN6qOn23r5XxwNKayqP3lNMAtGfy/hLLC2V+vydDEXN5z/s8scu1deO3SGx sUXRFPamrpe4y5Jizr/yxjwPiXjHCkJlz4Fs/AZT32uuIOrcMOHRAXmI/gsYLofA fBVsfY8tFOmzt4TWCKqNrsqhMj2aKRfQP919gQZ/U6hG4NsHR7cyI6iWpdllQ7sF VtVINJOApDr+y62+qoJMq6ehjCfM5jzz15SDGgI5B94osQLYObn6lZAiiEaxDsWk +2w3zRFUERBe0MfTsU3yXc90NB1hSpPkWx4XBZJLgSK39NTNgWkrr86Hk1j+JBMK N4wzS6a0sOqEZjt3RbgfjcCKw4nzKyGwjfGfVAXnS0Kag0HDluU/oijMXYY7fc/r fr6mO/Cb/kr1eq66Er7XqhjMlATpB4h79PwfLNT44V3TdlLQWJHs0OtSm2SFr0pR SjNYWCKgZ7nMA709mGloEB6wt7mlvijdol7ihGMJ1ElFC9wxaMA1WtrTo3S2NDDD Qp5O2xOLpHixSIwvccQbYkom0kkhj0Vmtmc/2YK3ASuBQvOr/f0= =CFv8 -END PGP SIGNATURE- --- This annotated tag includes the following new commits: new f507fb71c67 Preparing hbase release 2.4.14RC0; tagging and updates to CHANGES.md and RELEASENOTES.md The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[hbase] branch branch-2.5 updated: HBASE-23723 Ensure MOB compaction works in optimized mode after snapshot clone (#4617) (#4704)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch branch-2.5
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2.5 by this push:
new e595f154d10 HBASE-23723 Ensure MOB compaction works in optimized mode
after snapshot clone (#4617) (#4704)
e595f154d10 is described below
commit e595f154d108f335c3025dee8ef85b8da586b0f2
Author: huaxiangsun
AuthorDate: Mon Aug 15 13:16:00 2022 -0700
HBASE-23723 Ensure MOB compaction works in optimized mode after snapshot
clone (#4617) (#4704)
* HBASE-23723 Ensure MOB compaction works in optimized mode after snapshot
clone (#1446)
* Reorganize MOB compaction tests for more reuse.
* Add tests for mob compaction after snapshot clone operations
* note the original table used to write a given mob hfile and use that to
find it later.
Signed-off-by: Esteban Gutierrez
* spotless:apply to fix HBaseTestingUtility
* Fix error-prone errors
Signed-off-by: Esteban Gutierrez
Co-authored-by: Sean Busbey
Signed-off-by: Esteban Gutierrez
Co-authored-by: Andrew Purtell
Co-authored-by: Sean Busbey
---
.../org/apache/hadoop/hbase/PrivateCellUtil.java | 2 +-
.../java/org/apache/hadoop/hbase/TableName.java| 60 +++-
.../hadoop/hbase/io/hfile/HFilePrettyPrinter.java | 8 +-
.../hadoop/hbase/mob/DefaultMobStoreCompactor.java | 127 +---
.../hadoop/hbase/mob/DefaultMobStoreFlusher.java | 6 +-
.../hadoop/hbase/mob/MobFileCleanerChore.java | 41 +--
.../java/org/apache/hadoop/hbase/mob/MobUtils.java | 150 --
.../hadoop/hbase/regionserver/HMobStore.java | 55 ++--
.../hadoop/hbase/regionserver/HStoreFile.java | 3 +-
.../hadoop/hbase/regionserver/StoreFileWriter.java | 17 +-
.../apache/hadoop/hbase/HBaseTestingUtility.java | 23 ++
.../hadoop/hbase/mob/FaultyMobStoreCompactor.java | 33 ++-
.../hadoop/hbase/mob/TestMobCompactionOptMode.java | 35 +--
.../mob/TestMobCompactionOptRegionBatchMode.java | 39 ++-
.../TestMobCompactionRegularRegionBatchMode.java | 38 +--
.../hbase/mob/TestMobCompactionWithDefaults.java | 322 +
16 files changed, 739 insertions(+), 220 deletions(-)
diff --git
a/hbase-common/src/main/java/org/apache/hadoop/hbase/PrivateCellUtil.java
b/hbase-common/src/main/java/org/apache/hadoop/hbase/PrivateCellUtil.java
index b3e70132dfc..001dfaae6d5 100644
--- a/hbase-common/src/main/java/org/apache/hadoop/hbase/PrivateCellUtil.java
+++ b/hbase-common/src/main/java/org/apache/hadoop/hbase/PrivateCellUtil.java
@@ -916,7 +916,7 @@ public final class PrivateCellUtil {
* Retrieve Cell's first tag, matching the passed in type
* @param cell The Cell
* @param type Type of the Tag to retrieve
- * @return null if there is no tag of the passed in tag type
+ * @return Optional, empty if there is no tag of the passed in tag type
*/
public static Optional getTag(Cell cell, byte type) {
boolean bufferBacked = cell instanceof ByteBufferExtendedCell;
diff --git a/hbase-common/src/main/java/org/apache/hadoop/hbase/TableName.java
b/hbase-common/src/main/java/org/apache/hadoop/hbase/TableName.java
index 18eba1ebb53..174a4169c0d 100644
--- a/hbase-common/src/main/java/org/apache/hadoop/hbase/TableName.java
+++ b/hbase-common/src/main/java/org/apache/hadoop/hbase/TableName.java
@@ -22,9 +22,12 @@ import java.nio.charset.StandardCharsets;
import java.util.Arrays;
import java.util.Set;
import java.util.concurrent.CopyOnWriteArraySet;
+import org.apache.commons.lang3.ArrayUtils;
import org.apache.hadoop.hbase.util.Bytes;
import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.hbase.thirdparty.com.google.common.base.Preconditions;
+
/**
* Immutable POJO class for representing a table name. Which is of the form:
table
* namespace:table qualifier Two special namespaces: 1. hbase -
system namespace, used
@@ -386,28 +389,69 @@ public final class TableName implements
Comparable {
}
/**
+ * @param fullName will use the entire byte array
* @throws IllegalArgumentException if fullName equals old root or old meta.
Some code depends on
* this. The test is buried in the table
creation to save on
* array comparison when we're creating a
standard table object
* that will be in the cache.
*/
public static TableName valueOf(byte[] fullName) throws
IllegalArgumentException {
+return valueOf(fullName, 0, fullName.length);
+ }
+
+ /**
+ * @param fullName byte array to look into
+ * @param offset within said array
+ * @param length within said array
+ * @throws IllegalArgumentException if fullName equals old root or old meta.
+ */
+ public static TableName valueOf(byte[] fullName, int offset, int length
[hbase] branch branch-2.4 updated: HBASE-27296 Some Cell's implementation of toString() such as IndividualBytesFieldCell prints out value and tags which is too verbose (#4695) (#4703)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2.4 by this push:
new a925b333c25 HBASE-27296 Some Cell's implementation of toString() such
as IndividualBytesFieldCell prints out value and tags which is too verbose
(#4695) (#4703)
a925b333c25 is described below
commit a925b333c25a8e9ad9abdfd6f48aa99586b14852
Author: huaxiangsun
AuthorDate: Mon Aug 15 09:17:10 2022 -0700
HBASE-27296 Some Cell's implementation of toString() such as
IndividualBytesFieldCell prints out value and tags which is too verbose (#4695)
(#4703)
Signed-off-by: Nick Dimiduk
---
.../src/main/java/org/apache/hadoop/hbase/ByteBufferKeyValue.java | 2 +-
.../src/main/java/org/apache/hadoop/hbase/IndividualBytesFieldCell.java | 2 +-
.../src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java | 2 +-
3 files changed, 3 insertions(+), 3 deletions(-)
diff --git
a/hbase-common/src/main/java/org/apache/hadoop/hbase/ByteBufferKeyValue.java
b/hbase-common/src/main/java/org/apache/hadoop/hbase/ByteBufferKeyValue.java
index 6c1c510ccb8..9c668c1cc3f 100644
--- a/hbase-common/src/main/java/org/apache/hadoop/hbase/ByteBufferKeyValue.java
+++ b/hbase-common/src/main/java/org/apache/hadoop/hbase/ByteBufferKeyValue.java
@@ -294,7 +294,7 @@ public class ByteBufferKeyValue extends
ByteBufferExtendedCell {
@Override
public String toString() {
-return CellUtil.toString(this, true);
+return CellUtil.toString(this, false);
}
@Override
diff --git
a/hbase-common/src/main/java/org/apache/hadoop/hbase/IndividualBytesFieldCell.java
b/hbase-common/src/main/java/org/apache/hadoop/hbase/IndividualBytesFieldCell.java
index 77d35db3ffb..8601232b09d 100644
---
a/hbase-common/src/main/java/org/apache/hadoop/hbase/IndividualBytesFieldCell.java
+++
b/hbase-common/src/main/java/org/apache/hadoop/hbase/IndividualBytesFieldCell.java
@@ -293,6 +293,6 @@ public class IndividualBytesFieldCell implements
ExtendedCell, Cloneable {
@Override
public String toString() {
-return CellUtil.toString(this, true);
+return CellUtil.toString(this, false);
}
}
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java
index b8a201d666d..4deee6e029d 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java
@@ -1506,7 +1506,7 @@ public abstract class HFileReaderImpl implements
HFile.Reader, Configurable {
@Override
public String getKeyString() {
- return CellUtil.toString(getKey(), true);
+ return CellUtil.toString(getKey(), false);
}
@Override
[hbase] branch branch-2.5 updated: HBASE-27296 Some Cell's implementation of toString() such as IndividualBytesFieldCell prints out value and tags which is too verbose (#4695) (#4702)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch branch-2.5
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2.5 by this push:
new 7f6f184f067 HBASE-27296 Some Cell's implementation of toString() such
as IndividualBytesFieldCell prints out value and tags which is too verbose
(#4695) (#4702)
7f6f184f067 is described below
commit 7f6f184f067f10c4b896d9e45311eee60f28833b
Author: huaxiangsun
AuthorDate: Mon Aug 15 09:16:51 2022 -0700
HBASE-27296 Some Cell's implementation of toString() such as
IndividualBytesFieldCell prints out value and tags which is too verbose (#4695)
(#4702)
Signed-off-by: Nick Dimiduk
---
.../src/main/java/org/apache/hadoop/hbase/ByteBufferKeyValue.java | 2 +-
.../src/main/java/org/apache/hadoop/hbase/IndividualBytesFieldCell.java | 2 +-
.../src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java | 2 +-
3 files changed, 3 insertions(+), 3 deletions(-)
diff --git
a/hbase-common/src/main/java/org/apache/hadoop/hbase/ByteBufferKeyValue.java
b/hbase-common/src/main/java/org/apache/hadoop/hbase/ByteBufferKeyValue.java
index 6c1c510ccb8..9c668c1cc3f 100644
--- a/hbase-common/src/main/java/org/apache/hadoop/hbase/ByteBufferKeyValue.java
+++ b/hbase-common/src/main/java/org/apache/hadoop/hbase/ByteBufferKeyValue.java
@@ -294,7 +294,7 @@ public class ByteBufferKeyValue extends
ByteBufferExtendedCell {
@Override
public String toString() {
-return CellUtil.toString(this, true);
+return CellUtil.toString(this, false);
}
@Override
diff --git
a/hbase-common/src/main/java/org/apache/hadoop/hbase/IndividualBytesFieldCell.java
b/hbase-common/src/main/java/org/apache/hadoop/hbase/IndividualBytesFieldCell.java
index 77d35db3ffb..8601232b09d 100644
---
a/hbase-common/src/main/java/org/apache/hadoop/hbase/IndividualBytesFieldCell.java
+++
b/hbase-common/src/main/java/org/apache/hadoop/hbase/IndividualBytesFieldCell.java
@@ -293,6 +293,6 @@ public class IndividualBytesFieldCell implements
ExtendedCell, Cloneable {
@Override
public String toString() {
-return CellUtil.toString(this, true);
+return CellUtil.toString(this, false);
}
}
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java
index 852306bd251..63d1cee2b13 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java
@@ -1510,7 +1510,7 @@ public abstract class HFileReaderImpl implements
HFile.Reader, Configurable {
@Override
public String getKeyString() {
- return CellUtil.toString(getKey(), true);
+ return CellUtil.toString(getKey(), false);
}
@Override
[hbase] branch branch-2 updated: HBASE-27296 Some Cell's implementation of toString() such as IndividualBytesFieldCell prints out value and tags which is too verbose (#4695) (#4701)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2 by this push:
new b25ded026b7 HBASE-27296 Some Cell's implementation of toString() such
as IndividualBytesFieldCell prints out value and tags which is too verbose
(#4695) (#4701)
b25ded026b7 is described below
commit b25ded026b7ce01e2b661352cce6464885a0d738
Author: huaxiangsun
AuthorDate: Mon Aug 15 09:16:29 2022 -0700
HBASE-27296 Some Cell's implementation of toString() such as
IndividualBytesFieldCell prints out value and tags which is too verbose (#4695)
(#4701)
Signed-off-by: Nick Dimiduk
---
.../src/main/java/org/apache/hadoop/hbase/ByteBufferKeyValue.java | 2 +-
.../src/main/java/org/apache/hadoop/hbase/IndividualBytesFieldCell.java | 2 +-
.../src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java | 2 +-
3 files changed, 3 insertions(+), 3 deletions(-)
diff --git
a/hbase-common/src/main/java/org/apache/hadoop/hbase/ByteBufferKeyValue.java
b/hbase-common/src/main/java/org/apache/hadoop/hbase/ByteBufferKeyValue.java
index 6c1c510ccb8..9c668c1cc3f 100644
--- a/hbase-common/src/main/java/org/apache/hadoop/hbase/ByteBufferKeyValue.java
+++ b/hbase-common/src/main/java/org/apache/hadoop/hbase/ByteBufferKeyValue.java
@@ -294,7 +294,7 @@ public class ByteBufferKeyValue extends
ByteBufferExtendedCell {
@Override
public String toString() {
-return CellUtil.toString(this, true);
+return CellUtil.toString(this, false);
}
@Override
diff --git
a/hbase-common/src/main/java/org/apache/hadoop/hbase/IndividualBytesFieldCell.java
b/hbase-common/src/main/java/org/apache/hadoop/hbase/IndividualBytesFieldCell.java
index 77d35db3ffb..8601232b09d 100644
---
a/hbase-common/src/main/java/org/apache/hadoop/hbase/IndividualBytesFieldCell.java
+++
b/hbase-common/src/main/java/org/apache/hadoop/hbase/IndividualBytesFieldCell.java
@@ -293,6 +293,6 @@ public class IndividualBytesFieldCell implements
ExtendedCell, Cloneable {
@Override
public String toString() {
-return CellUtil.toString(this, true);
+return CellUtil.toString(this, false);
}
}
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java
index 852306bd251..63d1cee2b13 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java
@@ -1510,7 +1510,7 @@ public abstract class HFileReaderImpl implements
HFile.Reader, Configurable {
@Override
public String getKeyString() {
- return CellUtil.toString(getKey(), true);
+ return CellUtil.toString(getKey(), false);
}
@Override
[hbase] branch branch-2 updated: HBASE-23723 Ensure MOB compaction works in optimized mode after snapshot clone (#4617)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2 by this push:
new f36b34984fc HBASE-23723 Ensure MOB compaction works in optimized mode
after snapshot clone (#4617)
f36b34984fc is described below
commit f36b34984fc6b08ffdc7abfbb7cc493010adb89b
Author: Andrew Purtell
AuthorDate: Fri Aug 12 14:29:26 2022 -0700
HBASE-23723 Ensure MOB compaction works in optimized mode after snapshot
clone (#4617)
* HBASE-23723 Ensure MOB compaction works in optimized mode after snapshot
clone (#1446)
* Reorganize MOB compaction tests for more reuse.
* Add tests for mob compaction after snapshot clone operations
* note the original table used to write a given mob hfile and use that to
find it later.
Signed-off-by: Esteban Gutierrez
* spotless:apply to fix HBaseTestingUtility
* Fix error-prone errors
Signed-off-by: Esteban Gutierrez
Co-authored-by: Sean Busbey
---
.../org/apache/hadoop/hbase/PrivateCellUtil.java | 2 +-
.../java/org/apache/hadoop/hbase/TableName.java| 60 +++-
.../hadoop/hbase/io/hfile/HFilePrettyPrinter.java | 8 +-
.../hadoop/hbase/mob/DefaultMobStoreCompactor.java | 127 +---
.../hadoop/hbase/mob/DefaultMobStoreFlusher.java | 6 +-
.../hadoop/hbase/mob/MobFileCleanerChore.java | 41 +--
.../java/org/apache/hadoop/hbase/mob/MobUtils.java | 150 --
.../hadoop/hbase/regionserver/HMobStore.java | 55 ++--
.../hadoop/hbase/regionserver/HStoreFile.java | 3 +-
.../hadoop/hbase/regionserver/StoreFileWriter.java | 17 +-
.../apache/hadoop/hbase/HBaseTestingUtility.java | 23 ++
.../hadoop/hbase/mob/FaultyMobStoreCompactor.java | 33 ++-
.../hadoop/hbase/mob/TestMobCompactionOptMode.java | 35 +--
.../mob/TestMobCompactionOptRegionBatchMode.java | 39 ++-
.../TestMobCompactionRegularRegionBatchMode.java | 38 +--
.../hbase/mob/TestMobCompactionWithDefaults.java | 322 +
16 files changed, 739 insertions(+), 220 deletions(-)
diff --git
a/hbase-common/src/main/java/org/apache/hadoop/hbase/PrivateCellUtil.java
b/hbase-common/src/main/java/org/apache/hadoop/hbase/PrivateCellUtil.java
index b3e70132dfc..001dfaae6d5 100644
--- a/hbase-common/src/main/java/org/apache/hadoop/hbase/PrivateCellUtil.java
+++ b/hbase-common/src/main/java/org/apache/hadoop/hbase/PrivateCellUtil.java
@@ -916,7 +916,7 @@ public final class PrivateCellUtil {
* Retrieve Cell's first tag, matching the passed in type
* @param cell The Cell
* @param type Type of the Tag to retrieve
- * @return null if there is no tag of the passed in tag type
+ * @return Optional, empty if there is no tag of the passed in tag type
*/
public static Optional getTag(Cell cell, byte type) {
boolean bufferBacked = cell instanceof ByteBufferExtendedCell;
diff --git a/hbase-common/src/main/java/org/apache/hadoop/hbase/TableName.java
b/hbase-common/src/main/java/org/apache/hadoop/hbase/TableName.java
index 18eba1ebb53..174a4169c0d 100644
--- a/hbase-common/src/main/java/org/apache/hadoop/hbase/TableName.java
+++ b/hbase-common/src/main/java/org/apache/hadoop/hbase/TableName.java
@@ -22,9 +22,12 @@ import java.nio.charset.StandardCharsets;
import java.util.Arrays;
import java.util.Set;
import java.util.concurrent.CopyOnWriteArraySet;
+import org.apache.commons.lang3.ArrayUtils;
import org.apache.hadoop.hbase.util.Bytes;
import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.hbase.thirdparty.com.google.common.base.Preconditions;
+
/**
* Immutable POJO class for representing a table name. Which is of the form:
table
* namespace:table qualifier Two special namespaces: 1. hbase -
system namespace, used
@@ -386,28 +389,69 @@ public final class TableName implements
Comparable {
}
/**
+ * @param fullName will use the entire byte array
* @throws IllegalArgumentException if fullName equals old root or old meta.
Some code depends on
* this. The test is buried in the table
creation to save on
* array comparison when we're creating a
standard table object
* that will be in the cache.
*/
public static TableName valueOf(byte[] fullName) throws
IllegalArgumentException {
+return valueOf(fullName, 0, fullName.length);
+ }
+
+ /**
+ * @param fullName byte array to look into
+ * @param offset within said array
+ * @param length within said array
+ * @throws IllegalArgumentException if fullName equals old root or old meta.
+ */
+ public static TableName valueOf(byte[] fullName, int offset, int length)
+throws IllegalArgumentException {
+Preconditions.checkArgument(offset >= 0, "offset must be non-negative but
[hbase] branch master updated: HBASE-27296 Some Cell's implementation of toString() such as IndividualBytesFieldCell prints out value and tags which is too verbose (#4695)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/master by this push:
new 73759be3960 HBASE-27296 Some Cell's implementation of toString() such
as IndividualBytesFieldCell prints out value and tags which is too verbose
(#4695)
73759be3960 is described below
commit 73759be396036014a92e7d174dfb34c2d2a91a7f
Author: huaxiangsun
AuthorDate: Fri Aug 12 12:17:14 2022 -0700
HBASE-27296 Some Cell's implementation of toString() such as
IndividualBytesFieldCell prints out value and tags which is too verbose (#4695)
Signed-off-by: Nick Dimiduk
---
.../src/main/java/org/apache/hadoop/hbase/ByteBufferKeyValue.java | 2 +-
.../src/main/java/org/apache/hadoop/hbase/IndividualBytesFieldCell.java | 2 +-
.../src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java | 2 +-
3 files changed, 3 insertions(+), 3 deletions(-)
diff --git
a/hbase-common/src/main/java/org/apache/hadoop/hbase/ByteBufferKeyValue.java
b/hbase-common/src/main/java/org/apache/hadoop/hbase/ByteBufferKeyValue.java
index 21c2468a286..24ae1ac9619 100644
--- a/hbase-common/src/main/java/org/apache/hadoop/hbase/ByteBufferKeyValue.java
+++ b/hbase-common/src/main/java/org/apache/hadoop/hbase/ByteBufferKeyValue.java
@@ -294,7 +294,7 @@ public class ByteBufferKeyValue extends
ByteBufferExtendedCell {
@Override
public String toString() {
-return CellUtil.toString(this, true);
+return CellUtil.toString(this, false);
}
@Override
diff --git
a/hbase-common/src/main/java/org/apache/hadoop/hbase/IndividualBytesFieldCell.java
b/hbase-common/src/main/java/org/apache/hadoop/hbase/IndividualBytesFieldCell.java
index b86b5e9c382..18cd6a8e4f9 100644
---
a/hbase-common/src/main/java/org/apache/hadoop/hbase/IndividualBytesFieldCell.java
+++
b/hbase-common/src/main/java/org/apache/hadoop/hbase/IndividualBytesFieldCell.java
@@ -288,6 +288,6 @@ public class IndividualBytesFieldCell implements
ExtendedCell, Cloneable {
@Override
public String toString() {
-return CellUtil.toString(this, true);
+return CellUtil.toString(this, false);
}
}
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java
index e72dada5403..a7afa785d9b 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java
@@ -1510,7 +1510,7 @@ public abstract class HFileReaderImpl implements
HFile.Reader, Configurable {
@Override
public String getKeyString() {
- return CellUtil.toString(getKey(), true);
+ return CellUtil.toString(getKey(), false);
}
@Override
[hbase] branch branch-2.4 updated: HBASE-27251 Rolling back from 2.5.0-SNAPSHOT to 2.4.13 fails due to 'File does not exist: /hbase/MasterData/data/master/store/.initialized/.regioninfo' Addendum (#46
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2.4 by this push:
new e93ff10b014 HBASE-27251 Rolling back from 2.5.0-SNAPSHOT to 2.4.13
fails due to 'File does not exist:
/hbase/MasterData/data/master/store/.initialized/.regioninfo' Addendum (#4665)
e93ff10b014 is described below
commit e93ff10b014701a12cd56f9f4893b3f18b914beb
Author: huaxiangsun
AuthorDate: Mon Aug 1 16:42:13 2022 -0700
HBASE-27251 Rolling back from 2.5.0-SNAPSHOT to 2.4.13 fails due to 'File
does not exist: /hbase/MasterData/data/master/store/.initialized/.regioninfo'
Addendum (#4665)
Signed-off-by: Andrew Purtell
---
.../java/org/apache/hadoop/hbase/master/region/MasterRegion.java| 6 ++
1 file changed, 6 insertions(+)
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/region/MasterRegion.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/region/MasterRegion.java
index daf1e5d1de6..a4801c9dec3 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/region/MasterRegion.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/region/MasterRegion.java
@@ -225,6 +225,12 @@ public final class MasterRegion {
FileSystem walFs, Path walRootDir, WALFactory walFactory,
MasterRegionWALRoller walRoller,
String serverName) throws IOException {
Path tableDir = CommonFSUtils.getTableDir(rootDir, td.getTableName());
+FileStatus[] nonRegionDirs = fs.listStatus(tableDir, p ->
p.getName().startsWith("."));
+if (nonRegionDirs.length > 0) {
+ LOG.warn("There are non-region directories under " + tableDir + ", such
as "
++ nonRegionDirs[0].getPath() + ", ignored");
+}
+
// on branch-2, the RegionInfo.isEncodedRegionName will returns true for
.initializing and
// .initialized, see HBASE-25368. Since RegionInfo is IA.Public, changing
the implementation may
// raise compatibility concerns, so here we just skip them by our own.
[hbase] branch branch-2.4 updated: HBASE-27251 Rolling back from 2.5.0-SNAPSHOT to 2.4.13 fails due to 'File does not exist: /hbase/MasterData/data/master/store/.initialized/.regioninfo' (#4663)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2.4 by this push:
new aa0fa58d062 HBASE-27251 Rolling back from 2.5.0-SNAPSHOT to 2.4.13
fails due to 'File does not exist:
/hbase/MasterData/data/master/store/.initialized/.regioninfo' (#4663)
aa0fa58d062 is described below
commit aa0fa58d062f789cd284ba6d38fc354f08de9e95
Author: huaxiangsun
AuthorDate: Thu Jul 28 10:42:14 2022 -0700
HBASE-27251 Rolling back from 2.5.0-SNAPSHOT to 2.4.13 fails due to 'File
does not exist: /hbase/MasterData/data/master/store/.initialized/.regioninfo'
(#4663)
Signed-off-by: Duo Zhang
---
.../java/org/apache/hadoop/hbase/master/region/MasterRegion.java | 8 +---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/region/MasterRegion.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/region/MasterRegion.java
index 3a31567a64f..daf1e5d1de6 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/region/MasterRegion.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/region/MasterRegion.java
@@ -225,9 +225,11 @@ public final class MasterRegion {
FileSystem walFs, Path walRootDir, WALFactory walFactory,
MasterRegionWALRoller walRoller,
String serverName) throws IOException {
Path tableDir = CommonFSUtils.getTableDir(rootDir, td.getTableName());
-Path regionDir =
- fs.listStatus(tableDir, p ->
RegionInfo.isEncodedRegionName(Bytes.toBytes(p.getName([0]
-.getPath();
+// on branch-2, the RegionInfo.isEncodedRegionName will returns true for
.initializing and
+// .initialized, see HBASE-25368. Since RegionInfo is IA.Public, changing
the implementation may
+// raise compatibility concerns, so here we just skip them by our own.
+Path regionDir = fs.listStatus(tableDir, p -> !p.getName().startsWith(".")
+ &&
RegionInfo.isEncodedRegionName(Bytes.toBytes(p.getName([0].getPath();
RegionInfo regionInfo = HRegionFileSystem.loadRegionInfoFileContent(fs,
regionDir);
Path walRegionDir = FSUtils.getRegionDirFromRootDir(walRootDir,
regionInfo);
[hbase] branch master updated: HBASE-27025 Change Hbase book's description for 74.7.3. Load Balancing META table load (#4530)
This is an automated email from the ASF dual-hosted git repository. huaxiangsun pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hbase.git The following commit(s) were added to refs/heads/master by this push: new d447fa01ba3 HBASE-27025 Change Hbase book's description for 74.7.3. Load Balancing META table load (#4530) d447fa01ba3 is described below commit d447fa01ba36a11d57927b78cce1bbca361b1d52 Author: huaxiangsun AuthorDate: Fri Jun 24 08:01:24 2022 -0700 HBASE-27025 Change Hbase book's description for 74.7.3. Load Balancing META table load (#4530) Signed-off-by: Bryan Beaudreault Signed-off-by: Duo Zhang --- src/main/asciidoc/_chapters/architecture.adoc | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/src/main/asciidoc/_chapters/architecture.adoc b/src/main/asciidoc/_chapters/architecture.adoc index 8eb7dfb689f..91192f9614c 100644 --- a/src/main/asciidoc/_chapters/architecture.adoc +++ b/src/main/asciidoc/_chapters/architecture.adoc @@ -2955,7 +2955,10 @@ client-side, clients will try to read META replicas first before falling back on the primary. Before this, the replica lookup mode -- now named `HedgedRead` in hbase-2.4.0 -- had clients read the primary and if no response after a configurable amount of time had elapsed, it would start up reads against the -replicas. +replicas. Starting from hbase-2.4.12(and all higher minor versions), with +client-side `LoadBalance` mode, clients load balance META scan requests across +all META replica regions, including the primary META region. In case of exceptions +such as NotServingRegionException, it will fall back on the primary META region. The new 'LoadBalance' mode helps alleviate hotspotting on the META table distributing META read load.
[hbase] branch branch-2.4 updated: HBASE-26649 Support meta replica LoadBalance mode for RegionLocator#getAllRegionLocations() (#4442) (#4485)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2.4 by this push:
new 976dbe8b2b4 HBASE-26649 Support meta replica LoadBalance mode for
RegionLocator#getAllRegionLocations() (#4442) (#4485)
976dbe8b2b4 is described below
commit 976dbe8b2b4bed0a998c3abbb92ae9a64ad22347
Author: huaxiangsun
AuthorDate: Fri Jun 3 15:28:19 2022 -0700
HBASE-26649 Support meta replica LoadBalance mode for
RegionLocator#getAllRegionLocations() (#4442) (#4485)
Signed-off-by: Duo Zhang
---
.../hadoop/hbase/AsyncMetaTableAccessor.java | 34 ++-
.../hbase/{client => }/CatalogReplicaMode.java | 4 +--
.../org/apache/hadoop/hbase/MetaTableAccessor.java | 38 ++
.../hbase/client/AsyncNonMetaRegionLocator.java| 1 +
.../hbase/client/ConnectionImplementation.java | 1 +
.../apache/hadoop/hbase/client/HRegionLocator.java | 5 ++-
.../client/TestAsyncNonMetaRegionLocator.java | 1 +
.../TestMetaRegionReplicaReplicationEndpoint.java | 20
8 files changed, 88 insertions(+), 16 deletions(-)
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/AsyncMetaTableAccessor.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/AsyncMetaTableAccessor.java
index 367d4dad785..414d4ee7b49 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/AsyncMetaTableAccessor.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/AsyncMetaTableAccessor.java
@@ -17,6 +17,7 @@
*/
package org.apache.hadoop.hbase;
+import static
org.apache.hadoop.hbase.client.RegionLocator.LOCATOR_META_REPLICAS_MODE;
import static org.apache.hadoop.hbase.util.FutureUtils.addListener;
import java.io.IOException;
@@ -29,6 +30,7 @@ import java.util.NavigableMap;
import java.util.Optional;
import java.util.SortedMap;
import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ThreadLocalRandom;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import java.util.stream.Collectors;
@@ -295,7 +297,37 @@ public class AsyncMetaTableAccessor {
}
CompletableFuture future = new CompletableFuture();
-metaTable.scan(scan, new MetaTableScanResultConsumer(rowUpperLimit,
visitor, future));
+// Get the region locator's meta replica mode.
+CatalogReplicaMode metaReplicaMode =
CatalogReplicaMode.fromString(metaTable.getConfiguration()
+ .get(LOCATOR_META_REPLICAS_MODE, CatalogReplicaMode.NONE.toString()));
+
+if (metaReplicaMode == CatalogReplicaMode.LOAD_BALANCE) {
+ addListener(metaTable.getDescriptor(), (desc, error) -> {
+if (error != null) {
+ LOG.error("Failed to get meta table descriptor, error: ", error);
+ future.completeExceptionally(error);
+ return;
+}
+
+int numOfReplicas = desc.getRegionReplication();
+if (numOfReplicas > 1) {
+ int replicaId = ThreadLocalRandom.current().nextInt(numOfReplicas);
+
+ // When the replicaId is 0, do not set to Consistency.TIMELINE
+ if (replicaId > 0) {
+scan.setReplicaId(replicaId);
+scan.setConsistency(Consistency.TIMELINE);
+ }
+}
+metaTable.scan(scan, new MetaTableScanResultConsumer(rowUpperLimit,
visitor, future));
+ });
+} else {
+ if (metaReplicaMode == CatalogReplicaMode.HEDGED_READ) {
+scan.setConsistency(Consistency.TIMELINE);
+ }
+ metaTable.scan(scan, new MetaTableScanResultConsumer(rowUpperLimit,
visitor, future));
+}
+
return future;
}
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/CatalogReplicaMode.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/CatalogReplicaMode.java
similarity index 96%
rename from
hbase-client/src/main/java/org/apache/hadoop/hbase/client/CatalogReplicaMode.java
rename to
hbase-client/src/main/java/org/apache/hadoop/hbase/CatalogReplicaMode.java
index 647d5dcf38f..b89673d45a8 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/CatalogReplicaMode.java
+++ b/hbase-client/src/main/java/org/apache/hadoop/hbase/CatalogReplicaMode.java
@@ -15,7 +15,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.hadoop.hbase.client;
+package org.apache.hadoop.hbase;
import org.apache.yetus.audience.InterfaceAudience;
@@ -33,7 +33,7 @@ import org.apache.yetus.audience.InterfaceAudience;
*
*/
@InterfaceAudience.Private
-enum CatalogReplicaMode {
+public enum CatalogReplicaMode {
NONE {
@Override
public String toString() {
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTa
[hbase] branch branch-2.5 updated: HBASE-26649 Support meta replica LoadBalance mode for RegionLocator#getAllRegionLocations() (#4442) (#4484)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch branch-2.5
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2.5 by this push:
new 5a13ad8137a HBASE-26649 Support meta replica LoadBalance mode for
RegionLocator#getAllRegionLocations() (#4442) (#4484)
5a13ad8137a is described below
commit 5a13ad8137a29cb3f80c11f0fd60c6855aa99820
Author: huaxiangsun
AuthorDate: Fri Jun 3 15:27:00 2022 -0700
HBASE-26649 Support meta replica LoadBalance mode for
RegionLocator#getAllRegionLocations() (#4442) (#4484)
Signed-off-by: Duo Zhang
---
.../hadoop/hbase/AsyncMetaTableAccessor.java | 34 ++-
.../hbase/{client => }/CatalogReplicaMode.java | 4 +--
.../org/apache/hadoop/hbase/MetaTableAccessor.java | 38 ++
.../hbase/client/AsyncNonMetaRegionLocator.java| 1 +
.../hbase/client/ConnectionImplementation.java | 1 +
.../apache/hadoop/hbase/client/HRegionLocator.java | 5 ++-
.../client/TestAsyncNonMetaRegionLocator.java | 1 +
.../TestMetaRegionReplicaReplicationEndpoint.java | 20
8 files changed, 88 insertions(+), 16 deletions(-)
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/AsyncMetaTableAccessor.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/AsyncMetaTableAccessor.java
index 367d4dad785..414d4ee7b49 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/AsyncMetaTableAccessor.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/AsyncMetaTableAccessor.java
@@ -17,6 +17,7 @@
*/
package org.apache.hadoop.hbase;
+import static
org.apache.hadoop.hbase.client.RegionLocator.LOCATOR_META_REPLICAS_MODE;
import static org.apache.hadoop.hbase.util.FutureUtils.addListener;
import java.io.IOException;
@@ -29,6 +30,7 @@ import java.util.NavigableMap;
import java.util.Optional;
import java.util.SortedMap;
import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ThreadLocalRandom;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import java.util.stream.Collectors;
@@ -295,7 +297,37 @@ public class AsyncMetaTableAccessor {
}
CompletableFuture future = new CompletableFuture();
-metaTable.scan(scan, new MetaTableScanResultConsumer(rowUpperLimit,
visitor, future));
+// Get the region locator's meta replica mode.
+CatalogReplicaMode metaReplicaMode =
CatalogReplicaMode.fromString(metaTable.getConfiguration()
+ .get(LOCATOR_META_REPLICAS_MODE, CatalogReplicaMode.NONE.toString()));
+
+if (metaReplicaMode == CatalogReplicaMode.LOAD_BALANCE) {
+ addListener(metaTable.getDescriptor(), (desc, error) -> {
+if (error != null) {
+ LOG.error("Failed to get meta table descriptor, error: ", error);
+ future.completeExceptionally(error);
+ return;
+}
+
+int numOfReplicas = desc.getRegionReplication();
+if (numOfReplicas > 1) {
+ int replicaId = ThreadLocalRandom.current().nextInt(numOfReplicas);
+
+ // When the replicaId is 0, do not set to Consistency.TIMELINE
+ if (replicaId > 0) {
+scan.setReplicaId(replicaId);
+scan.setConsistency(Consistency.TIMELINE);
+ }
+}
+metaTable.scan(scan, new MetaTableScanResultConsumer(rowUpperLimit,
visitor, future));
+ });
+} else {
+ if (metaReplicaMode == CatalogReplicaMode.HEDGED_READ) {
+scan.setConsistency(Consistency.TIMELINE);
+ }
+ metaTable.scan(scan, new MetaTableScanResultConsumer(rowUpperLimit,
visitor, future));
+}
+
return future;
}
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/CatalogReplicaMode.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/CatalogReplicaMode.java
similarity index 96%
rename from
hbase-client/src/main/java/org/apache/hadoop/hbase/client/CatalogReplicaMode.java
rename to
hbase-client/src/main/java/org/apache/hadoop/hbase/CatalogReplicaMode.java
index 647d5dcf38f..b89673d45a8 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/CatalogReplicaMode.java
+++ b/hbase-client/src/main/java/org/apache/hadoop/hbase/CatalogReplicaMode.java
@@ -15,7 +15,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.hadoop.hbase.client;
+package org.apache.hadoop.hbase;
import org.apache.yetus.audience.InterfaceAudience;
@@ -33,7 +33,7 @@ import org.apache.yetus.audience.InterfaceAudience;
*
*/
@InterfaceAudience.Private
-enum CatalogReplicaMode {
+public enum CatalogReplicaMode {
NONE {
@Override
public String toString() {
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTa
[hbase] branch branch-2 updated: Revert "HBASE-26962: Add mob info in web UI (#4359)"
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2 by this push:
new 58aaed19b04 Revert "HBASE-26962: Add mob info in web UI (#4359)"
58aaed19b04 is described below
commit 58aaed19b04abd17a35519d522c3e6086bd32902
Author: Huaxiang Sun
AuthorDate: Thu Jun 2 14:04:27 2022 -0700
Revert "HBASE-26962: Add mob info in web UI (#4359)"
This reverts commit 7a02cbfd7ee6418594a1cd241e10b54a0c837e23.
---
.../hbase-webapps/regionserver/region.jsp | 53 +-
1 file changed, 1 insertion(+), 52 deletions(-)
diff --git
a/hbase-server/src/main/resources/hbase-webapps/regionserver/region.jsp
b/hbase-server/src/main/resources/hbase-webapps/regionserver/region.jsp
index 72bc638202f..382783ea4ba 100644
--- a/hbase-server/src/main/resources/hbase-webapps/regionserver/region.jsp
+++ b/hbase-server/src/main/resources/hbase-webapps/regionserver/region.jsp
@@ -18,18 +18,11 @@
*/
--%>
<%@ page contentType="text/html;charset=UTF-8"
- import="java.net.URLEncoder"
import="java.util.Collection"
import="java.util.Date"
import="java.util.List"
- import="org.apache.hadoop.fs.FileStatus"
- import="org.apache.hadoop.fs.Path"
- import="org.apache.hadoop.hbase.HConstants"
import="org.apache.hadoop.hbase.client.RegionInfoDisplay"
- import="org.apache.hadoop.hbase.mob.MobUtils"
import="org.apache.hadoop.hbase.regionserver.HRegionServer"
- import="org.apache.hadoop.hbase.regionserver.HMobStore"
- import="org.apache.hadoop.hbase.regionserver.HStoreFile"
import="org.apache.hadoop.hbase.regionserver.Region"
import="org.apache.hadoop.hbase.regionserver.Store"
import="org.apache.hadoop.hbase.regionserver.StoreFile"
@@ -87,51 +80,7 @@
<%= storeFiles.size() %> StoreFile(s) in set.
-
- <% if (store instanceof HMobStore) { %>
- MOB Files
-
-
- MOB File
- Size (MB)
- Modification time
-
-
- <%
- int mobCnt = 0;
- for (StoreFile sf : storeFiles) {
- try {
- byte[] value =
((HStoreFile)sf).getMetadataValue(HStoreFile.MOB_FILE_REFS);
- if (value == null) {
- continue;
- }
-
- Collection fileNames =
MobUtils.deserializeMobFileRefs(value).build().values();
- mobCnt += fileNames.size();
- for (String fileName : fileNames) {
- Path mobPath = new Path(((HMobStore) store).getPath(),
fileName);
- FileStatus status = rs.getFileSystem().getFileStatus(mobPath);
- String mobPathStr = mobPath.toString();
- String encodedStr = URLEncoder.encode(mobPathStr,
HConstants.UTF8_ENCODING); %>
-
-
- <%=
mobPathStr%>
- <%= status.getLen() / 1024 / 1024 %>
- <%= new Date(status.getModificationTime()) %>
-
-
- <% }
- } catch (Exception e) { %>
-
- <%= e %>
-
- <% }
- } %>
-
- <%= mobCnt %> MobFile(s) in set.
-
- <% }
- }
+ <% }
}%>
[hbase] branch branch-2 updated: HBASE-26649 Support meta replica LoadBalance mode for RegionLocator#getAllRegionLocations() (#4442) (#4464)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2 by this push:
new 672b0a5f5cd HBASE-26649 Support meta replica LoadBalance mode for
RegionLocator#getAllRegionLocations() (#4442) (#4464)
672b0a5f5cd is described below
commit 672b0a5f5cdfc3ca4124b1209b5c28e3bf942c7e
Author: huaxiangsun
AuthorDate: Thu Jun 2 12:35:11 2022 -0700
HBASE-26649 Support meta replica LoadBalance mode for
RegionLocator#getAllRegionLocations() (#4442) (#4464)
Signed-off-by: Duo Zhang
---
.../hadoop/hbase/AsyncMetaTableAccessor.java | 34 ++-
.../hbase/{client => }/CatalogReplicaMode.java | 4 +--
.../org/apache/hadoop/hbase/MetaTableAccessor.java | 38 ++
.../hbase/client/AsyncNonMetaRegionLocator.java| 1 +
.../hbase/client/ConnectionImplementation.java | 1 +
.../apache/hadoop/hbase/client/HRegionLocator.java | 5 ++-
.../client/TestAsyncNonMetaRegionLocator.java | 1 +
.../TestMetaRegionReplicaReplicationEndpoint.java | 20
8 files changed, 88 insertions(+), 16 deletions(-)
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/AsyncMetaTableAccessor.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/AsyncMetaTableAccessor.java
index 367d4dad785..414d4ee7b49 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/AsyncMetaTableAccessor.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/AsyncMetaTableAccessor.java
@@ -17,6 +17,7 @@
*/
package org.apache.hadoop.hbase;
+import static
org.apache.hadoop.hbase.client.RegionLocator.LOCATOR_META_REPLICAS_MODE;
import static org.apache.hadoop.hbase.util.FutureUtils.addListener;
import java.io.IOException;
@@ -29,6 +30,7 @@ import java.util.NavigableMap;
import java.util.Optional;
import java.util.SortedMap;
import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ThreadLocalRandom;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import java.util.stream.Collectors;
@@ -295,7 +297,37 @@ public class AsyncMetaTableAccessor {
}
CompletableFuture future = new CompletableFuture();
-metaTable.scan(scan, new MetaTableScanResultConsumer(rowUpperLimit,
visitor, future));
+// Get the region locator's meta replica mode.
+CatalogReplicaMode metaReplicaMode =
CatalogReplicaMode.fromString(metaTable.getConfiguration()
+ .get(LOCATOR_META_REPLICAS_MODE, CatalogReplicaMode.NONE.toString()));
+
+if (metaReplicaMode == CatalogReplicaMode.LOAD_BALANCE) {
+ addListener(metaTable.getDescriptor(), (desc, error) -> {
+if (error != null) {
+ LOG.error("Failed to get meta table descriptor, error: ", error);
+ future.completeExceptionally(error);
+ return;
+}
+
+int numOfReplicas = desc.getRegionReplication();
+if (numOfReplicas > 1) {
+ int replicaId = ThreadLocalRandom.current().nextInt(numOfReplicas);
+
+ // When the replicaId is 0, do not set to Consistency.TIMELINE
+ if (replicaId > 0) {
+scan.setReplicaId(replicaId);
+scan.setConsistency(Consistency.TIMELINE);
+ }
+}
+metaTable.scan(scan, new MetaTableScanResultConsumer(rowUpperLimit,
visitor, future));
+ });
+} else {
+ if (metaReplicaMode == CatalogReplicaMode.HEDGED_READ) {
+scan.setConsistency(Consistency.TIMELINE);
+ }
+ metaTable.scan(scan, new MetaTableScanResultConsumer(rowUpperLimit,
visitor, future));
+}
+
return future;
}
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/CatalogReplicaMode.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/CatalogReplicaMode.java
similarity index 96%
rename from
hbase-client/src/main/java/org/apache/hadoop/hbase/client/CatalogReplicaMode.java
rename to
hbase-client/src/main/java/org/apache/hadoop/hbase/CatalogReplicaMode.java
index 647d5dcf38f..b89673d45a8 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/CatalogReplicaMode.java
+++ b/hbase-client/src/main/java/org/apache/hadoop/hbase/CatalogReplicaMode.java
@@ -15,7 +15,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.hadoop.hbase.client;
+package org.apache.hadoop.hbase;
import org.apache.yetus.audience.InterfaceAudience;
@@ -33,7 +33,7 @@ import org.apache.yetus.audience.InterfaceAudience;
*
*/
@InterfaceAudience.Private
-enum CatalogReplicaMode {
+public enum CatalogReplicaMode {
NONE {
@Override
public String toString() {
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.
[hbase] branch master updated: HBASE-26649 Support meta replica LoadBalance mode for RegionLocator#getAllRegionLocations() (#4442)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/master by this push:
new 623f8affe2c HBASE-26649 Support meta replica LoadBalance mode for
RegionLocator#getAllRegionLocations() (#4442)
623f8affe2c is described below
commit 623f8affe2c92208e1c63b3f7446a7fa6ce760b6
Author: huaxiangsun
AuthorDate: Mon May 23 08:54:00 2022 -0700
HBASE-26649 Support meta replica LoadBalance mode for
RegionLocator#getAllRegionLocations() (#4442)
Signed-off-by: Duo Zhang
---
.../hbase/{client => }/CatalogReplicaMode.java | 4 +--
.../hadoop/hbase/ClientMetaTableAccessor.java | 34 +-
.../hbase/client/AsyncNonMetaRegionLocator.java| 1 +
.../client/TestAsyncNonMetaRegionLocator.java | 1 +
.../TestMetaRegionReplicaReplication.java | 21 +
5 files changed, 52 insertions(+), 9 deletions(-)
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/CatalogReplicaMode.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/CatalogReplicaMode.java
similarity index 96%
rename from
hbase-client/src/main/java/org/apache/hadoop/hbase/client/CatalogReplicaMode.java
rename to
hbase-client/src/main/java/org/apache/hadoop/hbase/CatalogReplicaMode.java
index 647d5dcf38f..b89673d45a8 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/CatalogReplicaMode.java
+++ b/hbase-client/src/main/java/org/apache/hadoop/hbase/CatalogReplicaMode.java
@@ -15,7 +15,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.hadoop.hbase.client;
+package org.apache.hadoop.hbase;
import org.apache.yetus.audience.InterfaceAudience;
@@ -33,7 +33,7 @@ import org.apache.yetus.audience.InterfaceAudience;
*
*/
@InterfaceAudience.Private
-enum CatalogReplicaMode {
+public enum CatalogReplicaMode {
NONE {
@Override
public String toString() {
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/ClientMetaTableAccessor.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/ClientMetaTableAccessor.java
index 14076f0d7b0..e948048325f 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/ClientMetaTableAccessor.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/ClientMetaTableAccessor.java
@@ -17,6 +17,7 @@
*/
package org.apache.hadoop.hbase;
+import static
org.apache.hadoop.hbase.client.RegionLocator.LOCATOR_META_REPLICAS_MODE;
import static org.apache.hadoop.hbase.util.FutureUtils.addListener;
import java.io.Closeable;
@@ -26,6 +27,7 @@ import java.util.Collections;
import java.util.List;
import java.util.Optional;
import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ThreadLocalRandom;
import java.util.stream.Collectors;
import org.apache.hadoop.hbase.client.AdvancedScanResultConsumer;
import org.apache.hadoop.hbase.client.AsyncTable;
@@ -257,7 +259,37 @@ public final class ClientMetaTableAccessor {
}
CompletableFuture future = new CompletableFuture();
-metaTable.scan(scan, new MetaTableScanResultConsumer(rowUpperLimit,
visitor, future));
+// Get the region locator's meta replica mode.
+CatalogReplicaMode metaReplicaMode =
CatalogReplicaMode.fromString(metaTable.getConfiguration()
+ .get(LOCATOR_META_REPLICAS_MODE, CatalogReplicaMode.NONE.toString()));
+
+if (metaReplicaMode == CatalogReplicaMode.LOAD_BALANCE) {
+ addListener(metaTable.getDescriptor(), (desc, error) -> {
+if (error != null) {
+ LOG.error("Failed to get meta table descriptor, error: ", error);
+ future.completeExceptionally(error);
+ return;
+}
+
+int numOfReplicas = desc.getRegionReplication();
+if (numOfReplicas > 1) {
+ int replicaId = ThreadLocalRandom.current().nextInt(numOfReplicas);
+
+ // When the replicaId is 0, do not set to Consistency.TIMELINE
+ if (replicaId > 0) {
+scan.setReplicaId(replicaId);
+scan.setConsistency(Consistency.TIMELINE);
+ }
+}
+metaTable.scan(scan, new MetaTableScanResultConsumer(rowUpperLimit,
visitor, future));
+ });
+} else {
+ if (metaReplicaMode == CatalogReplicaMode.HEDGED_READ) {
+scan.setConsistency(Consistency.TIMELINE);
+ }
+ metaTable.scan(scan, new MetaTableScanResultConsumer(rowUpperLimit,
visitor, future));
+}
+
return future;
}
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncNonMetaRegionLocator.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncNonMetaRegionLocator.java
index 2e3b7f56691..2692a0b776b 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncNonMetaRegionLoc
[hbase] branch branch-2.5 updated: HBASE-26984 Chaos Monkey thread dies in ITBLL Chaos GracefulRollingRestartRsAction (#4383) (#4409)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch branch-2.5
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2.5 by this push:
new c337e391e2 HBASE-26984 Chaos Monkey thread dies in ITBLL Chaos
GracefulRollingRestartRsAction (#4383) (#4409)
c337e391e2 is described below
commit c337e391e25d28f2f667e465bdae78048158417d
Author: huaxiangsun
AuthorDate: Thu May 5 13:43:33 2022 -0500
HBASE-26984 Chaos Monkey thread dies in ITBLL Chaos
GracefulRollingRestartRsAction (#4383) (#4409)
There are two cases here:
1. Chaos Monkey thread died and there is no chaos after that.
2. Sometimes, regions are being moved back too quick that region server
has not finished its initliazation yet.
wait sometime to make sure that region server finishes its
initialization.
Signed-off-by: Wellington Chevreuil
---
.../hadoop/hbase/chaos/actions/GracefulRollingRestartRsAction.java | 2 ++
.../org/apache/hadoop/hbase/chaos/actions/RestartActionBaseAction.java | 2 ++
2 files changed, 4 insertions(+)
diff --git
a/hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/GracefulRollingRestartRsAction.java
b/hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/GracefulRollingRestartRsAction.java
index 236ed21290..6fe6332813 100644
---
a/hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/GracefulRollingRestartRsAction.java
+++
b/hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/GracefulRollingRestartRsAction.java
@@ -63,6 +63,8 @@ public class GracefulRollingRestartRsAction extends
RestartActionBaseAction {
rm.load();
} catch (Shell.ExitCodeException e) {
getLogger().info("Problem restarting but presume successful; code={}",
e.getExitCode(), e);
+ } catch (Exception e) {
+getLogger().info("Exception but continuing...", e);
}
sleep(rand.nextInt((int) sleepTime));
}
diff --git
a/hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/RestartActionBaseAction.java
b/hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/RestartActionBaseAction.java
index 31e9f578b9..3edb1fabdd 100644
---
a/hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/RestartActionBaseAction.java
+++
b/hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/RestartActionBaseAction.java
@@ -80,6 +80,8 @@ public abstract class RestartActionBaseAction extends Action {
sleep(sleepTime);
getLogger().info("Starting region server: {}", server);
startRs(server);
+// Sleep some time to make sure RS is online.
+sleep(sleepTime);
}
void restartZKNode(ServerName server, long sleepTime) throws IOException {
[hbase] branch branch-2 updated: HBASE-26984 Chaos Monkey thread dies in ITBLL Chaos GracefulRollingRestartRsAction (#4383) (#4408)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2 by this push:
new 8703cacba5 HBASE-26984 Chaos Monkey thread dies in ITBLL Chaos
GracefulRollingRestartRsAction (#4383) (#4408)
8703cacba5 is described below
commit 8703cacba594f83197b77d25cf33791388984aa5
Author: huaxiangsun
AuthorDate: Thu May 5 13:43:15 2022 -0500
HBASE-26984 Chaos Monkey thread dies in ITBLL Chaos
GracefulRollingRestartRsAction (#4383) (#4408)
There are two cases here:
1. Chaos Monkey thread died and there is no chaos after that.
2. Sometimes, regions are being moved back too quick that region server
has not finished its initliazation yet.
wait sometime to make sure that region server finishes its
initialization.
Signed-off-by: Wellington Chevreuil
---
.../hadoop/hbase/chaos/actions/GracefulRollingRestartRsAction.java | 2 ++
.../org/apache/hadoop/hbase/chaos/actions/RestartActionBaseAction.java | 2 ++
2 files changed, 4 insertions(+)
diff --git
a/hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/GracefulRollingRestartRsAction.java
b/hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/GracefulRollingRestartRsAction.java
index 236ed21290..6fe6332813 100644
---
a/hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/GracefulRollingRestartRsAction.java
+++
b/hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/GracefulRollingRestartRsAction.java
@@ -63,6 +63,8 @@ public class GracefulRollingRestartRsAction extends
RestartActionBaseAction {
rm.load();
} catch (Shell.ExitCodeException e) {
getLogger().info("Problem restarting but presume successful; code={}",
e.getExitCode(), e);
+ } catch (Exception e) {
+getLogger().info("Exception but continuing...", e);
}
sleep(rand.nextInt((int) sleepTime));
}
diff --git
a/hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/RestartActionBaseAction.java
b/hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/RestartActionBaseAction.java
index 31e9f578b9..3edb1fabdd 100644
---
a/hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/RestartActionBaseAction.java
+++
b/hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/RestartActionBaseAction.java
@@ -80,6 +80,8 @@ public abstract class RestartActionBaseAction extends Action {
sleep(sleepTime);
getLogger().info("Starting region server: {}", server);
startRs(server);
+// Sleep some time to make sure RS is online.
+sleep(sleepTime);
}
void restartZKNode(ServerName server, long sleepTime) throws IOException {
[hbase] branch master updated: HBASE-26984 Chaos Monkey thread dies in ITBLL Chaos GracefulRollingRestartRsAction (#4383)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/master by this push:
new 597098ebba HBASE-26984 Chaos Monkey thread dies in ITBLL Chaos
GracefulRollingRestartRsAction (#4383)
597098ebba is described below
commit 597098ebbafb0705662699f1cd573982d97aaca1
Author: huaxiangsun
AuthorDate: Thu May 5 09:37:41 2022 -0500
HBASE-26984 Chaos Monkey thread dies in ITBLL Chaos
GracefulRollingRestartRsAction (#4383)
There are two cases here:
1. Chaos Monkey thread died and there is no chaos after that.
2. Sometimes, regions are being moved back too quick that region server
has not finished its initliazation yet.
wait sometime to make sure that region server finishes its
initialization.
Signed-off-by: Wellington Chevreuil
---
.../hadoop/hbase/chaos/actions/GracefulRollingRestartRsAction.java | 2 ++
.../org/apache/hadoop/hbase/chaos/actions/RestartActionBaseAction.java | 2 ++
2 files changed, 4 insertions(+)
diff --git
a/hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/GracefulRollingRestartRsAction.java
b/hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/GracefulRollingRestartRsAction.java
index 236ed21290..6fe6332813 100644
---
a/hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/GracefulRollingRestartRsAction.java
+++
b/hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/GracefulRollingRestartRsAction.java
@@ -63,6 +63,8 @@ public class GracefulRollingRestartRsAction extends
RestartActionBaseAction {
rm.load();
} catch (Shell.ExitCodeException e) {
getLogger().info("Problem restarting but presume successful; code={}",
e.getExitCode(), e);
+ } catch (Exception e) {
+getLogger().info("Exception but continuing...", e);
}
sleep(rand.nextInt((int) sleepTime));
}
diff --git
a/hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/RestartActionBaseAction.java
b/hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/RestartActionBaseAction.java
index 31e9f578b9..3edb1fabdd 100644
---
a/hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/RestartActionBaseAction.java
+++
b/hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/RestartActionBaseAction.java
@@ -80,6 +80,8 @@ public abstract class RestartActionBaseAction extends Action {
sleep(sleepTime);
getLogger().info("Starting region server: {}", server);
startRs(server);
+// Sleep some time to make sure RS is online.
+sleep(sleepTime);
}
void restartZKNode(ServerName server, long sleepTime) throws IOException {
[hbase] branch branch-2.5 updated: HBASE-26618 Involving primary meta region in meta scan with CatalogRe… (#4321) (#4327)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch branch-2.5
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2.5 by this push:
new 43af814d1ac HBASE-26618 Involving primary meta region in meta scan
with CatalogRe… (#4321) (#4327)
43af814d1ac is described below
commit 43af814d1acd2ef9e0c2c4ba2e9292dbd22fa2ea
Author: huaxiangsun
AuthorDate: Mon Apr 11 10:28:55 2022 -0700
HBASE-26618 Involving primary meta region in meta scan with CatalogRe…
(#4321) (#4327)
Signed-off-by: Michael Stack
---
.../client/CatalogReplicaLoadBalanceSimpleSelector.java | 14 +++---
.../TestCatalogReplicaLoadBalanceSimpleSelector.java | 11 ---
.../TestMetaRegionReplicaReplicationEndpoint.java | 15 ---
3 files changed, 27 insertions(+), 13 deletions(-)
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/CatalogReplicaLoadBalanceSimpleSelector.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/CatalogReplicaLoadBalanceSimpleSelector.java
index bbafc650b29..c03d8310994 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/CatalogReplicaLoadBalanceSimpleSelector.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/CatalogReplicaLoadBalanceSimpleSelector.java
@@ -46,15 +46,15 @@ import
org.apache.hbase.thirdparty.com.google.common.base.Preconditions;
* balancing algorithm. It maintains a stale location cache for each table.
Whenever client looks
* up location, it first check if the row is the stale location cache. If yes,
the location from
* catalog replica is stale, it will go to the primary region to look up
update-to-date location;
- * otherwise, it will randomly pick up a replica region for lookup. When
clients receive
- * RegionNotServedException from region servers, it will add these region
locations to the stale
- * location cache. The stale cache will be cleaned up periodically by a
chore.
+ * otherwise, it will randomly pick up a replica region or primary region for
lookup. When clients
+ * receive RegionNotServedException from region servers, it will add these
region locations to the
+ * stale location cache. The stale cache will be cleaned up periodically by a
chore.
*
- * It follows a simple algorithm to choose a replica to go:
+ * It follows a simple algorithm to choose a meta replica region (including
primary meta) to go:
*
*
* If there is no stale location entry for rows it looks up, it will
randomly
- * pick a replica region to do lookup.
+ * pick a meta replica region (including primary meta) to do lookup.
* If the location from the replica region is stale, client gets
RegionNotServedException
* from region server, in this case, it will create
StaleLocationCacheEntry in
* CatalogReplicaLoadBalanceReplicaSimpleSelector.
@@ -141,7 +141,7 @@ class CatalogReplicaLoadBalanceSimpleSelector implements
}
/**
- * Select an random replica id. In case there is no replica region
configured, return
+ * Select an random replica id (including the primary replica id). In case
there is no replica region configured, return
* the primary replica id.
* @return Replica id
*/
@@ -155,7 +155,7 @@ class CatalogReplicaLoadBalanceSimpleSelector implements
if (cachedNumOfReplicas <= 1) {
return RegionInfo.DEFAULT_REPLICA_ID;
}
-return 1 + ThreadLocalRandom.current().nextInt(cachedNumOfReplicas - 1);
+return ThreadLocalRandom.current().nextInt(cachedNumOfReplicas);
}
/**
diff --git
a/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestCatalogReplicaLoadBalanceSimpleSelector.java
b/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestCatalogReplicaLoadBalanceSimpleSelector.java
index 17b6d74af8c..eaaac803489 100644
---
a/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestCatalogReplicaLoadBalanceSimpleSelector.java
+++
b/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestCatalogReplicaLoadBalanceSimpleSelector.java
@@ -24,6 +24,7 @@ import static org.junit.Assert.assertNotEquals;
import java.io.IOException;
import java.util.concurrent.TimeUnit;
+import java.util.stream.IntStream;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseClassTestRule;
import org.apache.hadoop.hbase.HBaseTestingUtility;
@@ -105,9 +106,13 @@ public class TestCatalogReplicaLoadBalanceSimpleSelector {
return numOfReplicas;
});
-assertNotEquals(
- metaSelector.select(TableName.valueOf("test"), EMPTY_START_ROW,
RegionLocateType.CURRENT),
- RegionReplicaUtil.DEFAULT_REPLICA_ID);
+// Loop for 100 times, it should cover all replica ids.
+int[] replicaIdCount = new int[numOfMetaReplica];
+IntStream.range(1, 100).forEach(i -> replicaIdCount[met
[hbase] branch branch-2 updated: HBASE-26618 Involving primary meta region in meta scan with CatalogRe… (#4321) (#4326)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2 by this push:
new f9abe3d8f1b HBASE-26618 Involving primary meta region in meta scan
with CatalogRe… (#4321) (#4326)
f9abe3d8f1b is described below
commit f9abe3d8f1b7c5c1a374499717f28e6e3734523f
Author: huaxiangsun
AuthorDate: Mon Apr 11 10:27:15 2022 -0700
HBASE-26618 Involving primary meta region in meta scan with CatalogRe…
(#4321) (#4326)
Signed-off-by: Michael Stack
---
.../client/CatalogReplicaLoadBalanceSimpleSelector.java | 14 +++---
.../TestCatalogReplicaLoadBalanceSimpleSelector.java | 11 ---
.../TestMetaRegionReplicaReplicationEndpoint.java | 15 ---
3 files changed, 27 insertions(+), 13 deletions(-)
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/CatalogReplicaLoadBalanceSimpleSelector.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/CatalogReplicaLoadBalanceSimpleSelector.java
index bbafc650b29..c03d8310994 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/CatalogReplicaLoadBalanceSimpleSelector.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/CatalogReplicaLoadBalanceSimpleSelector.java
@@ -46,15 +46,15 @@ import
org.apache.hbase.thirdparty.com.google.common.base.Preconditions;
* balancing algorithm. It maintains a stale location cache for each table.
Whenever client looks
* up location, it first check if the row is the stale location cache. If yes,
the location from
* catalog replica is stale, it will go to the primary region to look up
update-to-date location;
- * otherwise, it will randomly pick up a replica region for lookup. When
clients receive
- * RegionNotServedException from region servers, it will add these region
locations to the stale
- * location cache. The stale cache will be cleaned up periodically by a
chore.
+ * otherwise, it will randomly pick up a replica region or primary region for
lookup. When clients
+ * receive RegionNotServedException from region servers, it will add these
region locations to the
+ * stale location cache. The stale cache will be cleaned up periodically by a
chore.
*
- * It follows a simple algorithm to choose a replica to go:
+ * It follows a simple algorithm to choose a meta replica region (including
primary meta) to go:
*
*
* If there is no stale location entry for rows it looks up, it will
randomly
- * pick a replica region to do lookup.
+ * pick a meta replica region (including primary meta) to do lookup.
* If the location from the replica region is stale, client gets
RegionNotServedException
* from region server, in this case, it will create
StaleLocationCacheEntry in
* CatalogReplicaLoadBalanceReplicaSimpleSelector.
@@ -141,7 +141,7 @@ class CatalogReplicaLoadBalanceSimpleSelector implements
}
/**
- * Select an random replica id. In case there is no replica region
configured, return
+ * Select an random replica id (including the primary replica id). In case
there is no replica region configured, return
* the primary replica id.
* @return Replica id
*/
@@ -155,7 +155,7 @@ class CatalogReplicaLoadBalanceSimpleSelector implements
if (cachedNumOfReplicas <= 1) {
return RegionInfo.DEFAULT_REPLICA_ID;
}
-return 1 + ThreadLocalRandom.current().nextInt(cachedNumOfReplicas - 1);
+return ThreadLocalRandom.current().nextInt(cachedNumOfReplicas);
}
/**
diff --git
a/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestCatalogReplicaLoadBalanceSimpleSelector.java
b/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestCatalogReplicaLoadBalanceSimpleSelector.java
index 17b6d74af8c..eaaac803489 100644
---
a/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestCatalogReplicaLoadBalanceSimpleSelector.java
+++
b/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestCatalogReplicaLoadBalanceSimpleSelector.java
@@ -24,6 +24,7 @@ import static org.junit.Assert.assertNotEquals;
import java.io.IOException;
import java.util.concurrent.TimeUnit;
+import java.util.stream.IntStream;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseClassTestRule;
import org.apache.hadoop.hbase.HBaseTestingUtility;
@@ -105,9 +106,13 @@ public class TestCatalogReplicaLoadBalanceSimpleSelector {
return numOfReplicas;
});
-assertNotEquals(
- metaSelector.select(TableName.valueOf("test"), EMPTY_START_ROW,
RegionLocateType.CURRENT),
- RegionReplicaUtil.DEFAULT_REPLICA_ID);
+// Loop for 100 times, it should cover all replica ids.
+int[] replicaIdCount = new int[numOfMetaReplica];
+IntStream.range(1, 100).forEach(i -> replicaIdCount[metaSelector.select(
+ T
[hbase] branch branch-2.4 updated: HBASE-26618 Involving primary meta region in meta scan with CatalogRe… (#4321) (#4328)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2.4 by this push:
new 7ac7954372e HBASE-26618 Involving primary meta region in meta scan
with CatalogRe… (#4321) (#4328)
7ac7954372e is described below
commit 7ac7954372ea369ad9a3215b0f5ea1f40f23185c
Author: huaxiangsun
AuthorDate: Mon Apr 11 10:26:42 2022 -0700
HBASE-26618 Involving primary meta region in meta scan with CatalogRe…
(#4321) (#4328)
Signed-off-by: Michael Stack
---
.../client/CatalogReplicaLoadBalanceSimpleSelector.java | 14 +++---
.../TestCatalogReplicaLoadBalanceSimpleSelector.java | 11 ---
.../TestMetaRegionReplicaReplicationEndpoint.java | 15 ---
3 files changed, 27 insertions(+), 13 deletions(-)
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/CatalogReplicaLoadBalanceSimpleSelector.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/CatalogReplicaLoadBalanceSimpleSelector.java
index bbafc650b29..c03d8310994 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/CatalogReplicaLoadBalanceSimpleSelector.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/CatalogReplicaLoadBalanceSimpleSelector.java
@@ -46,15 +46,15 @@ import
org.apache.hbase.thirdparty.com.google.common.base.Preconditions;
* balancing algorithm. It maintains a stale location cache for each table.
Whenever client looks
* up location, it first check if the row is the stale location cache. If yes,
the location from
* catalog replica is stale, it will go to the primary region to look up
update-to-date location;
- * otherwise, it will randomly pick up a replica region for lookup. When
clients receive
- * RegionNotServedException from region servers, it will add these region
locations to the stale
- * location cache. The stale cache will be cleaned up periodically by a
chore.
+ * otherwise, it will randomly pick up a replica region or primary region for
lookup. When clients
+ * receive RegionNotServedException from region servers, it will add these
region locations to the
+ * stale location cache. The stale cache will be cleaned up periodically by a
chore.
*
- * It follows a simple algorithm to choose a replica to go:
+ * It follows a simple algorithm to choose a meta replica region (including
primary meta) to go:
*
*
* If there is no stale location entry for rows it looks up, it will
randomly
- * pick a replica region to do lookup.
+ * pick a meta replica region (including primary meta) to do lookup.
* If the location from the replica region is stale, client gets
RegionNotServedException
* from region server, in this case, it will create
StaleLocationCacheEntry in
* CatalogReplicaLoadBalanceReplicaSimpleSelector.
@@ -141,7 +141,7 @@ class CatalogReplicaLoadBalanceSimpleSelector implements
}
/**
- * Select an random replica id. In case there is no replica region
configured, return
+ * Select an random replica id (including the primary replica id). In case
there is no replica region configured, return
* the primary replica id.
* @return Replica id
*/
@@ -155,7 +155,7 @@ class CatalogReplicaLoadBalanceSimpleSelector implements
if (cachedNumOfReplicas <= 1) {
return RegionInfo.DEFAULT_REPLICA_ID;
}
-return 1 + ThreadLocalRandom.current().nextInt(cachedNumOfReplicas - 1);
+return ThreadLocalRandom.current().nextInt(cachedNumOfReplicas);
}
/**
diff --git
a/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestCatalogReplicaLoadBalanceSimpleSelector.java
b/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestCatalogReplicaLoadBalanceSimpleSelector.java
index 17b6d74af8c..eaaac803489 100644
---
a/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestCatalogReplicaLoadBalanceSimpleSelector.java
+++
b/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestCatalogReplicaLoadBalanceSimpleSelector.java
@@ -24,6 +24,7 @@ import static org.junit.Assert.assertNotEquals;
import java.io.IOException;
import java.util.concurrent.TimeUnit;
+import java.util.stream.IntStream;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseClassTestRule;
import org.apache.hadoop.hbase.HBaseTestingUtility;
@@ -105,9 +106,13 @@ public class TestCatalogReplicaLoadBalanceSimpleSelector {
return numOfReplicas;
});
-assertNotEquals(
- metaSelector.select(TableName.valueOf("test"), EMPTY_START_ROW,
RegionLocateType.CURRENT),
- RegionReplicaUtil.DEFAULT_REPLICA_ID);
+// Loop for 100 times, it should cover all replica ids.
+int[] replicaIdCount = new int[numOfMetaReplica];
+IntStream.range(1, 100).forEach(i -> replicaIdCount[met
[hbase] branch master updated (e68c61dd54c -> 0b4ff986143)
This is an automated email from the ASF dual-hosted git repository. huaxiangsun pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hbase.git from e68c61dd54c HBASE-26928 Fix several indentation problems (#4323) add 0b4ff986143 HBASE-26618 Involving primary meta region in meta scan with CatalogRe… (#4321) No new revisions were added by this update. Summary of changes: .../client/CatalogReplicaLoadBalanceSimpleSelector.java | 14 +++--- .../TestCatalogReplicaLoadBalanceSimpleSelector.java | 11 --- .../regionserver/TestMetaRegionReplicaReplication.java| 15 --- 3 files changed, 27 insertions(+), 13 deletions(-)
[hbase] branch master updated: Revert "HBASE-26649 Support meta replica LoadBalance mode for RegionLocator#getAllRegionLocations() (#4044)"
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/master by this push:
new 351caa5 Revert "HBASE-26649 Support meta replica LoadBalance mode for
RegionLocator#getAllRegionLocations() (#4044)"
351caa5 is described below
commit 351caa5bcca062afa9e02064af480b8bf4225d90
Author: Huaxiang Sun
AuthorDate: Mon Jan 31 09:48:46 2022 -0800
Revert "HBASE-26649 Support meta replica LoadBalance mode for
RegionLocator#getAllRegionLocations() (#4044)"
This reverts commit 8c607397c19a828a1be89eb3a1325f774f772c2a.
---
.../hadoop/hbase/ClientMetaTableAccessor.java | 38 ++
.../hbase/client/AsyncNonMetaRegionLocator.java| 1 -
.../hbase/{ => client}/CatalogReplicaMode.java | 4 +--
.../client/TestAsyncNonMetaRegionLocator.java | 1 -
.../TestMetaRegionReplicaReplication.java | 20 +---
5 files changed, 6 insertions(+), 58 deletions(-)
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/ClientMetaTableAccessor.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/ClientMetaTableAccessor.java
index fd85ede..ecc6573 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/ClientMetaTableAccessor.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/ClientMetaTableAccessor.java
@@ -17,7 +17,6 @@
*/
package org.apache.hadoop.hbase;
-import static
org.apache.hadoop.hbase.client.RegionLocator.LOCATOR_META_REPLICAS_MODE;
import static org.apache.hadoop.hbase.util.FutureUtils.addListener;
import java.io.Closeable;
@@ -27,7 +26,6 @@ import java.util.Collections;
import java.util.List;
import java.util.Optional;
import java.util.concurrent.CompletableFuture;
-import java.util.concurrent.ThreadLocalRandom;
import java.util.stream.Collectors;
import org.apache.hadoop.hbase.client.AdvancedScanResultConsumer;
import org.apache.hadoop.hbase.client.AsyncTable;
@@ -418,40 +416,10 @@ public final class ClientMetaTableAccessor {
Scan scan = new Scan();
int scannerCaching =
metaTable.getConfiguration().getInt(HConstants.HBASE_META_SCANNER_CACHING,
HConstants.DEFAULT_HBASE_META_SCANNER_CACHING);
-// Get the region locator's meta replica mode.
-CatalogReplicaMode metaReplicaMode =
CatalogReplicaMode.fromString(metaTable.getConfiguration()
- .get(LOCATOR_META_REPLICAS_MODE, CatalogReplicaMode.NONE.toString()));
-
-switch (metaReplicaMode) {
- case LOAD_BALANCE:
-int numOfReplicas = 1;
-try {
- numOfReplicas =
metaTable.getDescriptor().get().getRegionReplication();
-} catch (Exception e) {
- LOG.warn("Failed to get region replication for meta table");
-}
-if (numOfReplicas > 1) {
- int replicaId = ThreadLocalRandom.current().nextInt(numOfReplicas);
-
- // When the replicaId is 0, do not set to Consistency.TIMELINE
- if (replicaId > 0) {
-scan.setReplicaId(replicaId);
-scan.setConsistency(Consistency.TIMELINE);
- }
-}
-break;
- case NONE:
-// If user does not configure LOCATOR_META_REPLICAS_MODE, let's check
the legacy config.
-if
(metaTable.getConfiguration().getBoolean(HConstants.USE_META_REPLICAS,
- HConstants.DEFAULT_USE_META_REPLICAS)) {
- scan.setConsistency(Consistency.TIMELINE);
-}
-break;
-
- default:
-// Do nothing
+if (metaTable.getConfiguration().getBoolean(HConstants.USE_META_REPLICAS,
+ HConstants.DEFAULT_USE_META_REPLICAS)) {
+ scan.setConsistency(Consistency.TIMELINE);
}
-
if (rowUpperLimit <= scannerCaching) {
scan.setLimit(rowUpperLimit);
}
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncNonMetaRegionLocator.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncNonMetaRegionLocator.java
index ea1a1ee..1c686ac 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncNonMetaRegionLocator.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncNonMetaRegionLocator.java
@@ -50,7 +50,6 @@ import java.util.concurrent.ConcurrentSkipListMap;
import java.util.concurrent.TimeUnit;
import org.apache.commons.lang3.ObjectUtils;
import org.apache.hadoop.hbase.CatalogFamilyFormat;
-import org.apache.hadoop.hbase.CatalogReplicaMode;
import org.apache.hadoop.hbase.HBaseIOException;
import org.apache.hadoop.hbase.HConstants;
import org.apache.hadoop.hbase.HRegionLocation;
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/CatalogReplicaMode.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/CatalogReplicaMode.java
similarity index 97%
rename from
hbase-client/src/main/java/org/apache/hadoop/hbase/CatalogRe
[hbase] branch master updated: HBASE-26649 Support meta replica LoadBalance mode for RegionLocator#getAllRegionLocations() (#4044)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/master by this push:
new 8c60739 HBASE-26649 Support meta replica LoadBalance mode for
RegionLocator#getAllRegionLocations() (#4044)
8c60739 is described below
commit 8c607397c19a828a1be89eb3a1325f774f772c2a
Author: huaxiangsun
AuthorDate: Fri Jan 28 10:18:48 2022 -0800
HBASE-26649 Support meta replica LoadBalance mode for
RegionLocator#getAllRegionLocations() (#4044)
Signed-off-by: Duo Zhang
---
.../hbase/{client => }/CatalogReplicaMode.java | 4 +--
.../hadoop/hbase/ClientMetaTableAccessor.java | 38 --
.../hbase/client/AsyncNonMetaRegionLocator.java| 1 +
.../client/TestAsyncNonMetaRegionLocator.java | 1 +
.../TestMetaRegionReplicaReplication.java | 20 +++-
5 files changed, 58 insertions(+), 6 deletions(-)
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/CatalogReplicaMode.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/CatalogReplicaMode.java
similarity index 97%
rename from
hbase-client/src/main/java/org/apache/hadoop/hbase/client/CatalogReplicaMode.java
rename to
hbase-client/src/main/java/org/apache/hadoop/hbase/CatalogReplicaMode.java
index 40062e3..a86d2fb 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/CatalogReplicaMode.java
+++ b/hbase-client/src/main/java/org/apache/hadoop/hbase/CatalogReplicaMode.java
@@ -13,7 +13,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.hadoop.hbase.client;
+package org.apache.hadoop.hbase;
import org.apache.yetus.audience.InterfaceAudience;
@@ -33,7 +33,7 @@ import org.apache.yetus.audience.InterfaceAudience;
*
*/
@InterfaceAudience.Private
-enum CatalogReplicaMode {
+public enum CatalogReplicaMode {
NONE {
@Override
public String toString() {
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/ClientMetaTableAccessor.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/ClientMetaTableAccessor.java
index ecc6573..fd85ede 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/ClientMetaTableAccessor.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/ClientMetaTableAccessor.java
@@ -17,6 +17,7 @@
*/
package org.apache.hadoop.hbase;
+import static
org.apache.hadoop.hbase.client.RegionLocator.LOCATOR_META_REPLICAS_MODE;
import static org.apache.hadoop.hbase.util.FutureUtils.addListener;
import java.io.Closeable;
@@ -26,6 +27,7 @@ import java.util.Collections;
import java.util.List;
import java.util.Optional;
import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ThreadLocalRandom;
import java.util.stream.Collectors;
import org.apache.hadoop.hbase.client.AdvancedScanResultConsumer;
import org.apache.hadoop.hbase.client.AsyncTable;
@@ -416,10 +418,40 @@ public final class ClientMetaTableAccessor {
Scan scan = new Scan();
int scannerCaching =
metaTable.getConfiguration().getInt(HConstants.HBASE_META_SCANNER_CACHING,
HConstants.DEFAULT_HBASE_META_SCANNER_CACHING);
-if (metaTable.getConfiguration().getBoolean(HConstants.USE_META_REPLICAS,
- HConstants.DEFAULT_USE_META_REPLICAS)) {
- scan.setConsistency(Consistency.TIMELINE);
+// Get the region locator's meta replica mode.
+CatalogReplicaMode metaReplicaMode =
CatalogReplicaMode.fromString(metaTable.getConfiguration()
+ .get(LOCATOR_META_REPLICAS_MODE, CatalogReplicaMode.NONE.toString()));
+
+switch (metaReplicaMode) {
+ case LOAD_BALANCE:
+int numOfReplicas = 1;
+try {
+ numOfReplicas =
metaTable.getDescriptor().get().getRegionReplication();
+} catch (Exception e) {
+ LOG.warn("Failed to get region replication for meta table");
+}
+if (numOfReplicas > 1) {
+ int replicaId = ThreadLocalRandom.current().nextInt(numOfReplicas);
+
+ // When the replicaId is 0, do not set to Consistency.TIMELINE
+ if (replicaId > 0) {
+scan.setReplicaId(replicaId);
+scan.setConsistency(Consistency.TIMELINE);
+ }
+}
+break;
+ case NONE:
+// If user does not configure LOCATOR_META_REPLICAS_MODE, let's check
the legacy config.
+if
(metaTable.getConfiguration().getBoolean(HConstants.USE_META_REPLICAS,
+ HConstants.DEFAULT_USE_META_REPLICAS)) {
+ scan.setConsistency(Consistency.TIMELINE);
+}
+break;
+
+ default:
+// Do nothing
}
+
if (rowUpperLimit <= scannerCaching) {
scan.setLimit(rowUpperLimit);
}
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncNonMetaRegionLocator.jav
[hbase] branch branch-2.3 updated: HBASE-26590 Hbase-client Meta lookup performance regression between hbase-1 and hbase-2 (#4008) (#4013)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch branch-2.3
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2.3 by this push:
new 7fcbf62 HBASE-26590 Hbase-client Meta lookup performance regression
between hbase-1 and hbase-2 (#4008) (#4013)
7fcbf62 is described below
commit 7fcbf62403b54d5efc64803031baafbcfcf9558a
Author: huaxiangsun
AuthorDate: Thu Jan 6 11:56:07 2022 -0800
HBASE-26590 Hbase-client Meta lookup performance regression between hbase-1
and hbase-2 (#4008) (#4013)
Change scan caching back to 1 to avoid extra work at Region Server side.
For most cases, the extra 4 results
fetched are wasted at the client side.
Signed-off-by: Duo Zhang
---
.../java/org/apache/hadoop/hbase/client/ConnectionImplementation.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
index 2b28642..e980614 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
@@ -849,7 +849,7 @@ class ConnectionImplementation implements
ClusterConnection, Closeable {
byte[] metaStopKey =
RegionInfo.createRegionName(tableName, HConstants.EMPTY_START_ROW, "",
false);
Scan s = new Scan().withStartRow(metaStartKey).withStopRow(metaStopKey,
true)
- .addFamily(HConstants.CATALOG_FAMILY).setReversed(true).setCaching(5)
+ .addFamily(HConstants.CATALOG_FAMILY).setReversed(true).setCaching(1)
.setReadType(ReadType.PREAD);
if (this.useMetaReplicas) {
s.setConsistency(Consistency.TIMELINE);
[hbase] branch branch-2.4 updated: HBASE-26590 Hbase-client Meta lookup performance regression between hbase-1 and hbase-2 (#4008) (#4012)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2.4 by this push:
new f3639bc HBASE-26590 Hbase-client Meta lookup performance regression
between hbase-1 and hbase-2 (#4008) (#4012)
f3639bc is described below
commit f3639bc80c7aac9cbcf31f6566df9b929613f018
Author: huaxiangsun
AuthorDate: Thu Jan 6 11:55:51 2022 -0800
HBASE-26590 Hbase-client Meta lookup performance regression between hbase-1
and hbase-2 (#4008) (#4012)
Change scan caching back to 1 to avoid extra work at Region Server side.
For most cases, the extra 4 results
fetched are wasted at the client side.
Signed-off-by: Duo Zhang
---
.../java/org/apache/hadoop/hbase/client/ConnectionImplementation.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
index cfbc19c..145dc91 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
@@ -888,7 +888,7 @@ class ConnectionImplementation implements
ClusterConnection, Closeable {
byte[] metaStopKey =
RegionInfo.createRegionName(tableName, HConstants.EMPTY_START_ROW, "",
false);
Scan s = new Scan().withStartRow(metaStartKey).withStopRow(metaStopKey,
true)
- .addFamily(HConstants.CATALOG_FAMILY).setReversed(true).setCaching(5)
+ .addFamily(HConstants.CATALOG_FAMILY).setReversed(true).setCaching(1)
.setReadType(ReadType.PREAD);
switch (this.metaReplicaMode) {
[hbase] branch branch-2.5 updated: HBASE-26590 Hbase-client Meta lookup performance regression between hbase-1 and hbase-2 (#4008) (#4011)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch branch-2.5
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2.5 by this push:
new a6b0e2d HBASE-26590 Hbase-client Meta lookup performance regression
between hbase-1 and hbase-2 (#4008) (#4011)
a6b0e2d is described below
commit a6b0e2dc85f9cd754494a5158dfde975972515b0
Author: huaxiangsun
AuthorDate: Thu Jan 6 11:54:53 2022 -0800
HBASE-26590 Hbase-client Meta lookup performance regression between hbase-1
and hbase-2 (#4008) (#4011)
Change scan caching back to 1 to avoid extra work at Region Server side.
For most cases, the extra 4 results
fetched are wasted at the client side.
Signed-off-by: Duo Zhang
---
.../java/org/apache/hadoop/hbase/client/ConnectionImplementation.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
index 46863af..5fec87e 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
@@ -912,7 +912,7 @@ class ConnectionImplementation implements
ClusterConnection, Closeable {
byte[] metaStopKey =
RegionInfo.createRegionName(tableName, HConstants.EMPTY_START_ROW, "",
false);
Scan s = new Scan().withStartRow(metaStartKey).withStopRow(metaStopKey,
true)
- .addFamily(HConstants.CATALOG_FAMILY).setReversed(true).setCaching(5)
+ .addFamily(HConstants.CATALOG_FAMILY).setReversed(true).setCaching(1)
.setReadType(ReadType.PREAD);
switch (this.metaReplicaMode) {
[hbase] branch branch-2 updated: HBASE-26590 Hbase-client Meta lookup performance regression between hbase-1 and hbase-2 (#4008)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2 by this push:
new c897ce9 HBASE-26590 Hbase-client Meta lookup performance regression
between hbase-1 and hbase-2 (#4008)
c897ce9 is described below
commit c897ce904a74084f8b02b1f6e8ea177a9694e329
Author: huaxiangsun
AuthorDate: Thu Jan 6 10:31:21 2022 -0800
HBASE-26590 Hbase-client Meta lookup performance regression between hbase-1
and hbase-2 (#4008)
Change scan caching back to 1 to avoid extra work at Region Server side.
For most cases, the extra 4 results
fetched are wasted at the client side.
Signed-off-by: Duo Zhang
---
.../java/org/apache/hadoop/hbase/client/ConnectionImplementation.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
index 46863af..5fec87e 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
@@ -912,7 +912,7 @@ class ConnectionImplementation implements
ClusterConnection, Closeable {
byte[] metaStopKey =
RegionInfo.createRegionName(tableName, HConstants.EMPTY_START_ROW, "",
false);
Scan s = new Scan().withStartRow(metaStartKey).withStopRow(metaStopKey,
true)
- .addFamily(HConstants.CATALOG_FAMILY).setReversed(true).setCaching(5)
+ .addFamily(HConstants.CATALOG_FAMILY).setReversed(true).setCaching(1)
.setReadType(ReadType.PREAD);
switch (this.metaReplicaMode) {
[hbase] branch branch-2.4 updated: HBASE-26311 Balancer gets stuck in cohosted replica distribution (#3805)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2.4 by this push:
new 8ea5484 HBASE-26311 Balancer gets stuck in cohosted replica
distribution (#3805)
8ea5484 is described below
commit 8ea548441c347590baab25538a88126e9523a44a
Author: clarax
AuthorDate: Wed Nov 3 12:52:51 2021 -0700
HBASE-26311 Balancer gets stuck in cohosted replica distribution (#3805)
Signed-off-by: Huaxiang Sun
---
.../hbase/master/balancer/BaseLoadBalancer.java| 59 --
.../hbase/master/balancer/DoubleArrayCost.java | 23 ++---
.../master/balancer/StochasticLoadBalancer.java| 39 ++
.../hbase/master/balancer/BalancerTestBase.java| 1 -
.../master/balancer/TestBaseLoadBalancer.java | 2 -
.../hbase/master/balancer/TestDoubleArrayCost.java | 4 +-
.../balancer/TestStochasticBalancerJmxMetrics.java | 1 -
.../balancer/TestStochasticLoadBalancer.java | 6 +--
.../TestStochasticLoadBalancerBalanceCluster.java | 1 -
...ochasticLoadBalancerRegionReplicaSameHosts.java | 1 -
...ochasticLoadBalancerRegionReplicaWithRacks.java | 1 -
11 files changed, 61 insertions(+), 77 deletions(-)
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
index 03713c9..e8c6b1a 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
@@ -167,12 +167,8 @@ public abstract class BaseLoadBalancer implements
LoadBalancer {
int[] regionIndexToServerIndex;//regionIndex -> serverIndex
int[] initialRegionIndexToServerIndex;//regionIndex -> serverIndex
(initial cluster state)
int[] regionIndexToTableIndex; //regionIndex -> tableIndex
-int[][] numRegionsPerServerPerTable; // serverIndex -> tableIndex -> #
regions
+int[][] numRegionsPerServerPerTable; // tableIndex -> serverIndex -> #
regions
int[] numRegionsPerTable; // tableIndex -> region count
-double[] meanRegionsPerTable; // mean region count per table
-double[] regionSkewByTable; // skew on RS per by table
-double[] minRegionSkewByTable; // min skew on RS per by table
-double[] maxRegionSkewByTable; // max skew on RS per by table
int[] regionIndexToPrimaryIndex; //regionIndex -> regionIndex of the
primary
boolean hasRegionReplicas = false; //whether there is regions with
replicas
@@ -382,42 +378,24 @@ public abstract class BaseLoadBalancer implements
LoadBalancer {
}
numTables = tables.size();
- LOG.debug("Number of tables={}", numTables);
- numRegionsPerServerPerTable = new int[numServers][numTables];
+ LOG.debug("Number of tables={}, number of hosts={}, number of racks={}",
numTables,
+numHosts, numRacks);
+ numRegionsPerServerPerTable = new int[numTables][numServers];
numRegionsPerTable = new int[numTables];
- for (int i = 0; i < numServers; i++) {
-for (int j = 0; j < numTables; j++) {
+ for (int i = 0; i < numTables; i++) {
+for (int j = 0; j < numServers; j++) {
numRegionsPerServerPerTable[i][j] = 0;
}
}
for (int i=0; i < regionIndexToServerIndex.length; i++) {
if (regionIndexToServerIndex[i] >= 0) {
-
numRegionsPerServerPerTable[regionIndexToServerIndex[i]][regionIndexToTableIndex[i]]++;
+
numRegionsPerServerPerTable[regionIndexToTableIndex[i]][regionIndexToServerIndex[i]]++;
numRegionsPerTable[regionIndexToTableIndex[i]]++;
}
}
- // Avoid repeated computation for planning
- meanRegionsPerTable = new double[numTables];
- regionSkewByTable = new double[numTables];
- maxRegionSkewByTable = new double[numTables];
- minRegionSkewByTable = new double[numTables];
-
- for (int i = 0; i < numTables; i++) {
-meanRegionsPerTable[i] = Double.valueOf(numRegionsPerTable[i]) /
numServers;
-minRegionSkewByTable[i] +=
DoubleArrayCost.getMinSkew(numRegionsPerTable[i], numServers);
-maxRegionSkewByTable[i] +=
DoubleArrayCost.getMaxSkew(numRegionsPerTable[i], numServers);
- }
-
- for (int[] aNumRegionsPerServerPerTable : numRegionsPerServerPerTable) {
-for (int tableIdx = 0; tableIdx < aNumRegionsPerServerPerTable.length;
tableIdx++) {
- regionSkewByTable[tableIdx] +=
-Math.abs(aNumRegionsPerServerPerTable[tableIdx] -
meanRegionsPerTable[tableIdx]);
-}
- }
-
for (int
[hbase] branch branch-2 updated: HBASE-26311 Balancer gets stuck in cohosted replica distribution (#3804)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2 by this push:
new e06208f HBASE-26311 Balancer gets stuck in cohosted replica
distribution (#3804)
e06208f is described below
commit e06208f3d666434711d2342d74648a157ce595b5
Author: clarax
AuthorDate: Wed Nov 3 11:58:38 2021 -0700
HBASE-26311 Balancer gets stuck in cohosted replica distribution (#3804)
Signed-off-by: Huaxiang Sun
---
.../master/balancer/BalancerClusterState.java | 56 --
.../hbase/master/balancer/DoubleArrayCost.java | 23 ++---
.../hbase/master/balancer/MoveCostFunction.java| 4 +-
.../balancer/RegionCountSkewCostFunction.java | 14 +-
.../master/balancer/TableSkewCostFunction.java | 29 ++-
.../hbase/master/balancer/BalancerTestBase.java| 1 -
.../hbase/master/balancer/BalancerTestBase2.java | 1 -
.../master/balancer/TestBaseLoadBalancer.java | 2 -
.../hbase/master/balancer/TestDoubleArrayCost.java | 2 +-
.../balancer/TestStochasticBalancerJmxMetrics.java | 1 -
.../balancer/TestStochasticLoadBalancer.java | 6 +--
.../TestStochasticLoadBalancerBalanceCluster.java | 1 -
...ochasticLoadBalancerRegionReplicaSameHosts.java | 1 -
...ochasticLoadBalancerRegionReplicaWithRacks.java | 1 -
14 files changed, 61 insertions(+), 81 deletions(-)
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BalancerClusterState.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BalancerClusterState.java
index b49deb8..b6c4408 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BalancerClusterState.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BalancerClusterState.java
@@ -84,12 +84,8 @@ class BalancerClusterState {
int[] regionIndexToServerIndex; // regionIndex -> serverIndex
int[] initialRegionIndexToServerIndex; // regionIndex -> serverIndex
(initial cluster state)
int[] regionIndexToTableIndex; // regionIndex -> tableIndex
- int[][] numRegionsPerServerPerTable; // serverIndex -> tableIndex -> #
regions
+ int[][] numRegionsPerServerPerTable; // tableIndex -> serverIndex -> #
regions
int[] numRegionsPerTable; // tableIndex -> region count
- double[] meanRegionsPerTable; // mean region count per table
- double[] regionSkewByTable; // skew on RS per by table
- double[] minRegionSkewByTable; // min skew on RS per by table
- double[] maxRegionSkewByTable; // max skew on RS per by table
int[] numMaxRegionsPerTable; // tableIndex -> max number of regions in a
single RS
int[] regionIndexToPrimaryIndex; // regionIndex -> regionIndex of the primary
boolean hasRegionReplicas = false; // whether there is regions with replicas
@@ -302,42 +298,24 @@ class BalancerClusterState {
}
numTables = tables.size();
-LOG.debug("Number of tables={}", numTables);
-numRegionsPerServerPerTable = new int[numServers][numTables];
+LOG.debug("Number of tables={}, number of hosts={}, number of racks={}",
numTables,
+ numHosts, numRacks);
+numRegionsPerServerPerTable = new int[numTables][numServers];
numRegionsPerTable = new int[numTables];
-for (int i = 0; i < numServers; i++) {
- for (int j = 0; j < numTables; j++) {
+for (int i = 0; i < numTables; i++) {
+ for (int j = 0; j < numServers; j++) {
numRegionsPerServerPerTable[i][j] = 0;
}
}
for (int i = 0; i < regionIndexToServerIndex.length; i++) {
if (regionIndexToServerIndex[i] >= 0) {
-
numRegionsPerServerPerTable[regionIndexToServerIndex[i]][regionIndexToTableIndex[i]]++;
+
numRegionsPerServerPerTable[regionIndexToTableIndex[i]][regionIndexToServerIndex[i]]++;
numRegionsPerTable[regionIndexToTableIndex[i]]++;
}
}
-// Avoid repeated computation for planning
-meanRegionsPerTable = new double[numTables];
-regionSkewByTable = new double[numTables];
-maxRegionSkewByTable = new double[numTables];
-minRegionSkewByTable = new double[numTables];
-
-for (int i = 0; i < numTables; i++) {
- meanRegionsPerTable[i] = Double.valueOf(numRegionsPerTable[i]) /
numServers;
- minRegionSkewByTable[i] +=
DoubleArrayCost.getMinSkew(numRegionsPerTable[i], numServers);
- maxRegionSkewByTable[i] +=
DoubleArrayCost.getMaxSkew(numRegionsPerTable[i], numServers);
-}
-
-for (int[] aNumRegionsPerServerPerTable : numRegionsPerServerPerTable) {
- for (int tableIdx = 0; tableIdx < aNumRegionsPerServerPerTable.length;
tableIdx++) {
-regionSkewByTable[tableIdx] +=
- Math.abs(aNumRegionsPerServerPerTable[tableIdx] -
meanRegionsPerTabl
[hbase] branch master updated: HBASE-26311 Balancer gets stuck in cohosted replica distribution (#3724)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/master by this push:
new cd7a509 HBASE-26311 Balancer gets stuck in cohosted replica
distribution (#3724)
cd7a509 is described below
commit cd7a5099d45a8ed82377dbbe1f1561d8f51f4be4
Author: clarax
AuthorDate: Wed Nov 3 11:57:37 2021 -0700
HBASE-26311 Balancer gets stuck in cohosted replica distribution (#3724)
Signed-off-by: Huaxiang Sun
---
.../master/balancer/BalancerClusterState.java | 53 ++
.../hbase/master/balancer/DoubleArrayCost.java | 23 +++---
.../hbase/master/balancer/MoveCostFunction.java| 4 +-
.../balancer/RegionCountSkewCostFunction.java | 15 +-
.../master/balancer/TableSkewCostFunction.java | 30 +++-
.../balancer/StochasticBalancerTestBase.java | 1 -
.../balancer/StochasticBalancerTestBase2.java | 1 -
.../master/balancer/TestBaseLoadBalancer.java | 2 -
.../hbase/master/balancer/TestDoubleArrayCost.java | 2 +-
.../balancer/TestStochasticLoadBalancer.java | 6 +--
.../TestStochasticLoadBalancerBalanceCluster.java | 1 -
.../TestStochasticLoadBalancerLargeCluster.java| 1 -
...ochasticLoadBalancerRegionReplicaSameHosts.java | 1 -
...ochasticLoadBalancerRegionReplicaWithRacks.java | 6 ---
.../TestStochasticLoadBalancerSmallCluster.java| 2 +-
.../balancer/TestStochasticBalancerJmxMetrics.java | 1 -
16 files changed, 68 insertions(+), 81 deletions(-)
diff --git
a/hbase-balancer/src/main/java/org/apache/hadoop/hbase/master/balancer/BalancerClusterState.java
b/hbase-balancer/src/main/java/org/apache/hadoop/hbase/master/balancer/BalancerClusterState.java
index dbe23ef..f15ca92 100644
---
a/hbase-balancer/src/main/java/org/apache/hadoop/hbase/master/balancer/BalancerClusterState.java
+++
b/hbase-balancer/src/main/java/org/apache/hadoop/hbase/master/balancer/BalancerClusterState.java
@@ -84,12 +84,9 @@ class BalancerClusterState {
int[] regionIndexToServerIndex; // regionIndex -> serverIndex
int[] initialRegionIndexToServerIndex; // regionIndex -> serverIndex
(initial cluster state)
int[] regionIndexToTableIndex; // regionIndex -> tableIndex
- int[][] numRegionsPerServerPerTable; // serverIndex -> tableIndex -> #
regions
+ int[][] numRegionsPerServerPerTable; // tableIndex -> serverIndex ->
tableIndex -> # regions
int[] numRegionsPerTable; // tableIndex -> region count
double[] meanRegionsPerTable; // mean region count per table
- double[] regionSkewByTable; // skew on RS per by table
- double[] minRegionSkewByTable; // min skew on RS per by table
- double[] maxRegionSkewByTable; // max skew on RS per by table
int[] regionIndexToPrimaryIndex; // regionIndex -> regionIndex of the primary
boolean hasRegionReplicas = false; // whether there is regions with replicas
@@ -283,6 +280,11 @@ class BalancerClusterState {
regionIndex++;
}
+if (LOG.isDebugEnabled()) {
+ for (int i = 0; i < numServers; i++) {
+LOG.debug("server {} has {} regions", i, regionsPerServer[i].length);
+ }
+}
for (int i = 0; i < serversPerHostList.size(); i++) {
serversPerHost[i] = new int[serversPerHostList.get(i).size()];
for (int j = 0; j < serversPerHost[i].length; j++) {
@@ -303,40 +305,29 @@ class BalancerClusterState {
}
numTables = tables.size();
-LOG.debug("Number of tables={}", numTables);
-numRegionsPerServerPerTable = new int[numServers][numTables];
+LOG.debug("Number of tables={}, number of hosts={}, number of racks={}",
numTables,
+ numHosts, numRacks);
+numRegionsPerServerPerTable = new int[numTables][numServers];
numRegionsPerTable = new int[numTables];
-for (int i = 0; i < numServers; i++) {
- for (int j = 0; j < numTables; j++) {
+for (int i = 0; i < numTables; i++) {
+ for (int j = 0; j < numServers; j++) {
numRegionsPerServerPerTable[i][j] = 0;
}
}
for (int i = 0; i < regionIndexToServerIndex.length; i++) {
if (regionIndexToServerIndex[i] >= 0) {
-
numRegionsPerServerPerTable[regionIndexToServerIndex[i]][regionIndexToTableIndex[i]]++;
+
numRegionsPerServerPerTable[regionIndexToTableIndex[i]][regionIndexToServerIndex[i]]++;
numRegionsPerTable[regionIndexToTableIndex[i]]++;
}
}
// Avoid repeated computation for planning
meanRegionsPerTable = new double[numTables];
-regionSkewByTable = new double[numTables];
-maxRegionSkewByTable = new double[numTables];
-minRegionSkewByTable = new double[numTables];
for (int i = 0; i < numTables; i++) {
meanRegionsPerTable[i] = Double.valueOf(numRegion
[hbase] branch branch-2.4 updated: HBASE-26327 Replicas cohosted on a rack shouldn't keep triggering balancer (#3789)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2.4 by this push:
new 8392dc0 HBASE-26327 Replicas cohosted on a rack shouldn't keep
triggering balancer (#3789)
8392dc0 is described below
commit 8392dc0ad2039805262354e3335f528961f82094
Author: clarax
AuthorDate: Mon Oct 25 15:44:42 2021 -0700
HBASE-26327 Replicas cohosted on a rack shouldn't keep triggering balancer
(#3789)
Signed-off-by: Huaxiang Sun
---
.../apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java | 6 +-
.../master/balancer/TestStochasticLoadBalancerRegionReplica.java| 3 ++-
2 files changed, 3 insertions(+), 6 deletions(-)
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
index 8c8419d..de706cb 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
@@ -316,11 +316,7 @@ public class StochasticLoadBalancer extends
BaseLoadBalancer {
if (Math.abs(regionReplicaHostCostFunction.cost()) >
CostFunction.COST_EPSILON) {
return true;
}
-regionReplicaRackCostFunction.init(c);
-if (Math.abs(regionReplicaRackCostFunction.cost()) >
CostFunction.COST_EPSILON) {
- return true;
-}
-return false;
+return (Math.abs(regionReplicaRackCostFunction.cost()) >
CostFunction.COST_EPSILON);
}
@Override
diff --git
a/hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancerRegionReplica.java
b/hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancerRegionReplica.java
index 9634f9a..686c7cf 100644
---
a/hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancerRegionReplica.java
+++
b/hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancerRegionReplica.java
@@ -18,6 +18,7 @@
package org.apache.hadoop.hbase.master.balancer;
import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertTrue;
import java.util.ArrayList;
@@ -163,7 +164,7 @@ public class TestStochasticLoadBalancerRegionReplica
extends BalancerTestBase {
map.put(s2, regionsOnS2);
// add another server so that the cluster has some host on another rack
map.put(ServerName.valueOf("host2", 1000, 1), randomRegions(1));
-assertTrue(
+assertFalse(
loadBalancer.needsBalance(HConstants.ENSEMBLE_TABLE_NAME,
new Cluster(map, null, null, new ForTestRackManagerOne(;
}
[hbase] branch branch-2 updated: HBASE-26327 Replicas cohosted on a rack shouldn't keep triggering balancer (#3788)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2 by this push:
new 32eb23c HBASE-26327 Replicas cohosted on a rack shouldn't keep
triggering balancer (#3788)
32eb23c is described below
commit 32eb23c8f36381fc7da2c288133a04db7b799679
Author: clarax
AuthorDate: Mon Oct 25 15:44:11 2021 -0700
HBASE-26327 Replicas cohosted on a rack shouldn't keep triggering balancer
(#3788)
Signed-off-by: Huaxiang Sun
---
.../apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java | 6 +-
.../master/balancer/TestStochasticLoadBalancerRegionReplica.java| 4 ++--
.../balancer/TestStochasticLoadBalancerRegionReplicaWithRacks.java | 2 +-
3 files changed, 4 insertions(+), 8 deletions(-)
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
index 3dc7670..f2993de 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
@@ -337,11 +337,7 @@ public class StochasticLoadBalancer extends
BaseLoadBalancer {
if (Math.abs(regionReplicaHostCostFunction.cost()) >
CostFunction.COST_EPSILON) {
return true;
}
-regionReplicaRackCostFunction.prepare(c);
-if (Math.abs(regionReplicaRackCostFunction.cost()) >
CostFunction.COST_EPSILON) {
- return true;
-}
-return false;
+return (Math.abs(regionReplicaHostCostFunction.cost()) >
CostFunction.COST_EPSILON);
}
@RestrictedApi(explanation = "Should only be called in tests", link = "",
diff --git
a/hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancerRegionReplica.java
b/hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancerRegionReplica.java
index 8262e2c..cccb854 100644
---
a/hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancerRegionReplica.java
+++
b/hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancerRegionReplica.java
@@ -18,6 +18,7 @@
package org.apache.hadoop.hbase.master.balancer;
import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertTrue;
import java.util.ArrayList;
@@ -161,8 +162,7 @@ public class TestStochasticLoadBalancerRegionReplica
extends BalancerTestBase {
map.put(s2, regionsOnS2);
// add another server so that the cluster has some host on another rack
map.put(ServerName.valueOf("host2", 1000, 1), randomRegions(1));
-assertTrue(
- loadBalancer.needsBalance(HConstants.ENSEMBLE_TABLE_NAME,
+assertFalse(loadBalancer.needsBalance(HConstants.ENSEMBLE_TABLE_NAME,
new BalancerClusterState(map, null, null, new
ForTestRackManagerOne(;
}
diff --git
a/hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancerRegionReplicaWithRacks.java
b/hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancerRegionReplicaWithRacks.java
index b682c4b..ea305d3 100644
---
a/hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancerRegionReplicaWithRacks.java
+++
b/hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancerRegionReplicaWithRacks.java
@@ -51,7 +51,7 @@ public class TestStochasticLoadBalancerRegionReplicaWithRacks
extends BalancerTe
@Test
public void testRegionReplicationOnMidClusterWithRacks() {
-conf.setLong(StochasticLoadBalancer.MAX_STEPS_KEY, 1000L);
+conf.setLong(StochasticLoadBalancer.MAX_STEPS_KEY, 1L);
conf.setFloat("hbase.master.balancer.stochastic.maxMovePercent", 1.0f);
conf.setLong("hbase.master.balancer.stochastic.maxRunningTime", 120 *
1000); // 120 sec
loadBalancer.onConfigurationChange(conf);
[hbase] branch master updated (ffb20ca -> 7af5277)
This is an automated email from the ASF dual-hosted git repository. huaxiangsun pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hbase.git. from ffb20ca HBASE-26390 Upload src tarball to nightlies for nightly jobs (#3793) add 7af5277 HBASE-26327 Replicas cohosted on a rack shouldn't keep triggering Bal… (#3729) No new revisions were added by this update. Summary of changes: .../hadoop/hbase/master/balancer/StochasticLoadBalancer.java | 9 + .../master/balancer/TestStochasticLoadBalancerRegionReplica.java | 4 ++-- 2 files changed, 3 insertions(+), 10 deletions(-)
[hbase] branch branch-2.4 updated: HBASE-26308 Sum of multiplier of cost functions is not populated prop… (#3782)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2.4 by this push:
new d23a5b5 HBASE-26308 Sum of multiplier of cost functions is not
populated prop… (#3782)
d23a5b5 is described below
commit d23a5b5aac4fd2b9bf7d0b4e4b46a8a68b9f5195
Author: clarax
AuthorDate: Fri Oct 22 09:51:12 2021 -0700
HBASE-26308 Sum of multiplier of cost functions is not populated prop…
(#3782)
Signed-off-by: Duo Zhang
Signed-off-by: Huaxiang Sun
---
.../master/balancer/StochasticLoadBalancer.java| 42 --
1 file changed, 23 insertions(+), 19 deletions(-)
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
index 216b665..8c8419d 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
@@ -149,8 +149,7 @@ public class StochasticLoadBalancer extends
BaseLoadBalancer {
private List candidateGenerators;
private List costFunctions; // FindBugs: Wants this protected;
IS2_INCONSISTENT_SYNC
- // To save currently configed sum of multiplier. Defaulted at 1 for cases
that carry high cost
- private float sumMultiplier = 1.0f;
+ private float sumMultiplier;
// to save and report costs to JMX
private double curOverallCost = 0d;
private double[] tempFunctionCosts;
@@ -240,8 +239,9 @@ public class StochasticLoadBalancer extends
BaseLoadBalancer {
LOG.info(
"Loaded config; maxSteps=" + maxSteps + ", runMaxSteps=" + runMaxSteps +
", stepsPerRegion=" + stepsPerRegion +
-", maxRunningTime=" + maxRunningTime + ", isByTable=" + isByTable
-+ ", CostFunctions=" + Arrays.toString(getCostFunctionNames()) + "
etc.");
+", maxRunningTime=" + maxRunningTime + ", isByTable=" + isByTable +
+", CostFunctions=" + Arrays.toString(getCostFunctionNames()) +
+" , sum of multiplier of cost functions = " + sumMultiplier + " etc.");
}
private void loadCustomCostFunctions(Configuration conf) {
@@ -334,31 +334,24 @@ public class StochasticLoadBalancer extends
BaseLoadBalancer {
return false;
}
if (areSomeRegionReplicasColocated(cluster)) {
- LOG.info("Running balancer because at least one server hosts replicas of
the same region.");
+ LOG.info("Running balancer because at least one server hosts replicas of
the same region." +
+" function cost={}", functionCost());
return true;
}
if (idleRegionServerExist(cluster)){
- LOG.info("Running balancer because cluster has idle server(s).");
+ LOG.info("Running balancer because cluster has idle server(s)."+
+" function cost={}", functionCost());
return true;
}
-sumMultiplier = 0.0f;
double total = 0.0;
for (CostFunction c : costFunctions) {
- float multiplier = c.getMultiplier();
- double cost = c.cost();
if (!c.isNeeded()) {
LOG.trace("{} not needed", c.getClass().getSimpleName());
continue;
}
- total += cost * multiplier;
- sumMultiplier += multiplier;
-}
-if (sumMultiplier <= 0) {
- LOG.error("At least one cost function needs a multiplier > 0. For
example, set "
-+ "hbase.master.balancer.stochastic.regionCountCost to a positive
value or default");
- return false;
+ total += c.cost() * c.getMultiplier();
}
boolean balanced = (total / sumMultiplier < minCostNeedBalance);
@@ -434,6 +427,17 @@ public class StochasticLoadBalancer extends
BaseLoadBalancer {
long startTime = EnvironmentEdgeManager.currentTime();
initCosts(cluster);
+sumMultiplier = 0;
+for (CostFunction c : costFunctions) {
+ if(c.isNeeded()) {
+sumMultiplier += c.getMultiplier();
+ }
+}
+if (sumMultiplier <= 0) {
+ LOG.error("At least one cost function needs a multiplier > 0. For
example, set "
++ "hbase.master.balancer.stochastic.regionCountCost to a positive
value or default");
+ return null;
+}
if (!needsBalance(tableName, cluster)) {
return null;
@@ -605,8 +609,8 @@ public class StochasticLoadBalancer extends
BaseLoadBalancer {
builder.append(", ");
double cost = c.cost();
builder.append("imbalance=" + cost);
-if (cost < minCostNeedBalance) {
-
[hbase] branch branch-2 updated: HBASE-26308 Sum of multiplier of cost functions is not populated properly when we have a shortcut for trigger (#3783)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2 by this push:
new f299d56 HBASE-26308 Sum of multiplier of cost functions is not
populated properly when we have a shortcut for trigger (#3783)
f299d56 is described below
commit f299d569349c4d938c4ed60f67d6e06251ebc19d
Author: clarax
AuthorDate: Fri Oct 22 09:46:58 2021 -0700
HBASE-26308 Sum of multiplier of cost functions is not populated properly
when we have a shortcut for trigger (#3783)
---
.../master/balancer/StochasticLoadBalancer.java| 44 --
1 file changed, 25 insertions(+), 19 deletions(-)
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
index 8c01dea..3dc7670 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
@@ -136,7 +136,7 @@ public class StochasticLoadBalancer extends
BaseLoadBalancer {
private List candidateGenerators;
private List costFunctions; // FindBugs: Wants this protected;
IS2_INCONSISTENT_SYNC
// To save currently configed sum of multiplier. Defaulted at 1 for cases
that carry high cost
- private float sumMultiplier = 1.0f;
+ private float sumMultiplier;
// to save and report costs to JMX
private double curOverallCost = 0d;
private double[] tempFunctionCosts;
@@ -267,10 +267,12 @@ public class StochasticLoadBalancer extends
BaseLoadBalancer {
this.namedQueueRecorder = NamedQueueRecorder.getInstance(conf);
}
-LOG.info("Loaded config; maxSteps=" + maxSteps + ", runMaxSteps=" +
runMaxSteps +
- ", stepsPerRegion=" + stepsPerRegion +
- ", maxRunningTime=" + maxRunningTime + ", isByTable=" + isByTable + ",
CostFunctions=" +
- Arrays.toString(getCostFunctionNames()) + " etc.");
+LOG.info(
+ "Loaded config; maxSteps=" + maxSteps + ", runMaxSteps=" + runMaxSteps +
+", stepsPerRegion=" + stepsPerRegion +
+", maxRunningTime=" + maxRunningTime + ", isByTable=" + isByTable +
+", CostFunctions=" + Arrays.toString(getCostFunctionNames()) +
+" , sum of multiplier of cost functions = " + sumMultiplier + " etc.");
}
@Override
@@ -355,31 +357,24 @@ public class StochasticLoadBalancer extends
BaseLoadBalancer {
return false;
}
if (areSomeRegionReplicasColocated(cluster)) {
- LOG.info("Running balancer because at least one server hosts replicas of
the same region.");
+ LOG.info("Running balancer because at least one server hosts replicas of
the same region." +
+" function cost={}", functionCost());
return true;
}
if (idleRegionServerExist(cluster)){
- LOG.info("Running balancer because cluster has idle server(s).");
+ LOG.info("Running balancer because cluster has idle server(s)."+
+" function cost={}", functionCost());
return true;
}
-sumMultiplier = 0.0f;
double total = 0.0;
for (CostFunction c : costFunctions) {
- float multiplier = c.getMultiplier();
- double cost = c.cost();
if (!c.isNeeded()) {
LOG.trace("{} not needed", c.getClass().getSimpleName());
continue;
}
- total += cost * multiplier;
- sumMultiplier += multiplier;
-}
-if (sumMultiplier <= 0) {
- LOG.error("At least one cost function needs a multiplier > 0. For
example, set "
-+ "hbase.master.balancer.stochastic.regionCountCost to a positive
value or default");
- return false;
+ total += c.cost() * c.getMultiplier();
}
boolean balanced = (total / sumMultiplier < minCostNeedBalance);
@@ -467,6 +462,17 @@ public class StochasticLoadBalancer extends
BaseLoadBalancer {
long startTime = EnvironmentEdgeManager.currentTime();
initCosts(cluster);
+sumMultiplier = 0;
+for (CostFunction c : costFunctions) {
+ if(c.isNeeded()) {
+sumMultiplier += c.getMultiplier();
+ }
+}
+if (sumMultiplier <= 0) {
+ LOG.error("At least one cost function needs a multiplier > 0. For
example, set "
++ "hbase.master.balancer.stochastic.regionCountCost to a positive
value or default");
+ return null;
+}
double currentCost = computeCost(cluster, Double.MAX_VALUE);
curOverallCost = currentCost;
@@ -638,8 +644,8 @@
[hbase] branch master updated: HBASE-26308 Sum of multiplier of cost functions is not populated properly when we have a shortcut for trigger (#3710)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/master by this push:
new 0d49824 HBASE-26308 Sum of multiplier of cost functions is not
populated properly when we have a shortcut for trigger (#3710)
0d49824 is described below
commit 0d4982404cc7ed813ae4bbf8c4cc0f5c26f29d47
Author: clarax
AuthorDate: Thu Oct 21 11:19:21 2021 -0700
HBASE-26308 Sum of multiplier of cost functions is not populated properly
when we have a shortcut for trigger (#3710)
Signed-off-by: Duo Zhang
Signed-off-by: Huaxiang Sun
---
.../master/balancer/StochasticLoadBalancer.java| 49 --
1 file changed, 28 insertions(+), 21 deletions(-)
diff --git
a/hbase-balancer/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
b/hbase-balancer/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
index eaa923e..eb16cdf 100644
---
a/hbase-balancer/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
+++
b/hbase-balancer/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
@@ -132,7 +132,7 @@ public class StochasticLoadBalancer extends
BaseLoadBalancer {
private List candidateGenerators;
private List costFunctions; // FindBugs: Wants this protected;
IS2_INCONSISTENT_SYNC
// To save currently configed sum of multiplier. Defaulted at 1 for cases
that carry high cost
- private float sumMultiplier = 1.0f;
+ private float sumMultiplier;
// to save and report costs to JMX
private double curOverallCost = 0d;
private double[] tempFunctionCosts;
@@ -248,11 +248,12 @@ public class StochasticLoadBalancer extends
BaseLoadBalancer {
curFunctionCosts = new double[costFunctions.size()];
tempFunctionCosts = new double[costFunctions.size()];
-LOG.info("Loaded config; maxSteps=" + maxSteps + ", runMaxSteps=" +
runMaxSteps,
- ", stepsPerRegion=" + stepsPerRegion +
- ", maxRunningTime=" + maxRunningTime + ", isByTable=" + isByTable + ",
CostFunctions=" +
- Arrays.toString(getCostFunctionNames()) + " etc.");
- }
+LOG.info(
+ "Loaded config; maxSteps=" + maxSteps + ", runMaxSteps=" + runMaxSteps +
+", stepsPerRegion=" + stepsPerRegion +
+", maxRunningTime=" + maxRunningTime + ", isByTable=" + isByTable +
+", CostFunctions=" + Arrays.toString(getCostFunctionNames()) +
+" , sum of multiplier of cost functions = " + sumMultiplier + "
etc."); }
@Override
public void updateClusterMetrics(ClusterMetrics st) {
@@ -345,33 +346,27 @@ public class StochasticLoadBalancer extends
BaseLoadBalancer {
return false;
}
if (areSomeRegionReplicasColocated(cluster)) {
- LOG.info("Running balancer because at least one server hosts replicas of
the same region.");
+ LOG.info("Running balancer because at least one server hosts replicas of
the same region." +
+" function cost={}", functionCost());
return true;
}
if (idleRegionServerExist(cluster)){
- LOG.info("Running balancer because cluster has idle server(s).");
+ LOG.info("Running balancer because at least one server hosts replicas of
the same region." +
+"regionReplicaRackCostFunction={}",
regionReplicaRackCostFunction.cost());
+ LOG.info("Running balancer because cluster has idle server(s)."+
+" function cost={}", functionCost());
return true;
}
-sumMultiplier = 0.0f;
double total = 0.0;
for (CostFunction c : costFunctions) {
- float multiplier = c.getMultiplier();
- double cost = c.cost();
if (!c.isNeeded()) {
LOG.trace("{} not needed", c.getClass().getSimpleName());
continue;
}
- total += cost * multiplier;
- sumMultiplier += multiplier;
+ total += c.cost() * c.getMultiplier();
}
-if (sumMultiplier <= 0) {
- LOG.error("At least one cost function needs a multiplier > 0. For
example, set "
-+ "hbase.master.balancer.stochastic.regionCountCost to a positive
value or default");
- return false;
-}
-
boolean balanced = (total / sumMultiplier < minCostNeedBalance);
if (balanced) {
@@ -435,6 +430,18 @@ public class StochasticLoadBalancer extends
BaseLoadBalancer {
initCosts(cluster);
+sumMultiplier = 0;
+for (CostFunction c : costFunctions) {
+ if(c.isNeeded()) {
+sumMultiplier += c.getMultiplier();
+ }
+}
+if (sumMultiplier <= 0) {
+ LOG.error("At l
[hbase-operator-tools] branch master updated: HBASE-26338 hbck2 setRegionState cannot set replica region state (#94)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hbase-operator-tools.git
The following commit(s) were added to refs/heads/master by this push:
new 98bb584 HBASE-26338 hbck2 setRegionState cannot set replica region
state (#94)
98bb584 is described below
commit 98bb5849f86ba531bac3928fb63f04576795fc53
Author: huaxiangsun
AuthorDate: Mon Oct 11 13:45:25 2021 -0700
HBASE-26338 hbck2 setRegionState cannot set replica region state (#94)
Signed-off-by: Wellington Ramos Chevreuil
---
.../src/main/java/org/apache/hbase/HBCK2.java | 62 +++---
.../src/test/java/org/apache/hbase/TestHBCK2.java | 37 +
2 files changed, 92 insertions(+), 7 deletions(-)
diff --git a/hbase-hbck2/src/main/java/org/apache/hbase/HBCK2.java
b/hbase-hbck2/src/main/java/org/apache/hbase/HBCK2.java
index 932f39a..a788a83 100644
--- a/hbase-hbck2/src/main/java/org/apache/hbase/HBCK2.java
+++ b/hbase-hbck2/src/main/java/org/apache/hbase/HBCK2.java
@@ -22,7 +22,9 @@ import java.io.IOException;
import java.io.InputStream;
import java.io.PrintWriter;
import java.io.StringWriter;
+import java.io.UnsupportedEncodingException;
import java.lang.reflect.Method;
+import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
@@ -89,6 +91,9 @@ public class HBCK2 extends Configured implements
org.apache.hadoop.util.Tool {
private static final Logger LOG = LoggerFactory.getLogger(HBCK2.class);
private static final int EXIT_SUCCESS = 0;
static final int EXIT_FAILURE = 1;
+ /** The delimiter for meta columns for replicaIds 0 */
+ private static final char META_REPLICA_ID_DELIMITER = '_';
+
// Commands
private static final String SET_TABLE_STATE = "setTableState";
private static final String ASSIGNS = "assigns";
@@ -167,6 +172,18 @@ public class HBCK2 extends Configured implements
org.apache.hadoop.util.Tool {
}
}
+ public static byte[] getRegionStateColumn(int replicaId) {
+try {
+ return replicaId == 0 ? HConstants.STATE_QUALIFIER
+ : (HConstants.STATE_QUALIFIER_STR + META_REPLICA_ID_DELIMITER
+ + String.format(RegionInfo.REPLICA_ID_FORMAT,
+ replicaId)).getBytes(StandardCharsets.UTF_8.name());
+} catch (UnsupportedEncodingException e) {
+ // should never happen!
+ throw new IllegalArgumentException("UTF8 decoding is not supported", e);
+}
+ }
+
TableState setTableState(Hbck hbck, TableName tableName, TableState.State
state)
throws IOException {
return hbck.setTableStateInMeta(new TableState(tableName, state));
@@ -175,6 +192,12 @@ public class HBCK2 extends Configured implements
org.apache.hadoop.util.Tool {
int setRegionState(ClusterConnection connection, String region,
RegionState.State newState)
throws IOException {
+return setRegionState(connection, region, 0, newState);
+ }
+
+ int setRegionState(ClusterConnection connection, String region, int
replicaId,
+ RegionState.State newState)
+ throws IOException {
if (newState == null) {
throw new IllegalArgumentException("State can't be null.");
}
@@ -186,19 +209,27 @@ public class HBCK2 extends Configured implements
org.apache.hadoop.util.Tool {
Result result = table.getScanner(scan).next();
if (result != null) {
byte[] currentStateValue = result.getValue(HConstants.CATALOG_FAMILY,
-HConstants.STATE_QUALIFIER);
+ getRegionStateColumn(replicaId));
if (currentStateValue == null) {
System.out.println("WARN: Region state info on meta was NULL");
} else {
currentState = RegionState.State.valueOf(
-org.apache.hadoop.hbase.util.Bytes.toString(currentStateValue));
+
org.apache.hadoop.hbase.util.Bytes.toString(currentStateValue));
}
Put put = new Put(result.getRow());
- put.addColumn(HConstants.CATALOG_FAMILY, HConstants.STATE_QUALIFIER,
-org.apache.hadoop.hbase.util.Bytes.toBytes(newState.name()));
+ put.addColumn(HConstants.CATALOG_FAMILY, getRegionStateColumn(replicaId),
+ org.apache.hadoop.hbase.util.Bytes.toBytes(newState.name()));
table.put(put);
- System.out.println("Changed region " + region + " STATE from "
-+ currentState + " to " + newState);
+
+ if (replicaId == 0) {
+System.out.println("Changed region " + region + " STATE from "
++ currentState + " to " + newState);
+ } else {
+System.out.println("Changed STATE for replica reigon " + replicaId +
+" of primary region " + region +
+"from " + cur
[hbase] branch branch-2.4 updated: HBASE-26255 Add an option to use region location from meta table in TableSnapshotInputFormat (#3661) (#3669)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2.4 by this push:
new be23541 HBASE-26255 Add an option to use region location from meta
table in TableSnapshotInputFormat (#3661) (#3669)
be23541 is described below
commit be23541f1182c355d4ae367607abeb5d8a97d06d
Author: huaxiangsun
AuthorDate: Thu Sep 9 10:43:53 2021 -0700
HBASE-26255 Add an option to use region location from meta table in
TableSnapshotInputFormat (#3661) (#3669)
Signed-off-by: Anoop Sam John
---
.../mapreduce/TableSnapshotInputFormatImpl.java| 61 ++
.../mapreduce/TestTableSnapshotInputFormat.java| 29 ++
2 files changed, 79 insertions(+), 11 deletions(-)
diff --git
a/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java
b/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java
index 5ed9be6..c3f05f4 100644
---
a/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java
+++
b/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java
@@ -32,9 +32,13 @@ import org.apache.hadoop.fs.Path;
import org.apache.hadoop.hbase.HDFSBlocksDistribution;
import org.apache.hadoop.hbase.HDFSBlocksDistribution.HostAndWeight;
import org.apache.hadoop.hbase.HRegionInfo;
+import org.apache.hadoop.hbase.HRegionLocation;
import org.apache.hadoop.hbase.PrivateCellUtil;
import org.apache.hadoop.hbase.client.ClientSideRegionScanner;
+import org.apache.hadoop.hbase.client.Connection;
+import org.apache.hadoop.hbase.client.ConnectionFactory;
import org.apache.hadoop.hbase.client.IsolationLevel;
+import org.apache.hadoop.hbase.client.RegionLocator;
import org.apache.hadoop.hbase.client.Result;
import org.apache.hadoop.hbase.client.Scan;
import org.apache.hadoop.hbase.client.TableDescriptor;
@@ -102,6 +106,15 @@ public class TableSnapshotInputFormatImpl {
public static final boolean SNAPSHOT_INPUTFORMAT_LOCALITY_ENABLED_DEFAULT =
true;
/**
+ * Whether to calculate the Snapshot region location by region location from
meta.
+ * It is much faster than computing block locations for splits.
+ */
+ public static final String SNAPSHOT_INPUTFORMAT_LOCALITY_BY_REGION_LOCATION
=
+"hbase.TableSnapshotInputFormat.locality.by.region.location";
+
+ public static final boolean
SNAPSHOT_INPUTFORMAT_LOCALITY_BY_REGION_LOCATION_DEFAULT = false;
+
+ /**
* In some scenario, scan limited rows on each InputSplit for sampling data
extraction
*/
public static final String SNAPSHOT_INPUTFORMAT_ROW_LIMIT_PER_INPUTSPLIT =
@@ -392,17 +405,49 @@ public class TableSnapshotInputFormatImpl {
SNAPSHOT_INPUTFORMAT_SCAN_METRICS_ENABLED_DEFAULT);
scan.setScanMetricsEnabled(scanMetricsEnabled);
+boolean useRegionLoc =
conf.getBoolean(SNAPSHOT_INPUTFORMAT_LOCALITY_BY_REGION_LOCATION,
+ SNAPSHOT_INPUTFORMAT_LOCALITY_BY_REGION_LOCATION_DEFAULT);
+
+Connection connection = null;
+RegionLocator regionLocator = null;
+if (localityEnabled && useRegionLoc) {
+ Configuration newConf = new Configuration(conf);
+ newConf.setInt("hbase.hconnection.threads.max", 1);
+ try {
+connection = ConnectionFactory.createConnection(newConf);
+regionLocator = connection.getRegionLocator(htd.getTableName());
+
+/* Get all locations for the table and cache it */
+regionLocator.getAllRegionLocations();
+ } finally {
+if (connection != null) {
+ connection.close();
+}
+ }
+}
+
List splits = new ArrayList<>();
for (HRegionInfo hri : regionManifests) {
// load region descriptor
+ List hosts = null;
+ if (localityEnabled) {
+if (regionLocator != null) {
+ /* Get Location from the local cache */
+ HRegionLocation
+location = regionLocator.getRegionLocation(hri.getStartKey(),
false);
+
+ hosts = new ArrayList<>(1);
+ hosts.add(location.getHostname());
+} else {
+ hosts = calculateLocationsForInputSplit(conf, htd, hri, tableDir);
+}
+ }
if (numSplits > 1) {
byte[][] sp = sa.split(hri.getStartKey(), hri.getEndKey(), numSplits,
true);
for (int i = 0; i < sp.length - 1; i++) {
if (PrivateCellUtil.overlappingKeys(scan.getStartRow(),
scan.getStopRow(), sp[i],
sp[i + 1])) {
-List hosts =
-calculateLocationsForInputSplit(conf, htd, hri, tableDir,
localityEnabled);
Scan boundedScan = new Scan(scan);
if (scan.getStartRow().length == 0) {
@@ -425,8 +470,7 @@ public clas
[hbase] branch branch-2 updated: HBASE-26255 Add an option to use region location from meta table in TableSnapshotInputFormat (#3661) (#3668)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2 by this push:
new 21d9741 HBASE-26255 Add an option to use region location from meta
table in TableSnapshotInputFormat (#3661) (#3668)
21d9741 is described below
commit 21d9741adb60fea71fdf1996233430a75a324968
Author: huaxiangsun
AuthorDate: Thu Sep 9 10:43:36 2021 -0700
HBASE-26255 Add an option to use region location from meta table in
TableSnapshotInputFormat (#3661) (#3668)
Signed-off-by: Anoop Sam John
---
.../mapreduce/TableSnapshotInputFormatImpl.java| 61 ++
.../mapreduce/TestTableSnapshotInputFormat.java| 29 ++
2 files changed, 79 insertions(+), 11 deletions(-)
diff --git
a/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java
b/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java
index 5ed9be6..c3f05f4 100644
---
a/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java
+++
b/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java
@@ -32,9 +32,13 @@ import org.apache.hadoop.fs.Path;
import org.apache.hadoop.hbase.HDFSBlocksDistribution;
import org.apache.hadoop.hbase.HDFSBlocksDistribution.HostAndWeight;
import org.apache.hadoop.hbase.HRegionInfo;
+import org.apache.hadoop.hbase.HRegionLocation;
import org.apache.hadoop.hbase.PrivateCellUtil;
import org.apache.hadoop.hbase.client.ClientSideRegionScanner;
+import org.apache.hadoop.hbase.client.Connection;
+import org.apache.hadoop.hbase.client.ConnectionFactory;
import org.apache.hadoop.hbase.client.IsolationLevel;
+import org.apache.hadoop.hbase.client.RegionLocator;
import org.apache.hadoop.hbase.client.Result;
import org.apache.hadoop.hbase.client.Scan;
import org.apache.hadoop.hbase.client.TableDescriptor;
@@ -102,6 +106,15 @@ public class TableSnapshotInputFormatImpl {
public static final boolean SNAPSHOT_INPUTFORMAT_LOCALITY_ENABLED_DEFAULT =
true;
/**
+ * Whether to calculate the Snapshot region location by region location from
meta.
+ * It is much faster than computing block locations for splits.
+ */
+ public static final String SNAPSHOT_INPUTFORMAT_LOCALITY_BY_REGION_LOCATION
=
+"hbase.TableSnapshotInputFormat.locality.by.region.location";
+
+ public static final boolean
SNAPSHOT_INPUTFORMAT_LOCALITY_BY_REGION_LOCATION_DEFAULT = false;
+
+ /**
* In some scenario, scan limited rows on each InputSplit for sampling data
extraction
*/
public static final String SNAPSHOT_INPUTFORMAT_ROW_LIMIT_PER_INPUTSPLIT =
@@ -392,17 +405,49 @@ public class TableSnapshotInputFormatImpl {
SNAPSHOT_INPUTFORMAT_SCAN_METRICS_ENABLED_DEFAULT);
scan.setScanMetricsEnabled(scanMetricsEnabled);
+boolean useRegionLoc =
conf.getBoolean(SNAPSHOT_INPUTFORMAT_LOCALITY_BY_REGION_LOCATION,
+ SNAPSHOT_INPUTFORMAT_LOCALITY_BY_REGION_LOCATION_DEFAULT);
+
+Connection connection = null;
+RegionLocator regionLocator = null;
+if (localityEnabled && useRegionLoc) {
+ Configuration newConf = new Configuration(conf);
+ newConf.setInt("hbase.hconnection.threads.max", 1);
+ try {
+connection = ConnectionFactory.createConnection(newConf);
+regionLocator = connection.getRegionLocator(htd.getTableName());
+
+/* Get all locations for the table and cache it */
+regionLocator.getAllRegionLocations();
+ } finally {
+if (connection != null) {
+ connection.close();
+}
+ }
+}
+
List splits = new ArrayList<>();
for (HRegionInfo hri : regionManifests) {
// load region descriptor
+ List hosts = null;
+ if (localityEnabled) {
+if (regionLocator != null) {
+ /* Get Location from the local cache */
+ HRegionLocation
+location = regionLocator.getRegionLocation(hri.getStartKey(),
false);
+
+ hosts = new ArrayList<>(1);
+ hosts.add(location.getHostname());
+} else {
+ hosts = calculateLocationsForInputSplit(conf, htd, hri, tableDir);
+}
+ }
if (numSplits > 1) {
byte[][] sp = sa.split(hri.getStartKey(), hri.getEndKey(), numSplits,
true);
for (int i = 0; i < sp.length - 1; i++) {
if (PrivateCellUtil.overlappingKeys(scan.getStartRow(),
scan.getStopRow(), sp[i],
sp[i + 1])) {
-List hosts =
-calculateLocationsForInputSplit(conf, htd, hri, tableDir,
localityEnabled);
Scan boundedScan = new Scan(scan);
if (scan.getStartRow().length == 0) {
@@ -425,8 +470,7 @@ public clas
[hbase] branch branch-2.3 updated: HBASE-26255 Add an option to use region location from meta table in TableSnapshotInputFormat (#3661) (#3670)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch branch-2.3
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2.3 by this push:
new 66fed7c HBASE-26255 Add an option to use region location from meta
table in TableSnapshotInputFormat (#3661) (#3670)
66fed7c is described below
commit 66fed7c743c3e247a9eb1ee323e3f7c3a7f609c2
Author: huaxiangsun
AuthorDate: Thu Sep 9 10:42:49 2021 -0700
HBASE-26255 Add an option to use region location from meta table in
TableSnapshotInputFormat (#3661) (#3670)
Signed-off-by: Anoop Sam John
---
.../mapreduce/TableSnapshotInputFormatImpl.java| 61 ++
.../mapreduce/TestTableSnapshotInputFormat.java| 29 ++
2 files changed, 79 insertions(+), 11 deletions(-)
diff --git
a/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java
b/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java
index 5ed9be6..c3f05f4 100644
---
a/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java
+++
b/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java
@@ -32,9 +32,13 @@ import org.apache.hadoop.fs.Path;
import org.apache.hadoop.hbase.HDFSBlocksDistribution;
import org.apache.hadoop.hbase.HDFSBlocksDistribution.HostAndWeight;
import org.apache.hadoop.hbase.HRegionInfo;
+import org.apache.hadoop.hbase.HRegionLocation;
import org.apache.hadoop.hbase.PrivateCellUtil;
import org.apache.hadoop.hbase.client.ClientSideRegionScanner;
+import org.apache.hadoop.hbase.client.Connection;
+import org.apache.hadoop.hbase.client.ConnectionFactory;
import org.apache.hadoop.hbase.client.IsolationLevel;
+import org.apache.hadoop.hbase.client.RegionLocator;
import org.apache.hadoop.hbase.client.Result;
import org.apache.hadoop.hbase.client.Scan;
import org.apache.hadoop.hbase.client.TableDescriptor;
@@ -102,6 +106,15 @@ public class TableSnapshotInputFormatImpl {
public static final boolean SNAPSHOT_INPUTFORMAT_LOCALITY_ENABLED_DEFAULT =
true;
/**
+ * Whether to calculate the Snapshot region location by region location from
meta.
+ * It is much faster than computing block locations for splits.
+ */
+ public static final String SNAPSHOT_INPUTFORMAT_LOCALITY_BY_REGION_LOCATION
=
+"hbase.TableSnapshotInputFormat.locality.by.region.location";
+
+ public static final boolean
SNAPSHOT_INPUTFORMAT_LOCALITY_BY_REGION_LOCATION_DEFAULT = false;
+
+ /**
* In some scenario, scan limited rows on each InputSplit for sampling data
extraction
*/
public static final String SNAPSHOT_INPUTFORMAT_ROW_LIMIT_PER_INPUTSPLIT =
@@ -392,17 +405,49 @@ public class TableSnapshotInputFormatImpl {
SNAPSHOT_INPUTFORMAT_SCAN_METRICS_ENABLED_DEFAULT);
scan.setScanMetricsEnabled(scanMetricsEnabled);
+boolean useRegionLoc =
conf.getBoolean(SNAPSHOT_INPUTFORMAT_LOCALITY_BY_REGION_LOCATION,
+ SNAPSHOT_INPUTFORMAT_LOCALITY_BY_REGION_LOCATION_DEFAULT);
+
+Connection connection = null;
+RegionLocator regionLocator = null;
+if (localityEnabled && useRegionLoc) {
+ Configuration newConf = new Configuration(conf);
+ newConf.setInt("hbase.hconnection.threads.max", 1);
+ try {
+connection = ConnectionFactory.createConnection(newConf);
+regionLocator = connection.getRegionLocator(htd.getTableName());
+
+/* Get all locations for the table and cache it */
+regionLocator.getAllRegionLocations();
+ } finally {
+if (connection != null) {
+ connection.close();
+}
+ }
+}
+
List splits = new ArrayList<>();
for (HRegionInfo hri : regionManifests) {
// load region descriptor
+ List hosts = null;
+ if (localityEnabled) {
+if (regionLocator != null) {
+ /* Get Location from the local cache */
+ HRegionLocation
+location = regionLocator.getRegionLocation(hri.getStartKey(),
false);
+
+ hosts = new ArrayList<>(1);
+ hosts.add(location.getHostname());
+} else {
+ hosts = calculateLocationsForInputSplit(conf, htd, hri, tableDir);
+}
+ }
if (numSplits > 1) {
byte[][] sp = sa.split(hri.getStartKey(), hri.getEndKey(), numSplits,
true);
for (int i = 0; i < sp.length - 1; i++) {
if (PrivateCellUtil.overlappingKeys(scan.getStartRow(),
scan.getStopRow(), sp[i],
sp[i + 1])) {
-List hosts =
-calculateLocationsForInputSplit(conf, htd, hri, tableDir,
localityEnabled);
Scan boundedScan = new Scan(scan);
if (scan.getStartRow().length == 0) {
@@ -425,8 +470,7 @@ public clas
[hbase] branch master updated: HBASE-26255 Add an option to use region location from meta table in TableSnapshotInputFormat (#3661)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/master by this push:
new b8ba3f7 HBASE-26255 Add an option to use region location from meta
table in TableSnapshotInputFormat (#3661)
b8ba3f7 is described below
commit b8ba3f788bac747d8266b90b6966cfdb8435450f
Author: huaxiangsun
AuthorDate: Wed Sep 8 09:46:34 2021 -0700
HBASE-26255 Add an option to use region location from meta table in
TableSnapshotInputFormat (#3661)
Signed-off-by: Anoop Sam John
---
.../mapreduce/TableSnapshotInputFormatImpl.java| 63 +-
.../mapreduce/TestTableSnapshotInputFormat.java| 29 ++
2 files changed, 80 insertions(+), 12 deletions(-)
diff --git
a/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java
b/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java
index abdd8f4..22c19be 100644
---
a/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java
+++
b/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java
@@ -31,10 +31,14 @@ import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.hbase.HDFSBlocksDistribution;
import org.apache.hadoop.hbase.HDFSBlocksDistribution.HostAndWeight;
+import org.apache.hadoop.hbase.HRegionLocation;
import org.apache.hadoop.hbase.PrivateCellUtil;
import org.apache.hadoop.hbase.client.ClientSideRegionScanner;
+import org.apache.hadoop.hbase.client.Connection;
+import org.apache.hadoop.hbase.client.ConnectionFactory;
import org.apache.hadoop.hbase.client.IsolationLevel;
import org.apache.hadoop.hbase.client.RegionInfo;
+import org.apache.hadoop.hbase.client.RegionLocator;
import org.apache.hadoop.hbase.client.Result;
import org.apache.hadoop.hbase.client.Scan;
import org.apache.hadoop.hbase.client.TableDescriptor;
@@ -102,6 +106,15 @@ public class TableSnapshotInputFormatImpl {
public static final boolean SNAPSHOT_INPUTFORMAT_LOCALITY_ENABLED_DEFAULT =
true;
/**
+ * Whether to calculate the Snapshot region location by region location from
meta.
+ * It is much faster than computing block locations for splits.
+ */
+ public static final String SNAPSHOT_INPUTFORMAT_LOCALITY_BY_REGION_LOCATION
=
+"hbase.TableSnapshotInputFormat.locality.by.region.location";
+
+ public static final boolean
SNAPSHOT_INPUTFORMAT_LOCALITY_BY_REGION_LOCATION_DEFAULT = false;
+
+ /**
* In some scenario, scan limited rows on each InputSplit for sampling data
extraction
*/
public static final String SNAPSHOT_INPUTFORMAT_ROW_LIMIT_PER_INPUTSPLIT =
@@ -392,17 +405,49 @@ public class TableSnapshotInputFormatImpl {
SNAPSHOT_INPUTFORMAT_SCAN_METRICS_ENABLED_DEFAULT);
scan.setScanMetricsEnabled(scanMetricsEnabled);
+boolean useRegionLoc =
conf.getBoolean(SNAPSHOT_INPUTFORMAT_LOCALITY_BY_REGION_LOCATION,
+ SNAPSHOT_INPUTFORMAT_LOCALITY_BY_REGION_LOCATION_DEFAULT);
+
+Connection connection = null;
+RegionLocator regionLocator = null;
+if (localityEnabled && useRegionLoc) {
+ Configuration newConf = new Configuration(conf);
+ newConf.setInt("hbase.hconnection.threads.max", 1);
+ try {
+connection = ConnectionFactory.createConnection(newConf);
+regionLocator = connection.getRegionLocator(htd.getTableName());
+
+/* Get all locations for the table and cache it */
+regionLocator.getAllRegionLocations();
+ } finally {
+if (connection != null) {
+ connection.close();
+}
+ }
+}
+
List splits = new ArrayList<>();
for (RegionInfo hri : regionManifests) {
// load region descriptor
+ List hosts = null;
+ if (localityEnabled) {
+if (regionLocator != null) {
+ /* Get Location from the local cache */
+ HRegionLocation
+location = regionLocator.getRegionLocation(hri.getStartKey(),
false);
+
+ hosts = new ArrayList<>(1);
+ hosts.add(location.getHostname());
+} else {
+ hosts = calculateLocationsForInputSplit(conf, htd, hri, tableDir);
+}
+ }
if (numSplits > 1) {
byte[][] sp = sa.split(hri.getStartKey(), hri.getEndKey(), numSplits,
true);
for (int i = 0; i < sp.length - 1; i++) {
if (PrivateCellUtil.overlappingKeys(scan.getStartRow(),
scan.getStopRow(), sp[i],
sp[i + 1])) {
-List hosts =
-calculateLocationsForInputSplit(conf, htd, hri, tableDir,
localityEnabled);
Scan boundedScan = new Scan(scan);
if (scan.getStartRow().length == 0) {
@@ -425,8 +470,7 @@ public clas
[hbase] branch master updated (dd293c8 -> afbdd41)
This is an automated email from the ASF dual-hosted git repository. huaxiangsun pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hbase.git. from dd293c8 HBASE-26230 Start an in process HRegionServer in maintenance mode (#3659) add afbdd41 HBASE-26237 Improve computation complexity for primaryRegionCountSkewCostFunction (#3657) No new revisions were added by this update. Summary of changes: .../master/balancer/PrimaryRegionCountSkewCostFunction.java| 10 +++--- 1 file changed, 7 insertions(+), 3 deletions(-)
[hbase] branch branch-2.3 updated: HBASE-26108 add option to disable scanMetrics in TableSnapshotInputFormat (#3521)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch branch-2.3
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2.3 by this push:
new 31dc541 HBASE-26108 add option to disable scanMetrics in
TableSnapshotInputFormat (#3521)
31dc541 is described below
commit 31dc5412f3bc85a2a130531ba0fba518dbaa3b7a
Author: huaxiangsun
AuthorDate: Thu Jul 22 12:22:53 2021 -0700
HBASE-26108 add option to disable scanMetrics in TableSnapshotInputFormat
(#3521)
Signed-off-by Anoop Sam John
---
.../hbase/mapreduce/TableSnapshotInputFormatImpl.java | 13 -
1 file changed, 12 insertions(+), 1 deletion(-)
diff --git
a/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java
b/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java
index 28b832e..5ed9be6 100644
---
a/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java
+++
b/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java
@@ -108,6 +108,14 @@ public class TableSnapshotInputFormatImpl {
"hbase.TableSnapshotInputFormat.row.limit.per.inputsplit";
/**
+ * Whether to enable scan metrics on Scan, default to true
+ */
+ public static final String SNAPSHOT_INPUTFORMAT_SCAN_METRICS_ENABLED =
+"hbase.TableSnapshotInputFormat.scan_metrics.enabled";
+
+ public static final boolean
SNAPSHOT_INPUTFORMAT_SCAN_METRICS_ENABLED_DEFAULT = true;
+
+ /**
* Implementation class for InputSplit logic common between mapred and
mapreduce.
*/
public static class InputSplit implements Writable {
@@ -240,7 +248,6 @@ public class TableSnapshotInputFormatImpl {
scan.setIsolationLevel(IsolationLevel.READ_UNCOMMITTED);
// disable caching of data blocks
scan.setCacheBlocks(false);
- scan.setScanMetricsEnabled(true);
scanner =
new ClientSideRegionScanner(conf, fs, new Path(split.restoreDir),
htd, hri, scan, null);
@@ -381,6 +388,10 @@ public class TableSnapshotInputFormatImpl {
boolean localityEnabled =
conf.getBoolean(SNAPSHOT_INPUTFORMAT_LOCALITY_ENABLED_KEY,
SNAPSHOT_INPUTFORMAT_LOCALITY_ENABLED_DEFAULT);
+boolean scanMetricsEnabled =
conf.getBoolean(SNAPSHOT_INPUTFORMAT_SCAN_METRICS_ENABLED,
+ SNAPSHOT_INPUTFORMAT_SCAN_METRICS_ENABLED_DEFAULT);
+scan.setScanMetricsEnabled(scanMetricsEnabled);
+
List splits = new ArrayList<>();
for (HRegionInfo hri : regionManifests) {
// load region descriptor
[hbase] branch branch-2.4 updated: HBASE-26108 add option to disable scanMetrics in TableSnapshotInputFormat (#3520)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2.4 by this push:
new d10847c HBASE-26108 add option to disable scanMetrics in
TableSnapshotInputFormat (#3520)
d10847c is described below
commit d10847cc3fc4d5c29e25fd08dbd6a806f1c2
Author: huaxiangsun
AuthorDate: Thu Jul 22 12:22:31 2021 -0700
HBASE-26108 add option to disable scanMetrics in TableSnapshotInputFormat
(#3520)
Signed-off-by Anoop Sam John
---
.../hbase/mapreduce/TableSnapshotInputFormatImpl.java | 13 -
1 file changed, 12 insertions(+), 1 deletion(-)
diff --git
a/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java
b/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java
index 28b832e..5ed9be6 100644
---
a/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java
+++
b/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java
@@ -108,6 +108,14 @@ public class TableSnapshotInputFormatImpl {
"hbase.TableSnapshotInputFormat.row.limit.per.inputsplit";
/**
+ * Whether to enable scan metrics on Scan, default to true
+ */
+ public static final String SNAPSHOT_INPUTFORMAT_SCAN_METRICS_ENABLED =
+"hbase.TableSnapshotInputFormat.scan_metrics.enabled";
+
+ public static final boolean
SNAPSHOT_INPUTFORMAT_SCAN_METRICS_ENABLED_DEFAULT = true;
+
+ /**
* Implementation class for InputSplit logic common between mapred and
mapreduce.
*/
public static class InputSplit implements Writable {
@@ -240,7 +248,6 @@ public class TableSnapshotInputFormatImpl {
scan.setIsolationLevel(IsolationLevel.READ_UNCOMMITTED);
// disable caching of data blocks
scan.setCacheBlocks(false);
- scan.setScanMetricsEnabled(true);
scanner =
new ClientSideRegionScanner(conf, fs, new Path(split.restoreDir),
htd, hri, scan, null);
@@ -381,6 +388,10 @@ public class TableSnapshotInputFormatImpl {
boolean localityEnabled =
conf.getBoolean(SNAPSHOT_INPUTFORMAT_LOCALITY_ENABLED_KEY,
SNAPSHOT_INPUTFORMAT_LOCALITY_ENABLED_DEFAULT);
+boolean scanMetricsEnabled =
conf.getBoolean(SNAPSHOT_INPUTFORMAT_SCAN_METRICS_ENABLED,
+ SNAPSHOT_INPUTFORMAT_SCAN_METRICS_ENABLED_DEFAULT);
+scan.setScanMetricsEnabled(scanMetricsEnabled);
+
List splits = new ArrayList<>();
for (HRegionInfo hri : regionManifests) {
// load region descriptor
[hbase] branch branch-2 updated: HBASE-26108 add option to disable scanMetrics in TableSnapshotInputFormat (#3519)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2 by this push:
new 6854933 HBASE-26108 add option to disable scanMetrics in
TableSnapshotInputFormat (#3519)
6854933 is described below
commit 68549338d0bb3e4fd8b3902a798e45ee53882257
Author: huaxiangsun
AuthorDate: Thu Jul 22 12:03:52 2021 -0700
HBASE-26108 add option to disable scanMetrics in TableSnapshotInputFormat
(#3519)
Signed-off-by Anoop Sam John
---
.../hbase/mapreduce/TableSnapshotInputFormatImpl.java | 13 -
1 file changed, 12 insertions(+), 1 deletion(-)
diff --git
a/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java
b/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java
index 28b832e..5ed9be6 100644
---
a/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java
+++
b/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java
@@ -108,6 +108,14 @@ public class TableSnapshotInputFormatImpl {
"hbase.TableSnapshotInputFormat.row.limit.per.inputsplit";
/**
+ * Whether to enable scan metrics on Scan, default to true
+ */
+ public static final String SNAPSHOT_INPUTFORMAT_SCAN_METRICS_ENABLED =
+"hbase.TableSnapshotInputFormat.scan_metrics.enabled";
+
+ public static final boolean
SNAPSHOT_INPUTFORMAT_SCAN_METRICS_ENABLED_DEFAULT = true;
+
+ /**
* Implementation class for InputSplit logic common between mapred and
mapreduce.
*/
public static class InputSplit implements Writable {
@@ -240,7 +248,6 @@ public class TableSnapshotInputFormatImpl {
scan.setIsolationLevel(IsolationLevel.READ_UNCOMMITTED);
// disable caching of data blocks
scan.setCacheBlocks(false);
- scan.setScanMetricsEnabled(true);
scanner =
new ClientSideRegionScanner(conf, fs, new Path(split.restoreDir),
htd, hri, scan, null);
@@ -381,6 +388,10 @@ public class TableSnapshotInputFormatImpl {
boolean localityEnabled =
conf.getBoolean(SNAPSHOT_INPUTFORMAT_LOCALITY_ENABLED_KEY,
SNAPSHOT_INPUTFORMAT_LOCALITY_ENABLED_DEFAULT);
+boolean scanMetricsEnabled =
conf.getBoolean(SNAPSHOT_INPUTFORMAT_SCAN_METRICS_ENABLED,
+ SNAPSHOT_INPUTFORMAT_SCAN_METRICS_ENABLED_DEFAULT);
+scan.setScanMetricsEnabled(scanMetricsEnabled);
+
List splits = new ArrayList<>();
for (HRegionInfo hri : regionManifests) {
// load region descriptor
[hbase] branch master updated: HBASE-26108 add option to disable scanMetrics in TableSnapshotInputFormat (#3516)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/master by this push:
new d15f3cb HBASE-26108 add option to disable scanMetrics in
TableSnapshotInputFormat (#3516)
d15f3cb is described below
commit d15f3cbea37a9027653a151e47177bfd59a6a3a9
Author: huaxiangsun
AuthorDate: Thu Jul 22 11:20:15 2021 -0700
HBASE-26108 add option to disable scanMetrics in TableSnapshotInputFormat
(#3516)
Signed-off-by Anoop Sam John
---
.../hbase/mapreduce/TableSnapshotInputFormatImpl.java | 13 -
1 file changed, 12 insertions(+), 1 deletion(-)
diff --git
a/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java
b/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java
index c9e3f68..abdd8f4 100644
---
a/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java
+++
b/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java
@@ -108,6 +108,14 @@ public class TableSnapshotInputFormatImpl {
"hbase.TableSnapshotInputFormat.row.limit.per.inputsplit";
/**
+ * Whether to enable scan metrics on Scan, default to true
+ */
+ public static final String SNAPSHOT_INPUTFORMAT_SCAN_METRICS_ENABLED =
+"hbase.TableSnapshotInputFormat.scan_metrics.enabled";
+
+ public static final boolean
SNAPSHOT_INPUTFORMAT_SCAN_METRICS_ENABLED_DEFAULT = true;
+
+ /**
* Implementation class for InputSplit logic common between mapred and
mapreduce.
*/
public static class InputSplit implements Writable {
@@ -240,7 +248,6 @@ public class TableSnapshotInputFormatImpl {
scan.setIsolationLevel(IsolationLevel.READ_UNCOMMITTED);
// disable caching of data blocks
scan.setCacheBlocks(false);
- scan.setScanMetricsEnabled(true);
scanner =
new ClientSideRegionScanner(conf, fs, new Path(split.restoreDir),
htd, hri, scan, null);
@@ -381,6 +388,10 @@ public class TableSnapshotInputFormatImpl {
boolean localityEnabled =
conf.getBoolean(SNAPSHOT_INPUTFORMAT_LOCALITY_ENABLED_KEY,
SNAPSHOT_INPUTFORMAT_LOCALITY_ENABLED_DEFAULT);
+boolean scanMetricsEnabled =
conf.getBoolean(SNAPSHOT_INPUTFORMAT_SCAN_METRICS_ENABLED,
+ SNAPSHOT_INPUTFORMAT_SCAN_METRICS_ENABLED_DEFAULT);
+scan.setScanMetricsEnabled(scanMetricsEnabled);
+
List splits = new ArrayList<>();
for (RegionInfo hri : regionManifests) {
// load region descriptor
[hbase] branch master updated: HBASE-25721 Add 2.3.5 to the downloads page (#3114)
This is an automated email from the ASF dual-hosted git repository. huaxiangsun pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hbase.git The following commit(s) were added to refs/heads/master by this push: new 446f22f HBASE-25721 Add 2.3.5 to the downloads page (#3114) 446f22f is described below commit 446f22f05cb19a8d591438c2a95575c5d1a40f26 Author: huaxiangsun AuthorDate: Wed Mar 31 20:16:39 2021 -0700 HBASE-25721 Add 2.3.5 to the downloads page (#3114) Signed-off-by: Michael Stack --- src/site/xdoc/downloads.xml | 16 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/src/site/xdoc/downloads.xml b/src/site/xdoc/downloads.xml index b17f9b5..0294dfe 100644 --- a/src/site/xdoc/downloads.xml +++ b/src/site/xdoc/downloads.xml @@ -68,24 +68,24 @@ under the License. -2.3.4 +2.3.5 -2021/01/22 +2021/03/31 -https://downloads.apache.org/hbase/2.3.4/api_compare_2.3.3_to_2.3.4RC4.html;>2.3.3 vs 2.3.4 +https://downloads.apache.org/hbase/2.3.5/api_compare_2.3.4_to_2.3.5RC1.html;>2.3.4 vs 2.3.5 -https://downloads.apache.org/hbase/2.3.4/CHANGES.md;>Changes +https://downloads.apache.org/hbase/2.3.5/CHANGES.md;>Changes -https://downloads.apache.org/hbase/2.3.4/RELEASENOTES.md;>Release Notes +https://downloads.apache.org/hbase/2.3.5/RELEASENOTES.md;>Release Notes -https://www.apache.org/dyn/closer.lua/hbase/2.3.4/hbase-2.3.4-src.tar.gz;>src (https://downloads.apache.org/hbase/2.3.4/hbase-2.3.4-src.tar.gz.sha512;>sha512 https://downloads.apache.org/hbase/2.3.4/hbase-2.3.4-src.tar.gz.asc;>asc) -https://www.apache.org/dyn/closer.lua/hbase/2.3.4/hbase-2.3.4-bin.tar.gz;>bin (https://downloads.apache.org/hbase/2.3.4/hbase-2.3.4-bin.tar.gz.sha512;>sha512 https://downloads.apache.org/hbase/2.3.4/hbase-2.3.4-bin.tar.gz.asc;>asc) -https://www.apache.org/dyn/closer.lua/hbase/2.3.4/hbase-2.3.4-client-bin.tar.gz;>client-bin (https://downloads.apache.org/hbase/2.3.4/hbase-2.3.4-client-bin.tar.gz.sha512;>sha512 https://downloads.apache.org/hbase/2.3.4/hbase-2.3.4-client-bin.tar.gz.asc;>asc) +https://www.apache.org/dyn/closer.lua/hbase/2.3.5/hbase-2.3.5-src.tar.gz;>src (https://downloads.apache.org/hbase/2.3.5/hbase-2.3.5-src.tar.gz.sha512;>sha512 https://downloads.apache.org/hbase/2.3.5/hbase-2.3.5-src.tar.gz.asc;>asc) +https://www.apache.org/dyn/closer.lua/hbase/2.3.5/hbase-2.3.5-bin.tar.gz;>bin (https://downloads.apache.org/hbase/2.3.5/hbase-2.3.5-bin.tar.gz.sha512;>sha512 https://downloads.apache.org/hbase/2.3.5/hbase-2.3.5-bin.tar.gz.asc;>asc) +https://www.apache.org/dyn/closer.lua/hbase/2.3.5/hbase-2.3.5-client-bin.tar.gz;>client-bin (https://downloads.apache.org/hbase/2.3.5/hbase-2.3.5-client-bin.tar.gz.sha512;>sha512 https://downloads.apache.org/hbase/2.3.5/hbase-2.3.5-client-bin.tar.gz.asc;>asc) stable release
[hbase] annotated tag rel/2.3.5 created (now d956bcf)
This is an automated email from the ASF dual-hosted git repository. huaxiangsun pushed a change to annotated tag rel/2.3.5 in repository https://gitbox.apache.org/repos/asf/hbase.git. at d956bcf (tag) tagging 5b1658202140f643b86a3a741d4007a8f97a500d (tag) length 988 bytes by Huaxiang Sun on Wed Mar 31 17:36:57 2021 -0700 - Log - Tag 2.3.5 -BEGIN PGP SIGNATURE- iQIzBAABCAAdFiEEZQ5ukLH6mbYnhA++I6kA1BF8g14FAmBlFakACgkQI6kA1BF8 g14NOw/7B8dda6/nsc9BvBol6L3rWvqZzRxj/9UYCX/eC6gnMZECBfUyFSNJg1Qe 5r1xj7HQ0sI0v0k7TREj7MuJIvQEL+IkrFCA8mxCQk0Fl/+15OsqbzqLNAPqHjrc XdV3+16aLZ35PLZhc8ZkGrljOHyJKZ9b2VCMdEV8YmyMeE6P438dUwj6FzuHOk7b hDj2py7cP+E/MwDtmcEsbOEtqVR7xlA//RulSAciCzk5WvcDm6PVGf/cwxDq3AbN bo3R/LezYCEXkLmJO3D0w7kYinvay/nC9rdQ+9nec303SGnTHo8NOIIA8ORX+Dr5 oFxD8/ENBosJLSjRDJvcusgpxH0ZTSYjDfN14nJvays8ud9KmzdR1WyYa3j0R9iT 8efVckzDQdQSgJNwkIzMuMsvEHS+YrLHiLR8xeeNcZ9sJ3PhNQlzoV42+rIm8aDc l37jtFxhavHBZwB+QFtfjaSdRiTuc9OZHdw3/ChiGL4T6C/eZRtYk0/t5bS/8Gp7 0epXWuDT2jxQB7HSq7ZMzZlGkNEVujxvRuzALwjzjKw2FbUe16NyZ2OSE+X6yZGh DS3ZOiCnQEnRFtENXTi0rGOuV5KhmpIGBQtzODvJRT5JsQlTvW1lbcXuoliDNBi9 PIxfFj5IQAhMeLJf0ALpB/LCGNJi8B1tlFBzWEQWezxAbSKX7PU= =wJhi -END PGP SIGNATURE- --- No new revisions were added by this update.
svn commit: r46735 [4/4] - /dev/hbase/2.3.5RC1/
Added: dev/hbase/2.3.5RC1/api_compare_2.3.4_to_2.3.5RC1.html
==
--- dev/hbase/2.3.5RC1/api_compare_2.3.4_to_2.3.5RC1.html (added)
+++ dev/hbase/2.3.5RC1/api_compare_2.3.4_to_2.3.5RC1.html Thu Mar 25 23:35:52
2021
@@ -0,0 +1,637 @@
+
+
+http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd;>
+http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
+
+
+
+
+
+hbase: rel/2.3.4 to 2.3.5RC1 compatibility report
+
+body {
+font-family:Arial, sans-serif;
+background-color:White;
+color:Black;
+}
+hr {
+color:Black;
+background-color:Black;
+height:1px;
+border:0;
+}
+h1 {
+margin-bottom:0px;
+padding-bottom:0px;
+font-size:1.625em;
+}
+h2 {
+margin-bottom:0px;
+padding-bottom:0px;
+font-size:1.25em;
+white-space:nowrap;
+}
+div.symbols {
+color:#003E69;
+}
+div.symbols i {
+color:Brown;
+}
+span.section {
+font-weight:bold;
+cursor:pointer;
+color:#003E69;
+white-space:nowrap;
+margin-left:0.3125em;
+}
+span:hover.section {
+color:#336699;
+}
+span.sect_aff {
+cursor:pointer;
+padding-left:1.55em;
+font-size:0.875em;
+color:#cc3300;
+}
+span.ext {
+font-weight:normal;
+}
+span.jar {
+color:#cc3300;
+font-size:0.875em;
+font-weight:bold;
+}
+div.jar_list {
+padding-left:0.4em;
+font-size:0.94em;
+}
+span.pkg_t {
+color:#408080;
+font-size:0.875em;
+}
+span.pkg {
+color:#408080;
+font-size:0.875em;
+font-weight:bold;
+}
+span.cname {
+color:Green;
+font-size:0.875em;
+font-weight:bold;
+}
+span.iname_b {
+font-weight:bold;
+}
+span.iname_a {
+color:#33;
+font-weight:bold;
+font-size:0.94em;
+}
+span.sym_p {
+font-weight:normal;
+white-space:normal;
+}
+span.sym_pd {
+white-space:normal;
+}
+span.sym_p span, span.sym_pd span {
+white-space:nowrap;
+}
+span.attr {
+color:Black;
+font-weight:normal;
+}
+span.deprecated {
+color:Red;
+font-weight:bold;
+font-family:Monaco, monospace;
+}
+div.affect {
+padding-left:1em;
+padding-bottom:10px;
+font-size:0.87em;
+font-style:italic;
+line-height:0.9em;
+}
+div.affected {
+padding-left:2em;
+padding-top:10px;
+}
+table.ptable {
+border-collapse:collapse;
+border:1px outset black;
+margin-left:0.95em;
+margin-top:3px;
+margin-bottom:3px;
+width:56.25em;
+}
+table.ptable td {
+border:1px solid Gray;
+padding:3px;
+font-size:0.875em;
+text-align:left;
+vertical-align:top;
+max-width:28em;
+word-wrap:break-word;
+}
+table.ptable th {
+background-color:#ee;
+font-weight:bold;
+color:#33;
+font-family:Verdana, Arial;
+font-size:0.875em;
+border:1px solid Gray;
+text-align:center;
+vertical-align:top;
+white-space:nowrap;
+padding:3px;
+}
+table.summary {
+border-collapse:collapse;
+border:1px outset black;
+}
+table.summary th {
+background-color:#ee;
+font-weight:normal;
+text-align:left;
+font-size:0.94em;
+white-space:nowrap;
+border:1px inset Gray;
+padding:3px;
+}
+table.summary td {
+text-align:right;
+white-space:nowrap;
+border:1px inset Gray;
+padding:3px 5px 3px 10px;
+}
+span.mngl {
+padding-left:1em;
+font-size:0.875em;
+cursor:text;
+color:#44;
+font-weight:bold;
+}
+span.pleft {
+padding-left:2.5em;
+}
+span.color_p {
+font-style:italic;
+color:Brown;
+}
+span.param {
+font-style:italic;
+}
+span.focus_p {
+font-style:italic;
+background-color:#DCDCDC;
+}
+span.ttype {
+font-weight:normal;
+}
+span.nowrap {
+white-space:nowrap;
+}
+span.value {
+white-space:nowrap;
+font-weight:bold;
+}
+.passed {
+background-color:#CCFFCC;
+font-weight:normal;
+}
+.warning {
+background-color:#F4F4AF;
+font-weight:normal;
+}
+.failed {
+background-color:#FF;
+font-weight:normal;
+}
+.new {
+background-color:#C6DEFF;
+font-weight:normal;
+}
+
+.compatible {
+background-color:#CCFFCC;
+font-weight:normal;
+}
+.almost_compatible {
+background-color:#FFDAA3;
+font-weight:normal;
+}
+.incompatible {
+background-color:#FF;
+font-weight:normal;
+}
+.gray {
+background-color:#DCDCDC;
+font-weight:normal;
+}
+
+.top_ref {
+font-size:0.69em;
+}
+.footer {
+font-size:0.8125em;
+}
+.tabset {
+float:left;
+}
+a.tab {
+border:1px solid Black;
+float:left;
+margin:0px 5px -1px 0px;
+padding:3px 5px 3px 5px;
+position:relative;
+font-size:0.875em;
+background-color:#DDD;
+text-decoration:none;
+color:Black;
+}
+a.disabled:hover
+{
+color:Black;
+background:#EEE;
+}
+a.active:hover
+{
+color:Black;
+background:White;
+}
+a.active {
+border-bottom-color:White;
+background-color:White;
+}
+div.tab {
+border-top:1px solid
svn commit: r46735 [2/4] - /dev/hbase/2.3.5RC1/
Added: dev/hbase/2.3.5RC1/CHANGES.md == --- dev/hbase/2.3.5RC1/CHANGES.md (added) +++ dev/hbase/2.3.5RC1/CHANGES.md Thu Mar 25 23:35:52 2021 @@ -0,0 +1,8708 @@ +# HBASE Changelog + + +## Release 2.3.5 - Unreleased (as of 2021-03-25) + + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | +|: |: | :--- |: | +| [HBASE-25587](https://issues.apache.org/jira/browse/HBASE-25587) | [hbck2] Schedule SCP for all unknown servers | Major | hbase-operator-tools, hbck2 | +| [HBASE-25460](https://issues.apache.org/jira/browse/HBASE-25460) | Expose drainingServers as cluster metric | Major | metrics | +| [HBASE-25496](https://issues.apache.org/jira/browse/HBASE-25496) | add get\_namespace\_rsgroup command | Major | . | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | +|: |: | :--- |: | +| [HBASE-25621](https://issues.apache.org/jira/browse/HBASE-25621) | Balancer should check region plan source to avoid misplace region groups | Major | Balancer | +| [HBASE-25374](https://issues.apache.org/jira/browse/HBASE-25374) | Make REST Client connection and socket time out configurable | Minor | REST | +| [HBASE-25597](https://issues.apache.org/jira/browse/HBASE-25597) | Add row info in Exception when cell size exceeds maxCellSize | Minor | . | +| [HBASE-25635](https://issues.apache.org/jira/browse/HBASE-25635) | CandidateGenerator may miss some region balance actions | Major | Balancer | +| [HBASE-25566](https://issues.apache.org/jira/browse/HBASE-25566) | RoundRobinTableInputFormat | Major | mapreduce | +| [HBASE-25548](https://issues.apache.org/jira/browse/HBASE-25548) | Optionally allow snapshots to preserve cluster's max filesize config by setting it into table descriptor | Major | . | +| [HBASE-25582](https://issues.apache.org/jira/browse/HBASE-25582) | Support setting scan ReadType to be STREAM at cluster level | Major | . | +| [HBASE-23578](https://issues.apache.org/jira/browse/HBASE-23578) | [UI] Master UI shows long stack traces when table is broken | Minor | master, UI | +| [HBASE-25541](https://issues.apache.org/jira/browse/HBASE-25541) | In WALEntryStream, set the current path to null while dequeing the log | Major | . | +| [HBASE-25507](https://issues.apache.org/jira/browse/HBASE-25507) | Leak of ESTABLISHED sockets when compaction encountered "java.io.IOException: Invalid HFile block magic" | Major | Compaction | +| [HBASE-25528](https://issues.apache.org/jira/browse/HBASE-25528) | Dedicated merge dispatch threadpool on master | Minor | master | +| [HBASE-25536](https://issues.apache.org/jira/browse/HBASE-25536) | Remove 0 length wal file from logQueue if it belongs to old sources. | Major | Replication | +| [HBASE-25329](https://issues.apache.org/jira/browse/HBASE-25329) | Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time | Minor | . | +| [HBASE-25475](https://issues.apache.org/jira/browse/HBASE-25475) | Improve unit test for HBASE-25445 : SplitWALRemoteProcedure failed to archive split WAL | Minor | wal | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | +|: |: | :--- |: | +| [HBASE-25568](https://issues.apache.org/jira/browse/HBASE-25568) | Upgrade Thrift jar to fix CVE-2020-13949 | Critical | Thrift | +| [HBASE-25590](https://issues.apache.org/jira/browse/HBASE-25590) | Bulkload replication HFileRefs cannot be cleared in some cases where set exclude-namespace/exclude-table-cfs | Major | Replication | +| [HBASE-25562](https://issues.apache.org/jira/browse/HBASE-25562) | ReplicationSourceWALReader log and handle exception immediately without retrying | Major | Replication | +| [HBASE-25685](https://issues.apache.org/jira/browse/HBASE-25685) | asyncprofiler2.0 no longer supports svg; wants html | Major | . | +| [HBASE-25674](https://issues.apache.org/jira/browse/HBASE-25674) | RegionInfo.parseFrom(DataInputStream) sometimes fails to read the protobuf magic marker | Minor | Client | +| [HBASE-25639](https://issues.apache.org/jira/browse/HBASE-25639) | meta replica state is not respected during active master switch | Critical | meta replicas | +| [HBASE-25595](https://issues.apache.org/jira/browse/HBASE-25595) | TestLruBlockCache.testBackgroundEvictionThread is flaky | Major | . | +| [HBASE-25662](https://issues.apache.org/jira/browse/HBASE-25662) | Fix spotbugs warning in RoundRobinTableInputFormat | Major | findbugs | +| [HBASE-25657](https://issues.apache.org/jira/browse/HBASE-25657) | Fix spotbugs warnings after upgrading spotbugs to 4.x | Major | findbugs | +| [HBASE-25646](https://issues.apache.org/jira/browse/HBASE-25646) | Possible Resource Leak in CatalogJanitor | Major | master | +| [HBASE-25626](https://issues.apache.org/jira/browse/HBASE-25626) | Possible Resource Leak in HeterogeneousRegionCountCostFunction | Major |
svn commit: r46735 [1/4] - /dev/hbase/2.3.5RC1/
Author: huaxiangsun Date: Thu Mar 25 23:35:52 2021 New Revision: 46735 Log: Apache hbase 2.3.5RC1 Added: dev/hbase/2.3.5RC1/ dev/hbase/2.3.5RC1/CHANGES.md dev/hbase/2.3.5RC1/RELEASENOTES.md dev/hbase/2.3.5RC1/api_compare_2.3.4_to_2.3.5RC1.html dev/hbase/2.3.5RC1/hbase-2.3.5-bin.tar.gz (with props) dev/hbase/2.3.5RC1/hbase-2.3.5-bin.tar.gz.asc dev/hbase/2.3.5RC1/hbase-2.3.5-bin.tar.gz.sha512 dev/hbase/2.3.5RC1/hbase-2.3.5-client-bin.tar.gz (with props) dev/hbase/2.3.5RC1/hbase-2.3.5-client-bin.tar.gz.asc dev/hbase/2.3.5RC1/hbase-2.3.5-client-bin.tar.gz.sha512 dev/hbase/2.3.5RC1/hbase-2.3.5-src.tar.gz (with props) dev/hbase/2.3.5RC1/hbase-2.3.5-src.tar.gz.asc dev/hbase/2.3.5RC1/hbase-2.3.5-src.tar.gz.sha512
svn commit: r46735 [3/4] - /dev/hbase/2.3.5RC1/
Added: dev/hbase/2.3.5RC1/RELEASENOTES.md == --- dev/hbase/2.3.5RC1/RELEASENOTES.md (added) +++ dev/hbase/2.3.5RC1/RELEASENOTES.md Thu Mar 25 23:35:52 2021 @@ -0,0 +1,12517 @@ +# RELEASENOTES + + +# HBASE 2.3.5 Release Notes + +These release notes cover new developer and user-facing incompatibilities, important issues, features, and major improvements. + + +--- + +* [HBASE-25685](https://issues.apache.org/jira/browse/HBASE-25685) | *Major* | **asyncprofiler2.0 no longer supports svg; wants html** + +If asyncprofiler 1.x, all is good. If asyncprofiler 2.x and it is hbase-2.3.x or hbase-2.4.x, add '?output=html' to get flamegraphs from the profiler. + +Otherwise, if hbase-2.5+ and asyncprofiler2, all works. If asyncprofiler1 and hbase-2.5+, you may have to add '?output=svg' to the query. + + +--- + +* [HBASE-25374](https://issues.apache.org/jira/browse/HBASE-25374) | *Minor* | **Make REST Client connection and socket time out configurable** + +Configuration parameter to set rest client connection timeout + +"hbase.rest.client.conn.timeout" Default is 2 \* 1000 + +"hbase.rest.client.socket.timeout" Default of 30 \* 1000 + + +--- + +* [HBASE-25566](https://issues.apache.org/jira/browse/HBASE-25566) | *Major* | **RoundRobinTableInputFormat** + +Adds RoundRobinTableInputFormat, a subclass of TableInputFormat, that takes the TIF#getSplits list and resorts it so as to spread the InputFormats as broadly about the cluster as possible. RRTIF works to frustrate bunching of InputSplits on RegionServers to avoid the scenario where a few RegionServers are working hard fielding many InputSplits while others idle hosting a few or none. + + +--- + +* [HBASE-25587](https://issues.apache.org/jira/browse/HBASE-25587) | *Major* | **[hbck2] Schedule SCP for all unknown servers** + +Adds scheduleSCPsForUnknownServers to Hbck Service. + + +--- + +* [HBASE-25582](https://issues.apache.org/jira/browse/HBASE-25582) | *Major* | **Support setting scan ReadType to be STREAM at cluster level** + +Adding a new meaning for the config 'hbase.storescanner.pread.max.bytes' when configured with a value \<0. +In HBase 2.x we allow the Scan op to specify a ReadType (PREAD / STREAM/ DEFAULT). When Scan comes with DEFAULT read type, we will start scan with preads and later switch to stream read once we see we are scanning a total data size \> value of hbase.storescanner.pread.max.bytes. (This is calculated for data per region:cf). This config defaults to 4 x of HFile block size = 256 KB by default. +This jira added a new meaning for this config when configured with a -ve value. In such case, for all scans with DEFAULT read type, we will start with STREAM read itself. (Switch at begin of the scan itself) + + +--- + +* [HBASE-25460](https://issues.apache.org/jira/browse/HBASE-25460) | *Major* | **Expose drainingServers as cluster metric** + +Exposed new jmx metrics: "draininigRegionServers" and "numDrainingRegionServers" to provide "comma separated names for regionservers that are put in draining mode" and "num of such regionservers" respectively. + + +--- + +* [HBASE-25615](https://issues.apache.org/jira/browse/HBASE-25615) | *Major* | **Upgrade java version in pre commit docker file** + +jdk8u232-b09 -\> jdk8u282-b08 +jdk-11.0.6\_10 -\> jdk-11.0.10\_9 + + + +# HBASE 2.3.4 Release Notes + +These release notes cover new developer and user-facing incompatibilities, important issues, features, and major improvements. + + +--- + +* [HBASE-25449](https://issues.apache.org/jira/browse/HBASE-25449) | *Major* | **'dfs.client.read.shortcircuit' should not be set in hbase-default.xml** + +The presence of HDFS short-circuit read configuration properties in hbase-default.xml inadvertently causes short-circuit reads to not happen inside of RegionServers, despite short-circuit reads being enabled in hdfs-site.xml. + + +--- + +* [HBASE-25441](https://issues.apache.org/jira/browse/HBASE-25441) | *Critical* | **add security check for some APIs in RSRpcServices** + +RsRpcServices APIs that can be accessed only through Admin rights: +- stopServer +- updateFavoredNodes +- updateConfiguration +- clearRegionBlockCache +- clearSlowLogsResponses + + +--- + +* [HBASE-25432](https://issues.apache.org/jira/browse/HBASE-25432) | *Blocker* | **we should add security checks for setTableStateInMeta and fixMeta** + +setTableStateInMeta and fixMeta can be accessed only through Admin rights + + +--- + +* [HBASE-25318](https://issues.apache.org/jira/browse/HBASE-25318) | *Minor* | **Configure where IntegrationTestImportTsv generates HFiles** + +Added IntegrationTestImportTsv.generatedHFileFolder configuration property to override the default location in IntegrationTestImportTsv. Useful for running the integration test when HDFS Transparent Encryption is enabled. + + +--- + +* [HBASE-25456](https://issues.apache.org/jira/browse/HBASE-25456) | *Critical* |
[hbase] branch branch-2.3 updated (ebcceb9 -> c64850e)
This is an automated email from the ASF dual-hosted git repository. huaxiangsun pushed a change to branch branch-2.3 in repository https://gitbox.apache.org/repos/asf/hbase.git. from ebcceb9 Preparing development version 2.3.6-SNAPSHOT add fd3fdc0 Preparing hbase release 2.3.5RC1; tagging and updates to CHANGES.md and RELEASENOTES.md new c64850e Preparing development version 2.3.6-SNAPSHOT The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.md | 1 + 1 file changed, 1 insertion(+)
[hbase] 01/01: Preparing development version 2.3.6-SNAPSHOT
This is an automated email from the ASF dual-hosted git repository. huaxiangsun pushed a commit to branch branch-2.3 in repository https://gitbox.apache.org/repos/asf/hbase.git commit c64850e85afbd0f80563a72b3b3d7c0aa43b88d3 Author: Huaxiang Sun AuthorDate: Thu Mar 25 20:31:50 2021 + Preparing development version 2.3.6-SNAPSHOT Signed-off-by: Huaxiang Sun --- hbase-annotations/pom.xml | 2 +- hbase-archetypes/hbase-archetype-builder/pom.xml | 2 +- hbase-archetypes/hbase-client-project/pom.xml | 2 +- hbase-archetypes/hbase-shaded-client-project/pom.xml | 2 +- hbase-archetypes/pom.xml | 2 +- hbase-assembly/pom.xml | 2 +- hbase-asyncfs/pom.xml | 2 +- hbase-build-configuration/pom.xml | 2 +- hbase-checkstyle/pom.xml | 4 ++-- hbase-client/pom.xml | 2 +- hbase-common/pom.xml | 2 +- hbase-endpoint/pom.xml | 2 +- hbase-examples/pom.xml | 2 +- hbase-external-blockcache/pom.xml | 2 +- hbase-hadoop-compat/pom.xml| 2 +- hbase-hadoop2-compat/pom.xml | 2 +- hbase-hbtop/pom.xml| 2 +- hbase-http/pom.xml | 2 +- hbase-it/pom.xml | 2 +- hbase-logging/pom.xml | 2 +- hbase-mapreduce/pom.xml| 2 +- hbase-metrics-api/pom.xml | 2 +- hbase-metrics/pom.xml | 2 +- hbase-procedure/pom.xml| 2 +- hbase-protocol-shaded/pom.xml | 2 +- hbase-protocol/pom.xml | 2 +- hbase-replication/pom.xml | 2 +- hbase-resource-bundle/pom.xml | 2 +- hbase-rest/pom.xml | 2 +- hbase-rsgroup/pom.xml | 2 +- hbase-server/pom.xml | 2 +- hbase-shaded/hbase-shaded-check-invariants/pom.xml | 2 +- hbase-shaded/hbase-shaded-client-byo-hadoop/pom.xml| 2 +- hbase-shaded/hbase-shaded-client/pom.xml | 2 +- hbase-shaded/hbase-shaded-mapreduce/pom.xml| 2 +- hbase-shaded/hbase-shaded-testing-util-tester/pom.xml | 2 +- hbase-shaded/hbase-shaded-testing-util/pom.xml | 2 +- hbase-shaded/hbase-shaded-with-hadoop-check-invariants/pom.xml | 2 +- hbase-shaded/pom.xml | 2 +- hbase-shell/pom.xml| 2 +- hbase-testing-util/pom.xml | 2 +- hbase-thrift/pom.xml | 2 +- hbase-zookeeper/pom.xml| 2 +- pom.xml| 2 +- 44 files changed, 45 insertions(+), 45 deletions(-) diff --git a/hbase-annotations/pom.xml b/hbase-annotations/pom.xml index c5a14d3..46d85d7 100644 --- a/hbase-annotations/pom.xml +++ b/hbase-annotations/pom.xml @@ -23,7 +23,7 @@ hbase org.apache.hbase -2.3.5 +2.3.6-SNAPSHOT .. diff --git a/hbase-archetypes/hbase-archetype-builder/pom.xml b/hbase-archetypes/hbase-archetype-builder/pom.xml index d1df340..31f1d7c 100644 --- a/hbase-archetypes/hbase-archetype-builder/pom.xml +++ b/hbase-archetypes/hbase-archetype-builder/pom.xml @@ -25,7 +25,7 @@ hbase-archetypes org.apache.hbase -2.3.5 +2.3.6-SNAPSHOT .. diff --git a/hbase-archetypes/hbase-client-project/pom.xml b/hbase-archetypes/hbase-client-project/pom.xml index 2a64ff9..6618cc4 100644 --- a/hbase-archetypes/hbase-client-project/pom.xml +++ b/hbase-archetypes/hbase-client-project/pom.xml @@ -26,7 +26,7 @@ hbase-archetypes org.apache.hbase -2.3.5 +2.3.6-SNAPSHOT .. hbase-client-project diff --git a/hbase-archetypes/hbase-shaded-client-project/pom.xml b/hbase-archetypes/hbase-shaded-client-project/pom.xml index 1d30621..661b2c0 100644 --- a/hbase-archetypes/hbase-shaded-client-project/pom.xml +++ b/hbase-archetypes/hbase-shaded-client-project/pom.xml @@ -26,7 +26,7 @@ hbase-archetypes org.apache.hbase -2.3.5 +2.3.6-SNAPSHOT .. hbase-shaded-client-project diff --git a/hbase
[hbase] 01/01: Preparing hbase release 2.3.5RC1; tagging and updates to CHANGES.md and RELEASENOTES.md
This is an automated email from the ASF dual-hosted git repository. huaxiangsun pushed a commit to annotated tag 2.3.5RC1 in repository https://gitbox.apache.org/repos/asf/hbase.git commit fd3fdc08d1cd43eb3432a1a70d31c3aece6ecabe Author: Huaxiang Sun AuthorDate: Thu Mar 25 20:31:39 2021 + Preparing hbase release 2.3.5RC1; tagging and updates to CHANGES.md and RELEASENOTES.md Signed-off-by: Huaxiang Sun --- CHANGES.md | 1 + hbase-annotations/pom.xml | 2 +- hbase-archetypes/hbase-archetype-builder/pom.xml | 2 +- hbase-archetypes/hbase-client-project/pom.xml | 2 +- hbase-archetypes/hbase-shaded-client-project/pom.xml | 2 +- hbase-archetypes/pom.xml | 2 +- hbase-assembly/pom.xml | 2 +- hbase-asyncfs/pom.xml | 2 +- hbase-build-configuration/pom.xml | 2 +- hbase-checkstyle/pom.xml | 4 ++-- hbase-client/pom.xml | 2 +- hbase-common/pom.xml | 2 +- hbase-endpoint/pom.xml | 2 +- hbase-examples/pom.xml | 2 +- hbase-external-blockcache/pom.xml | 2 +- hbase-hadoop-compat/pom.xml| 2 +- hbase-hadoop2-compat/pom.xml | 2 +- hbase-hbtop/pom.xml| 2 +- hbase-http/pom.xml | 2 +- hbase-it/pom.xml | 2 +- hbase-logging/pom.xml | 2 +- hbase-mapreduce/pom.xml| 2 +- hbase-metrics-api/pom.xml | 2 +- hbase-metrics/pom.xml | 2 +- hbase-procedure/pom.xml| 2 +- hbase-protocol-shaded/pom.xml | 2 +- hbase-protocol/pom.xml | 2 +- hbase-replication/pom.xml | 2 +- hbase-resource-bundle/pom.xml | 2 +- hbase-rest/pom.xml | 2 +- hbase-rsgroup/pom.xml | 2 +- hbase-server/pom.xml | 2 +- hbase-shaded/hbase-shaded-check-invariants/pom.xml | 2 +- hbase-shaded/hbase-shaded-client-byo-hadoop/pom.xml| 2 +- hbase-shaded/hbase-shaded-client/pom.xml | 2 +- hbase-shaded/hbase-shaded-mapreduce/pom.xml| 2 +- hbase-shaded/hbase-shaded-testing-util-tester/pom.xml | 2 +- hbase-shaded/hbase-shaded-testing-util/pom.xml | 2 +- hbase-shaded/hbase-shaded-with-hadoop-check-invariants/pom.xml | 2 +- hbase-shaded/pom.xml | 2 +- hbase-shell/pom.xml| 2 +- hbase-testing-util/pom.xml | 2 +- hbase-thrift/pom.xml | 2 +- hbase-zookeeper/pom.xml| 2 +- pom.xml| 2 +- 45 files changed, 46 insertions(+), 45 deletions(-) diff --git a/CHANGES.md b/CHANGES.md index 8da9214..b3e53d4 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -57,6 +57,7 @@ | JIRA | Summary | Priority | Component | |: |: | :--- |: | +| [HBASE-25568](https://issues.apache.org/jira/browse/HBASE-25568) | Upgrade Thrift jar to fix CVE-2020-13949 | Critical | Thrift | | [HBASE-25590](https://issues.apache.org/jira/browse/HBASE-25590) | Bulkload replication HFileRefs cannot be cleared in some cases where set exclude-namespace/exclude-table-cfs | Major | Replication | | [HBASE-25562](https://issues.apache.org/jira/browse/HBASE-25562) | ReplicationSourceWALReader log and handle exception immediately without retrying | Major | Replication | | [HBASE-25685](https://issues.apache.org/jira/browse/HBASE-25685) | asyncprofiler2.0 no longer supports svg; wants html | Major | . | diff --git a/hbase-annotations/pom.xml b/hbase-annotations/pom.xml index 46d85d7..c5a14d3 100644 --- a/hbase-annotations/pom.xml +++ b/hbase-annotations/pom.xml @@ -23,7 +23,7 @@ hbase org.apache.hbase -2.3.6-SNAPSHOT +2.3.5 .. diff --git a/hbase-archetypes/hbase-archetype-builder/pom.xml b/hbase-archetypes/hbase-archetype-builder/pom.xml index 31f1d7c..d1df340 100644 --- a/hbase-archetypes/hbase-archetype-builder/pom.xml
[hbase] annotated tag 2.3.5RC1 created (now 5b16582)
This is an automated email from the ASF dual-hosted git repository. huaxiangsun pushed a change to annotated tag 2.3.5RC1 in repository https://gitbox.apache.org/repos/asf/hbase.git. at 5b16582 (tag) tagging fd3fdc08d1cd43eb3432a1a70d31c3aece6ecabe (commit) replaces 2.3.5RC0 by Huaxiang Sun on Thu Mar 25 20:31:39 2021 + - Log - Via create-release -BEGIN PGP SIGNATURE- iQIzBAABCgAdFiEEZQ5ukLH6mbYnhA++I6kA1BF8g14FAmBc8ysACgkQI6kA1BF8 g141vA/9E7jJbJD2Rx6in9RLgScftNDdoR32t86gpRlFgAwqTBmh/vLB/KMvSCaA +8/B4X2O7riG4dowx2SjbVPCngCPYT37XLobvqK6ZVKJ6In43eAyE/9l5J2BcZH/ 6hf4R5iTlgFAHSZNZP2pL2+XslhSGp//KGspase+xOOCSfbT8KtUjOr2PgZXwFkz 1epM5g7u95F7lHGvqj6xXclyb06glbO7fd/ZxCVL1sXz2KzGr0vy3lTx8CjdnoBv 7dp34+TSnpqeMO/CtZlENFufmQlNgE8KZHVKNcb6J6eKmhiKCNca1N5Y6IUi/zNk 4L6BKmTk06C8Iq8/I8ojJpT/JYD+6dlg77RP/Al33EDT0K28Trvw8etrt/legW3C atX6fIdkPq8nruHqoGJg2zhpDQMA8YJDanN6WHBO/tTDqX1omZV3i9EQpTq+5eoq MLl6R57MOw35+oz6hnBB+/pkNGhVJu/g33WiVyq9Cxn+XP/PAg3ODPQcArcd4Sfs N6MCJCy1Otn2KbN7tFJDzdfuSb63XnLGKA07Ka0x9IDt8Mg4Ov43CAqx0fVft9ly 6pJqrNSckYw3b1mZ5qS85fUVA7ftfxvQ283kn4xh38of6LgjpE9FkCsM6QBJVksT 4cM4POWCfQyenTyxVJlS42oBUUHDazzeefozDoKye1SXrvLCyQI= =2IvW -END PGP SIGNATURE- --- This annotated tag includes the following new commits: new fd3fdc0 Preparing hbase release 2.3.5RC1; tagging and updates to CHANGES.md and RELEASENOTES.md The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
svn commit: r46734 [1/4] - /dev/hbase/2.3.5RC0/
Author: huaxiangsun Date: Thu Mar 25 18:37:27 2021 New Revision: 46734 Log: Apache hbase 2.3.5RC0 Added: dev/hbase/2.3.5RC0/ dev/hbase/2.3.5RC0/CHANGES.md dev/hbase/2.3.5RC0/RELEASENOTES.md dev/hbase/2.3.5RC0/api_compare_2.3.4_to_2.3.5RC0.html dev/hbase/2.3.5RC0/hbase-2.3.5-bin.tar.gz (with props) dev/hbase/2.3.5RC0/hbase-2.3.5-bin.tar.gz.asc dev/hbase/2.3.5RC0/hbase-2.3.5-bin.tar.gz.sha512 dev/hbase/2.3.5RC0/hbase-2.3.5-client-bin.tar.gz (with props) dev/hbase/2.3.5RC0/hbase-2.3.5-client-bin.tar.gz.asc dev/hbase/2.3.5RC0/hbase-2.3.5-client-bin.tar.gz.sha512 dev/hbase/2.3.5RC0/hbase-2.3.5-src.tar.gz (with props) dev/hbase/2.3.5RC0/hbase-2.3.5-src.tar.gz.asc dev/hbase/2.3.5RC0/hbase-2.3.5-src.tar.gz.sha512
svn commit: r46734 [2/4] - /dev/hbase/2.3.5RC0/
Added: dev/hbase/2.3.5RC0/CHANGES.md == --- dev/hbase/2.3.5RC0/CHANGES.md (added) +++ dev/hbase/2.3.5RC0/CHANGES.md Thu Mar 25 18:37:27 2021 @@ -0,0 +1,8707 @@ +# HBASE Changelog + + +## Release 2.3.5 - Unreleased (as of 2021-03-25) + + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | +|: |: | :--- |: | +| [HBASE-25587](https://issues.apache.org/jira/browse/HBASE-25587) | [hbck2] Schedule SCP for all unknown servers | Major | hbase-operator-tools, hbck2 | +| [HBASE-25460](https://issues.apache.org/jira/browse/HBASE-25460) | Expose drainingServers as cluster metric | Major | metrics | +| [HBASE-25496](https://issues.apache.org/jira/browse/HBASE-25496) | add get\_namespace\_rsgroup command | Major | . | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | +|: |: | :--- |: | +| [HBASE-25621](https://issues.apache.org/jira/browse/HBASE-25621) | Balancer should check region plan source to avoid misplace region groups | Major | Balancer | +| [HBASE-25374](https://issues.apache.org/jira/browse/HBASE-25374) | Make REST Client connection and socket time out configurable | Minor | REST | +| [HBASE-25597](https://issues.apache.org/jira/browse/HBASE-25597) | Add row info in Exception when cell size exceeds maxCellSize | Minor | . | +| [HBASE-25635](https://issues.apache.org/jira/browse/HBASE-25635) | CandidateGenerator may miss some region balance actions | Major | Balancer | +| [HBASE-25566](https://issues.apache.org/jira/browse/HBASE-25566) | RoundRobinTableInputFormat | Major | mapreduce | +| [HBASE-25548](https://issues.apache.org/jira/browse/HBASE-25548) | Optionally allow snapshots to preserve cluster's max filesize config by setting it into table descriptor | Major | . | +| [HBASE-25582](https://issues.apache.org/jira/browse/HBASE-25582) | Support setting scan ReadType to be STREAM at cluster level | Major | . | +| [HBASE-23578](https://issues.apache.org/jira/browse/HBASE-23578) | [UI] Master UI shows long stack traces when table is broken | Minor | master, UI | +| [HBASE-25541](https://issues.apache.org/jira/browse/HBASE-25541) | In WALEntryStream, set the current path to null while dequeing the log | Major | . | +| [HBASE-25507](https://issues.apache.org/jira/browse/HBASE-25507) | Leak of ESTABLISHED sockets when compaction encountered "java.io.IOException: Invalid HFile block magic" | Major | Compaction | +| [HBASE-25528](https://issues.apache.org/jira/browse/HBASE-25528) | Dedicated merge dispatch threadpool on master | Minor | master | +| [HBASE-25536](https://issues.apache.org/jira/browse/HBASE-25536) | Remove 0 length wal file from logQueue if it belongs to old sources. | Major | Replication | +| [HBASE-25329](https://issues.apache.org/jira/browse/HBASE-25329) | Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time | Minor | . | +| [HBASE-25475](https://issues.apache.org/jira/browse/HBASE-25475) | Improve unit test for HBASE-25445 : SplitWALRemoteProcedure failed to archive split WAL | Minor | wal | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | +|: |: | :--- |: | +| [HBASE-25590](https://issues.apache.org/jira/browse/HBASE-25590) | Bulkload replication HFileRefs cannot be cleared in some cases where set exclude-namespace/exclude-table-cfs | Major | Replication | +| [HBASE-25562](https://issues.apache.org/jira/browse/HBASE-25562) | ReplicationSourceWALReader log and handle exception immediately without retrying | Major | Replication | +| [HBASE-25685](https://issues.apache.org/jira/browse/HBASE-25685) | asyncprofiler2.0 no longer supports svg; wants html | Major | . | +| [HBASE-25674](https://issues.apache.org/jira/browse/HBASE-25674) | RegionInfo.parseFrom(DataInputStream) sometimes fails to read the protobuf magic marker | Minor | Client | +| [HBASE-25639](https://issues.apache.org/jira/browse/HBASE-25639) | meta replica state is not respected during active master switch | Critical | meta replicas | +| [HBASE-25595](https://issues.apache.org/jira/browse/HBASE-25595) | TestLruBlockCache.testBackgroundEvictionThread is flaky | Major | . | +| [HBASE-25662](https://issues.apache.org/jira/browse/HBASE-25662) | Fix spotbugs warning in RoundRobinTableInputFormat | Major | findbugs | +| [HBASE-25657](https://issues.apache.org/jira/browse/HBASE-25657) | Fix spotbugs warnings after upgrading spotbugs to 4.x | Major | findbugs | +| [HBASE-25646](https://issues.apache.org/jira/browse/HBASE-25646) | Possible Resource Leak in CatalogJanitor | Major | master | +| [HBASE-25626](https://issues.apache.org/jira/browse/HBASE-25626) | Possible Resource Leak in HeterogeneousRegionCountCostFunction | Major | . | +| [HBASE-25644](https://issues.apache.org/jira/browse/HBASE-25644) | Scan#setSmall blindly sets ReadType as PREAD | Critical |
svn commit: r46734 [3/4] - /dev/hbase/2.3.5RC0/
Added: dev/hbase/2.3.5RC0/RELEASENOTES.md == --- dev/hbase/2.3.5RC0/RELEASENOTES.md (added) +++ dev/hbase/2.3.5RC0/RELEASENOTES.md Thu Mar 25 18:37:27 2021 @@ -0,0 +1,12517 @@ +# RELEASENOTES + + +# HBASE 2.3.5 Release Notes + +These release notes cover new developer and user-facing incompatibilities, important issues, features, and major improvements. + + +--- + +* [HBASE-25685](https://issues.apache.org/jira/browse/HBASE-25685) | *Major* | **asyncprofiler2.0 no longer supports svg; wants html** + +If asyncprofiler 1.x, all is good. If asyncprofiler 2.x and it is hbase-2.3.x or hbase-2.4.x, add '?output=html' to get flamegraphs from the profiler. + +Otherwise, if hbase-2.5+ and asyncprofiler2, all works. If asyncprofiler1 and hbase-2.5+, you may have to add '?output=svg' to the query. + + +--- + +* [HBASE-25374](https://issues.apache.org/jira/browse/HBASE-25374) | *Minor* | **Make REST Client connection and socket time out configurable** + +Configuration parameter to set rest client connection timeout + +"hbase.rest.client.conn.timeout" Default is 2 \* 1000 + +"hbase.rest.client.socket.timeout" Default of 30 \* 1000 + + +--- + +* [HBASE-25566](https://issues.apache.org/jira/browse/HBASE-25566) | *Major* | **RoundRobinTableInputFormat** + +Adds RoundRobinTableInputFormat, a subclass of TableInputFormat, that takes the TIF#getSplits list and resorts it so as to spread the InputFormats as broadly about the cluster as possible. RRTIF works to frustrate bunching of InputSplits on RegionServers to avoid the scenario where a few RegionServers are working hard fielding many InputSplits while others idle hosting a few or none. + + +--- + +* [HBASE-25587](https://issues.apache.org/jira/browse/HBASE-25587) | *Major* | **[hbck2] Schedule SCP for all unknown servers** + +Adds scheduleSCPsForUnknownServers to Hbck Service. + + +--- + +* [HBASE-25582](https://issues.apache.org/jira/browse/HBASE-25582) | *Major* | **Support setting scan ReadType to be STREAM at cluster level** + +Adding a new meaning for the config 'hbase.storescanner.pread.max.bytes' when configured with a value \<0. +In HBase 2.x we allow the Scan op to specify a ReadType (PREAD / STREAM/ DEFAULT). When Scan comes with DEFAULT read type, we will start scan with preads and later switch to stream read once we see we are scanning a total data size \> value of hbase.storescanner.pread.max.bytes. (This is calculated for data per region:cf). This config defaults to 4 x of HFile block size = 256 KB by default. +This jira added a new meaning for this config when configured with a -ve value. In such case, for all scans with DEFAULT read type, we will start with STREAM read itself. (Switch at begin of the scan itself) + + +--- + +* [HBASE-25460](https://issues.apache.org/jira/browse/HBASE-25460) | *Major* | **Expose drainingServers as cluster metric** + +Exposed new jmx metrics: "draininigRegionServers" and "numDrainingRegionServers" to provide "comma separated names for regionservers that are put in draining mode" and "num of such regionservers" respectively. + + +--- + +* [HBASE-25615](https://issues.apache.org/jira/browse/HBASE-25615) | *Major* | **Upgrade java version in pre commit docker file** + +jdk8u232-b09 -\> jdk8u282-b08 +jdk-11.0.6\_10 -\> jdk-11.0.10\_9 + + + +# HBASE 2.3.4 Release Notes + +These release notes cover new developer and user-facing incompatibilities, important issues, features, and major improvements. + + +--- + +* [HBASE-25449](https://issues.apache.org/jira/browse/HBASE-25449) | *Major* | **'dfs.client.read.shortcircuit' should not be set in hbase-default.xml** + +The presence of HDFS short-circuit read configuration properties in hbase-default.xml inadvertently causes short-circuit reads to not happen inside of RegionServers, despite short-circuit reads being enabled in hdfs-site.xml. + + +--- + +* [HBASE-25441](https://issues.apache.org/jira/browse/HBASE-25441) | *Critical* | **add security check for some APIs in RSRpcServices** + +RsRpcServices APIs that can be accessed only through Admin rights: +- stopServer +- updateFavoredNodes +- updateConfiguration +- clearRegionBlockCache +- clearSlowLogsResponses + + +--- + +* [HBASE-25432](https://issues.apache.org/jira/browse/HBASE-25432) | *Blocker* | **we should add security checks for setTableStateInMeta and fixMeta** + +setTableStateInMeta and fixMeta can be accessed only through Admin rights + + +--- + +* [HBASE-25318](https://issues.apache.org/jira/browse/HBASE-25318) | *Minor* | **Configure where IntegrationTestImportTsv generates HFiles** + +Added IntegrationTestImportTsv.generatedHFileFolder configuration property to override the default location in IntegrationTestImportTsv. Useful for running the integration test when HDFS Transparent Encryption is enabled. + + +--- + +* [HBASE-25456](https://issues.apache.org/jira/browse/HBASE-25456) | *Critical* |
svn commit: r46734 [4/4] - /dev/hbase/2.3.5RC0/
Added: dev/hbase/2.3.5RC0/api_compare_2.3.4_to_2.3.5RC0.html
==
--- dev/hbase/2.3.5RC0/api_compare_2.3.4_to_2.3.5RC0.html (added)
+++ dev/hbase/2.3.5RC0/api_compare_2.3.4_to_2.3.5RC0.html Thu Mar 25 18:37:27
2021
@@ -0,0 +1,637 @@
+
+
+http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd;>
+http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
+
+
+
+
+
+hbase: rel/2.3.4 to 2.3.5RC0 compatibility report
+
+body {
+font-family:Arial, sans-serif;
+background-color:White;
+color:Black;
+}
+hr {
+color:Black;
+background-color:Black;
+height:1px;
+border:0;
+}
+h1 {
+margin-bottom:0px;
+padding-bottom:0px;
+font-size:1.625em;
+}
+h2 {
+margin-bottom:0px;
+padding-bottom:0px;
+font-size:1.25em;
+white-space:nowrap;
+}
+div.symbols {
+color:#003E69;
+}
+div.symbols i {
+color:Brown;
+}
+span.section {
+font-weight:bold;
+cursor:pointer;
+color:#003E69;
+white-space:nowrap;
+margin-left:0.3125em;
+}
+span:hover.section {
+color:#336699;
+}
+span.sect_aff {
+cursor:pointer;
+padding-left:1.55em;
+font-size:0.875em;
+color:#cc3300;
+}
+span.ext {
+font-weight:normal;
+}
+span.jar {
+color:#cc3300;
+font-size:0.875em;
+font-weight:bold;
+}
+div.jar_list {
+padding-left:0.4em;
+font-size:0.94em;
+}
+span.pkg_t {
+color:#408080;
+font-size:0.875em;
+}
+span.pkg {
+color:#408080;
+font-size:0.875em;
+font-weight:bold;
+}
+span.cname {
+color:Green;
+font-size:0.875em;
+font-weight:bold;
+}
+span.iname_b {
+font-weight:bold;
+}
+span.iname_a {
+color:#33;
+font-weight:bold;
+font-size:0.94em;
+}
+span.sym_p {
+font-weight:normal;
+white-space:normal;
+}
+span.sym_pd {
+white-space:normal;
+}
+span.sym_p span, span.sym_pd span {
+white-space:nowrap;
+}
+span.attr {
+color:Black;
+font-weight:normal;
+}
+span.deprecated {
+color:Red;
+font-weight:bold;
+font-family:Monaco, monospace;
+}
+div.affect {
+padding-left:1em;
+padding-bottom:10px;
+font-size:0.87em;
+font-style:italic;
+line-height:0.9em;
+}
+div.affected {
+padding-left:2em;
+padding-top:10px;
+}
+table.ptable {
+border-collapse:collapse;
+border:1px outset black;
+margin-left:0.95em;
+margin-top:3px;
+margin-bottom:3px;
+width:56.25em;
+}
+table.ptable td {
+border:1px solid Gray;
+padding:3px;
+font-size:0.875em;
+text-align:left;
+vertical-align:top;
+max-width:28em;
+word-wrap:break-word;
+}
+table.ptable th {
+background-color:#ee;
+font-weight:bold;
+color:#33;
+font-family:Verdana, Arial;
+font-size:0.875em;
+border:1px solid Gray;
+text-align:center;
+vertical-align:top;
+white-space:nowrap;
+padding:3px;
+}
+table.summary {
+border-collapse:collapse;
+border:1px outset black;
+}
+table.summary th {
+background-color:#ee;
+font-weight:normal;
+text-align:left;
+font-size:0.94em;
+white-space:nowrap;
+border:1px inset Gray;
+padding:3px;
+}
+table.summary td {
+text-align:right;
+white-space:nowrap;
+border:1px inset Gray;
+padding:3px 5px 3px 10px;
+}
+span.mngl {
+padding-left:1em;
+font-size:0.875em;
+cursor:text;
+color:#44;
+font-weight:bold;
+}
+span.pleft {
+padding-left:2.5em;
+}
+span.color_p {
+font-style:italic;
+color:Brown;
+}
+span.param {
+font-style:italic;
+}
+span.focus_p {
+font-style:italic;
+background-color:#DCDCDC;
+}
+span.ttype {
+font-weight:normal;
+}
+span.nowrap {
+white-space:nowrap;
+}
+span.value {
+white-space:nowrap;
+font-weight:bold;
+}
+.passed {
+background-color:#CCFFCC;
+font-weight:normal;
+}
+.warning {
+background-color:#F4F4AF;
+font-weight:normal;
+}
+.failed {
+background-color:#FF;
+font-weight:normal;
+}
+.new {
+background-color:#C6DEFF;
+font-weight:normal;
+}
+
+.compatible {
+background-color:#CCFFCC;
+font-weight:normal;
+}
+.almost_compatible {
+background-color:#FFDAA3;
+font-weight:normal;
+}
+.incompatible {
+background-color:#FF;
+font-weight:normal;
+}
+.gray {
+background-color:#DCDCDC;
+font-weight:normal;
+}
+
+.top_ref {
+font-size:0.69em;
+}
+.footer {
+font-size:0.8125em;
+}
+.tabset {
+float:left;
+}
+a.tab {
+border:1px solid Black;
+float:left;
+margin:0px 5px -1px 0px;
+padding:3px 5px 3px 5px;
+position:relative;
+font-size:0.875em;
+background-color:#DDD;
+text-decoration:none;
+color:Black;
+}
+a.disabled:hover
+{
+color:Black;
+background:#EEE;
+}
+a.active:hover
+{
+color:Black;
+background:White;
+}
+a.active {
+border-bottom-color:White;
+background-color:White;
+}
+div.tab {
+border-top:1px solid
[hbase] 01/01: Preparing development version 2.3.6-SNAPSHOT
This is an automated email from the ASF dual-hosted git repository. huaxiangsun pushed a commit to branch branch-2.3 in repository https://gitbox.apache.org/repos/asf/hbase.git commit ebcceb92625621bc96881c0b4159f39a45a08248 Author: Huaxiang Sun AuthorDate: Thu Mar 25 15:51:09 2021 + Preparing development version 2.3.6-SNAPSHOT Signed-off-by: Huaxiang Sun --- hbase-annotations/pom.xml | 2 +- hbase-archetypes/hbase-archetype-builder/pom.xml | 2 +- hbase-archetypes/hbase-client-project/pom.xml | 2 +- hbase-archetypes/hbase-shaded-client-project/pom.xml | 2 +- hbase-archetypes/pom.xml | 2 +- hbase-assembly/pom.xml | 2 +- hbase-asyncfs/pom.xml | 2 +- hbase-build-configuration/pom.xml | 2 +- hbase-checkstyle/pom.xml | 4 ++-- hbase-client/pom.xml | 2 +- hbase-common/pom.xml | 2 +- hbase-endpoint/pom.xml | 2 +- hbase-examples/pom.xml | 2 +- hbase-external-blockcache/pom.xml | 2 +- hbase-hadoop-compat/pom.xml| 2 +- hbase-hadoop2-compat/pom.xml | 2 +- hbase-hbtop/pom.xml| 2 +- hbase-http/pom.xml | 2 +- hbase-it/pom.xml | 2 +- hbase-logging/pom.xml | 2 +- hbase-mapreduce/pom.xml| 2 +- hbase-metrics-api/pom.xml | 2 +- hbase-metrics/pom.xml | 2 +- hbase-procedure/pom.xml| 2 +- hbase-protocol-shaded/pom.xml | 2 +- hbase-protocol/pom.xml | 2 +- hbase-replication/pom.xml | 2 +- hbase-resource-bundle/pom.xml | 2 +- hbase-rest/pom.xml | 2 +- hbase-rsgroup/pom.xml | 2 +- hbase-server/pom.xml | 2 +- hbase-shaded/hbase-shaded-check-invariants/pom.xml | 2 +- hbase-shaded/hbase-shaded-client-byo-hadoop/pom.xml| 2 +- hbase-shaded/hbase-shaded-client/pom.xml | 2 +- hbase-shaded/hbase-shaded-mapreduce/pom.xml| 2 +- hbase-shaded/hbase-shaded-testing-util-tester/pom.xml | 2 +- hbase-shaded/hbase-shaded-testing-util/pom.xml | 2 +- hbase-shaded/hbase-shaded-with-hadoop-check-invariants/pom.xml | 2 +- hbase-shaded/pom.xml | 2 +- hbase-shell/pom.xml| 2 +- hbase-testing-util/pom.xml | 2 +- hbase-thrift/pom.xml | 2 +- hbase-zookeeper/pom.xml| 2 +- pom.xml| 2 +- 44 files changed, 45 insertions(+), 45 deletions(-) diff --git a/hbase-annotations/pom.xml b/hbase-annotations/pom.xml index c5a14d3..46d85d7 100644 --- a/hbase-annotations/pom.xml +++ b/hbase-annotations/pom.xml @@ -23,7 +23,7 @@ hbase org.apache.hbase -2.3.5 +2.3.6-SNAPSHOT .. diff --git a/hbase-archetypes/hbase-archetype-builder/pom.xml b/hbase-archetypes/hbase-archetype-builder/pom.xml index d1df340..31f1d7c 100644 --- a/hbase-archetypes/hbase-archetype-builder/pom.xml +++ b/hbase-archetypes/hbase-archetype-builder/pom.xml @@ -25,7 +25,7 @@ hbase-archetypes org.apache.hbase -2.3.5 +2.3.6-SNAPSHOT .. diff --git a/hbase-archetypes/hbase-client-project/pom.xml b/hbase-archetypes/hbase-client-project/pom.xml index 2a64ff9..6618cc4 100644 --- a/hbase-archetypes/hbase-client-project/pom.xml +++ b/hbase-archetypes/hbase-client-project/pom.xml @@ -26,7 +26,7 @@ hbase-archetypes org.apache.hbase -2.3.5 +2.3.6-SNAPSHOT .. hbase-client-project diff --git a/hbase-archetypes/hbase-shaded-client-project/pom.xml b/hbase-archetypes/hbase-shaded-client-project/pom.xml index 1d30621..661b2c0 100644 --- a/hbase-archetypes/hbase-shaded-client-project/pom.xml +++ b/hbase-archetypes/hbase-shaded-client-project/pom.xml @@ -26,7 +26,7 @@ hbase-archetypes org.apache.hbase -2.3.5 +2.3.6-SNAPSHOT .. hbase-shaded-client-project diff --git a/hbase
[hbase] branch branch-2.3 updated (754caba -> ebcceb9)
This is an automated email from the ASF dual-hosted git repository. huaxiangsun pushed a change to branch branch-2.3 in repository https://gitbox.apache.org/repos/asf/hbase.git. from 754caba HBASE-25562 ReplicationSourceWALReader log and handle exception immediately without retrying (#2966) add 35146fd Preparing hbase release 2.3.5RC0; tagging and updates to CHANGES.md and RELEASENOTES.md new ebcceb9 Preparing development version 2.3.6-SNAPSHOT The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.md | 91 ++ RELEASENOTES.md| 64 +++ hbase-annotations/pom.xml | 2 +- hbase-archetypes/hbase-archetype-builder/pom.xml | 2 +- hbase-archetypes/hbase-client-project/pom.xml | 2 +- .../hbase-shaded-client-project/pom.xml| 2 +- hbase-archetypes/pom.xml | 2 +- hbase-assembly/pom.xml | 2 +- hbase-asyncfs/pom.xml | 2 +- hbase-build-configuration/pom.xml | 2 +- hbase-checkstyle/pom.xml | 4 +- hbase-client/pom.xml | 2 +- hbase-common/pom.xml | 2 +- hbase-endpoint/pom.xml | 2 +- hbase-examples/pom.xml | 2 +- hbase-external-blockcache/pom.xml | 2 +- hbase-hadoop-compat/pom.xml| 2 +- hbase-hadoop2-compat/pom.xml | 2 +- hbase-hbtop/pom.xml| 2 +- hbase-http/pom.xml | 2 +- hbase-it/pom.xml | 2 +- hbase-logging/pom.xml | 2 +- hbase-mapreduce/pom.xml| 2 +- hbase-metrics-api/pom.xml | 2 +- hbase-metrics/pom.xml | 2 +- hbase-procedure/pom.xml| 2 +- hbase-protocol-shaded/pom.xml | 2 +- hbase-protocol/pom.xml | 2 +- hbase-replication/pom.xml | 2 +- hbase-resource-bundle/pom.xml | 2 +- hbase-rest/pom.xml | 2 +- hbase-rsgroup/pom.xml | 2 +- hbase-server/pom.xml | 2 +- hbase-shaded/hbase-shaded-check-invariants/pom.xml | 2 +- .../hbase-shaded-client-byo-hadoop/pom.xml | 2 +- hbase-shaded/hbase-shaded-client/pom.xml | 2 +- hbase-shaded/hbase-shaded-mapreduce/pom.xml| 2 +- .../hbase-shaded-testing-util-tester/pom.xml | 2 +- hbase-shaded/hbase-shaded-testing-util/pom.xml | 2 +- .../pom.xml| 2 +- hbase-shaded/pom.xml | 2 +- hbase-shell/pom.xml| 2 +- hbase-testing-util/pom.xml | 2 +- hbase-thrift/pom.xml | 2 +- hbase-zookeeper/pom.xml| 2 +- pom.xml| 2 +- 46 files changed, 200 insertions(+), 45 deletions(-)
[hbase] 01/01: Preparing hbase release 2.3.5RC0; tagging and updates to CHANGES.md and RELEASENOTES.md
This is an automated email from the ASF dual-hosted git repository. huaxiangsun pushed a commit to annotated tag 2.3.5RC0 in repository https://gitbox.apache.org/repos/asf/hbase.git commit 35146fd84cde5c83ba3d8b9a19a81db85b678e8e Author: Huaxiang Sun AuthorDate: Thu Mar 25 15:50:58 2021 + Preparing hbase release 2.3.5RC0; tagging and updates to CHANGES.md and RELEASENOTES.md Signed-off-by: Huaxiang Sun --- CHANGES.md | 91 ++ RELEASENOTES.md| 64 +++ hbase-annotations/pom.xml | 2 +- hbase-archetypes/hbase-archetype-builder/pom.xml | 2 +- hbase-archetypes/hbase-client-project/pom.xml | 2 +- .../hbase-shaded-client-project/pom.xml| 2 +- hbase-archetypes/pom.xml | 2 +- hbase-assembly/pom.xml | 2 +- hbase-asyncfs/pom.xml | 2 +- hbase-build-configuration/pom.xml | 2 +- hbase-checkstyle/pom.xml | 4 +- hbase-client/pom.xml | 2 +- hbase-common/pom.xml | 2 +- hbase-endpoint/pom.xml | 2 +- hbase-examples/pom.xml | 2 +- hbase-external-blockcache/pom.xml | 2 +- hbase-hadoop-compat/pom.xml| 2 +- hbase-hadoop2-compat/pom.xml | 2 +- hbase-hbtop/pom.xml| 2 +- hbase-http/pom.xml | 2 +- hbase-it/pom.xml | 2 +- hbase-logging/pom.xml | 2 +- hbase-mapreduce/pom.xml| 2 +- hbase-metrics-api/pom.xml | 2 +- hbase-metrics/pom.xml | 2 +- hbase-procedure/pom.xml| 2 +- hbase-protocol-shaded/pom.xml | 2 +- hbase-protocol/pom.xml | 2 +- hbase-replication/pom.xml | 2 +- hbase-resource-bundle/pom.xml | 2 +- hbase-rest/pom.xml | 2 +- hbase-rsgroup/pom.xml | 2 +- hbase-server/pom.xml | 2 +- hbase-shaded/hbase-shaded-check-invariants/pom.xml | 2 +- .../hbase-shaded-client-byo-hadoop/pom.xml | 2 +- hbase-shaded/hbase-shaded-client/pom.xml | 2 +- hbase-shaded/hbase-shaded-mapreduce/pom.xml| 2 +- .../hbase-shaded-testing-util-tester/pom.xml | 2 +- hbase-shaded/hbase-shaded-testing-util/pom.xml | 2 +- .../pom.xml| 2 +- hbase-shaded/pom.xml | 2 +- hbase-shell/pom.xml| 2 +- hbase-testing-util/pom.xml | 2 +- hbase-thrift/pom.xml | 2 +- hbase-zookeeper/pom.xml| 2 +- pom.xml| 2 +- 46 files changed, 200 insertions(+), 45 deletions(-) diff --git a/CHANGES.md b/CHANGES.md index 562b5b6..8da9214 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -20,6 +20,97 @@ # Be careful doing manual edits in this file. Do not change format # of release header or remove the below marker. This file is generated. # DO NOT REMOVE THIS MARKER; FOR INTERPOLATING CHANGES!--> +## Release 2.3.5 - Unreleased (as of 2021-03-25) + + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | +|: |: | :--- |: | +| [HBASE-25587](https://issues.apache.org/jira/browse/HBASE-25587) | [hbck2] Schedule SCP for all unknown servers | Major | hbase-operator-tools, hbck2 | +| [HBASE-25460](https://issues.apache.org/jira/browse/HBASE-25460) | Expose drainingServers as cluster metric | Major | metrics | +| [HBASE-25496](https://issues.apache.org/jira/browse/HBASE-25496) | add get\_namespace\_rsgroup command | Major | . | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | +|: |: | :--- |: | +| [HBASE-25621](https://issues.apache.org/jira/browse/HBASE-25621) | Balancer should check region plan source to avoid misplace region groups | Major | Balancer | +| [HBASE-25374](https://issues.apache.org/jira/browse/HBASE-25374) | Make REST Client connection and socket time out configurable | Minor | REST | +| [HBASE-25597](https://issues.apache.org/jira/browse/HBASE-25597) | Add row info in Exception when cell size exceeds maxCellSize | Minor | . | +| [HBASE-25635](https://issues.apache.org/jira/browse/HBASE-25635) | CandidateGenerator may miss some region balance actions | Major | Balancer | +| [HBASE-25566](https://issues.apache.org/jira/browse/HBASE-25566) | RoundRobinTableInputFormat | Major | mapred
[hbase] annotated tag 2.3.5RC0 created (now a58a6fd)
This is an automated email from the ASF dual-hosted git repository. huaxiangsun pushed a change to annotated tag 2.3.5RC0 in repository https://gitbox.apache.org/repos/asf/hbase.git. at a58a6fd (tag) tagging 35146fd84cde5c83ba3d8b9a19a81db85b678e8e (commit) replaces rel/2.3.4 by Huaxiang Sun on Thu Mar 25 15:50:58 2021 + - Log - Via create-release -BEGIN PGP SIGNATURE- iQIzBAABCgAdFiEEZQ5ukLH6mbYnhA++I6kA1BF8g14FAmBcsWIACgkQI6kA1BF8 g17mKQ/9GyF79C8b/uYOFbJsVtnuTU61qoY2lDfNY3uzPFgqIwh21pCjRZ3P4AHw Kee1etX3JuZjmPfYzL5qnmelv33hxSFK7qVKIlprZi4e8E+7Cxw8pnhbhMkkfWuS /GFB5N3t6GaF2G+5iJMb63NVK8q5n4zIDJkqsp1R8SVbPHsyxMhBrwVSD6h/bX15 WGsDP//SND6AllW4rtjyrAcVRCKGFMVUqNukNLRooacjVjR2SxKvTueCDlmD9tIh aFrPNkzKQiufhoFHdO7QHJMQgEVCTLknuoak5ciJi/hswNBiHBAPrvFHMr26F2Tg bfFkdkkj8TYld/JKOqXUcyqVCg/PPvirS6uw55LPjy8NtmnJ2rqU+dqEQCgXrGL2 hxKgoVtisBM94jhm5qYi+F761FvmEkUJsCAJGS1XgiH6GTKVgFgrzaEWyXlEGIBy trsoFBce9bzT8h49n+iWPF+xq3U50CfrkeLsQ6iT4N9hqkcKOQYYqfTHauZNRq+A EP+SMCOwhZrFtq0Uw2t9jB7Jm+2HxewkUUtUNh8hCcPFgw3YeFSwoEMVOhLO7A51 ZXOvscIXMmH/73UHh6+y54lZ81b4kKE5/4TKb9eliNjwtiy1G84C4KlUfrRVcUtL 1aHsyxpH1YKDqWjM7IgnpxMO/4+E0Mxu4dHJzkdV4T4yW+cbqzQ= =Z4Z8 -END PGP SIGNATURE- --- This annotated tag includes the following new commits: new 35146fd Preparing hbase release 2.3.5RC0; tagging and updates to CHANGES.md and RELEASENOTES.md The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[hbase] branch branch-2.3 updated: HBASE-25691 Test failure: TestVerifyBucketCacheFile.testRetrieveFromFile (#3089)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch branch-2.3
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2.3 by this push:
new 8710772 HBASE-25691 Test failure:
TestVerifyBucketCacheFile.testRetrieveFromFile (#3089)
8710772 is described below
commit 871077241b24a01b6cfcaec81e7981525792dacf
Author: huaxiangsun
AuthorDate: Wed Mar 24 14:41:11 2021 -0700
HBASE-25691 Test failure: TestVerifyBucketCacheFile.testRetrieveFromFile
(#3089)
The issue is that FileInputStream is created with try-with-resources, so
its close() is called right after the try sentence.
FileInputStream is a finalize class, when this object is garbage collected,
its close() is called again.
To avoid this double-free resources, add guard against it.
Signed-off-by: stack
---
.../hadoop/hbase/io/hfile/bucket/BucketCache.java | 19 ---
1 file changed, 16 insertions(+), 3 deletions(-)
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
index fca1ffa..8e5f558 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
@@ -1118,14 +1118,27 @@ public class BucketCache implements BlockCache,
HeapSize {
*/
private FileInputStream deleteFileOnClose(final File file) throws
IOException {
return new FileInputStream(file) {
+ private File myFile;
+ private FileInputStream init(File file) {
+myFile = file;
+return this;
+ }
@Override
public void close() throws IOException {
+// close() will be called during try-with-resources and it will be
+// called by finalizer thread during GC. To avoid double-free resource,
+// set myFile to null after the first call.
+if (myFile == null) {
+ return;
+}
+
super.close();
-if (!file.delete()) {
- throw new IOException("Failed deleting persistence file " +
file.getAbsolutePath());
+if (!myFile.delete()) {
+ throw new IOException("Failed deleting persistence file " +
myFile.getAbsolutePath());
}
+myFile = null;
}
-};
+}.init(file);
}
private void verifyCapacityAndClasses(long capacitySize, String ioclass,
String mapclass)
[hbase] branch branch-2.4 updated: HBASE-25691 Test failure: TestVerifyBucketCacheFile.testRetrieveFromFile (#3088)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2.4 by this push:
new 91205bf HBASE-25691 Test failure:
TestVerifyBucketCacheFile.testRetrieveFromFile (#3088)
91205bf is described below
commit 91205bf227e8d5df21e34cd257514886f9a18365
Author: huaxiangsun
AuthorDate: Wed Mar 24 14:39:35 2021 -0700
HBASE-25691 Test failure: TestVerifyBucketCacheFile.testRetrieveFromFile
(#3088)
The issue is that FileInputStream is created with try-with-resources, so
its close() is called right after the try sentence.
FileInputStream is a finalize class, when this object is garbage collected,
its close() is called again.
To avoid this double-free resources, add guard against it.
Signed-off-by: stack
---
.../hadoop/hbase/io/hfile/bucket/BucketCache.java | 19 ---
1 file changed, 16 insertions(+), 3 deletions(-)
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
index de44ad7..1f9aac4 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
@@ -1101,14 +1101,27 @@ public class BucketCache implements BlockCache,
HeapSize {
*/
private FileInputStream deleteFileOnClose(final File file) throws
IOException {
return new FileInputStream(file) {
+ private File myFile;
+ private FileInputStream init(File file) {
+myFile = file;
+return this;
+ }
@Override
public void close() throws IOException {
+// close() will be called during try-with-resources and it will be
+// called by finalizer thread during GC. To avoid double-free resource,
+// set myFile to null after the first call.
+if (myFile == null) {
+ return;
+}
+
super.close();
-if (!file.delete()) {
- throw new IOException("Failed deleting persistence file " +
file.getAbsolutePath());
+if (!myFile.delete()) {
+ throw new IOException("Failed deleting persistence file " +
myFile.getAbsolutePath());
}
+myFile = null;
}
-};
+}.init(file);
}
private void verifyCapacityAndClasses(long capacitySize, String ioclass,
String mapclass)
[hbase] branch branch-2 updated (c8c043a -> 80529af)
This is an automated email from the ASF dual-hosted git repository. huaxiangsun pushed a change to branch branch-2 in repository https://gitbox.apache.org/repos/asf/hbase.git. from c8c043a HBASE-25685 asyncprofiler2.0 no longer supports svg; wants html (#3079) add 80529af HBASE-25691 Test failure: TestVerifyBucketCacheFile.testRetrieveFromFile (#3087) No new revisions were added by this update. Summary of changes: .../hadoop/hbase/io/hfile/bucket/BucketCache.java | 19 --- 1 file changed, 16 insertions(+), 3 deletions(-)
[hbase] branch master updated: HBASE-25691 Test failure: TestVerifyBucketCacheFile.testRetrieveFromFile (#3081)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/master by this push:
new 1e3fe3c HBASE-25691 Test failure:
TestVerifyBucketCacheFile.testRetrieveFromFile (#3081)
1e3fe3c is described below
commit 1e3fe3ceac323925d895d5e24b323bd056fc4b8e
Author: huaxiangsun
AuthorDate: Wed Mar 24 09:01:17 2021 -0700
HBASE-25691 Test failure: TestVerifyBucketCacheFile.testRetrieveFromFile
(#3081)
The issue is that FileInputStream is created with try-with-resources, so
its close() is called right after the try sentence.
FileInputStream is a finalize class, when this object is garbage collected,
its close() is called again.
To avoid this double-free resources, add guard against it.
Signed-off-by: stack
---
.../hadoop/hbase/io/hfile/bucket/BucketCache.java | 19 ---
1 file changed, 16 insertions(+), 3 deletions(-)
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
index a84d812..0187f5e 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
@@ -1112,14 +1112,27 @@ public class BucketCache implements BlockCache,
HeapSize {
*/
private FileInputStream deleteFileOnClose(final File file) throws
IOException {
return new FileInputStream(file) {
+ private File myFile;
+ private FileInputStream init(File file) {
+myFile = file;
+return this;
+ }
@Override
public void close() throws IOException {
+// close() will be called during try-with-resources and it will be
+// called by finalizer thread during GC. To avoid double-free resource,
+// set myFile to null after the first call.
+if (myFile == null) {
+ return;
+}
+
super.close();
-if (!file.delete()) {
- throw new IOException("Failed deleting persistence file " +
file.getAbsolutePath());
+if (!myFile.delete()) {
+ throw new IOException("Failed deleting persistence file " +
myFile.getAbsolutePath());
}
+myFile = null;
}
-};
+}.init(file);
}
private void verifyCapacityAndClasses(long capacitySize, String ioclass,
String mapclass)
[hbase] branch branch-2.3 updated: HBASE-25639 meta replica state is not respected during active master switch (#3052)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch branch-2.3
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2.3 by this push:
new 0a4ddd6 HBASE-25639 meta replica state is not respected during active
master switch (#3052)
0a4ddd6 is described below
commit 0a4ddd6c3bcf50ce7f3563e1699145d38cdf9de0
Author: huaxiangsun
AuthorDate: Thu Mar 18 11:35:46 2021 -0700
HBASE-25639 meta replica state is not respected during active master switch
(#3052)
Signed-off-by: stack
---
.../org/apache/hadoop/hbase/master/HMaster.java| 2 +-
.../hadoop/hbase/master/MasterMetaBootstrap.java | 27 +++---
.../hbase/master/assignment/AssignmentManager.java | 36 +---
.../master/TestMasterFailoverWithMetaReplica.java | 102 +
4 files changed, 138 insertions(+), 29 deletions(-)
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
index b78f7f3..5f8b63d 100644
--- a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
+++ b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
@@ -1018,7 +1018,7 @@ public class HMaster extends HRegionServer implements
MasterServices {
RegionState rs = this.assignmentManager.getRegionStates().
getRegionState(RegionInfoBuilder.FIRST_META_REGIONINFO);
LOG.info("hbase:meta {}", rs);
-if (rs != null && rs.isOffline()) {
+if ((rs == null) || (rs != null && rs.isOffline())) {
Optional optProc =
procedureExecutor.getProcedures().stream()
.filter(p -> p instanceof InitMetaProcedure).map(o ->
(InitMetaProcedure) o).findAny();
initMetaProc = optProc.orElseGet(() -> {
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterMetaBootstrap.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterMetaBootstrap.java
index da8d228..800ae97 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterMetaBootstrap.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterMetaBootstrap.java
@@ -21,7 +21,6 @@ package org.apache.hadoop.hbase.master;
import java.io.IOException;
import java.util.List;
import org.apache.hadoop.hbase.HConstants;
-import org.apache.hadoop.hbase.ServerName;
import org.apache.hadoop.hbase.client.RegionInfo;
import org.apache.hadoop.hbase.client.RegionInfoBuilder;
import org.apache.hadoop.hbase.client.RegionReplicaUtil;
@@ -61,25 +60,25 @@ class MasterMetaBootstrap {
throw new IllegalStateException("hbase:meta must be initialized first
before we can " +
"assign out its replicas");
}
-ServerName metaServername =
MetaTableLocator.getMetaRegionLocation(this.master.getZooKeeper());
+
for (int i = 1; i < numReplicas; i++) {
// Get current meta state for replica from zk.
- RegionState metaState =
MetaTableLocator.getMetaRegionState(master.getZooKeeper(), i);
RegionInfo hri = RegionReplicaUtil.getRegionInfoForReplica(
- RegionInfoBuilder.FIRST_META_REGIONINFO, i);
- LOG.debug(hri.getRegionNameAsString() + " replica region state from
zookeeper=" + metaState);
- if (metaServername.equals(metaState.getServerName())) {
-metaState = null;
-LOG.info(hri.getRegionNameAsString() +
- " old location is same as current hbase:meta location; setting
location as null...");
- }
+RegionInfoBuilder.FIRST_META_REGIONINFO, i);
+
+ RegionState rs = assignmentManager.getRegionStates().getRegionState(hri);
+ LOG.debug(hri.getRegionNameAsString() + " replica region state from
zookeeper=" + rs);
+
// These assigns run inline. All is blocked till they complete. Only
interrupt is shutting
// down hosting server which calls AM#stop.
- if (metaState != null && metaState.getServerName() != null) {
-// Try to retain old assignment.
-assignmentManager.assignAsync(hri, metaState.getServerName());
- } else {
+ if (rs == null) {
assignmentManager.assignAsync(hri);
+ } else if (rs != null && rs.isOffline()) {
+if (rs.getServerName() != null) {
+ assignmentManager.assignAsync(hri, rs.getServerName());
+} else {
+ assignmentManager.assignAsync(hri);
+}
}
}
unassignExcessMetaReplica(numReplicas);
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
index edc7dee..880de2a 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
+++
b/hbase-server
[hbase] branch master updated: HBASE-25416 Add 2.3.4 to the downloads page (#2902)
This is an automated email from the ASF dual-hosted git repository. huaxiangsun pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hbase.git The following commit(s) were added to refs/heads/master by this push: new f39636c HBASE-25416 Add 2.3.4 to the downloads page (#2902) f39636c is described below commit f39636c05e057edbbc7b81268599c305dcee496a Author: huaxiangsun AuthorDate: Fri Jan 22 19:25:06 2021 -0800 HBASE-25416 Add 2.3.4 to the downloads page (#2902) Signed-off-by: Nick Dimiduk --- src/site/xdoc/downloads.xml | 19 +-- 1 file changed, 9 insertions(+), 10 deletions(-) diff --git a/src/site/xdoc/downloads.xml b/src/site/xdoc/downloads.xml index fe6f3d8..a4123ba 100644 --- a/src/site/xdoc/downloads.xml +++ b/src/site/xdoc/downloads.xml @@ -68,26 +68,26 @@ under the License. -2.3.3 +2.3.4 -2020/11/02 +2021/01/22 -https://downloads.apache.org/hbase/2.3.3/api_compare_2.3.2_to_2.3.3RC1.html;>2.3.2 vs 2.3.3 +https://downloads.apache.org/hbase/2.3.4/api_compare_2.3.3_to_2.3.4RC4.html;>2.3.3 vs 2.3.4 -https://downloads.apache.org/hbase/2.3.3/CHANGES.md;>Changes +https://downloads.apache.org/hbase/2.3.4/CHANGES.md;>Changes -https://downloads.apache.org/hbase/2.3.3/RELEASENOTES.md;>Release Notes +https://downloads.apache.org/hbase/2.3.4/RELEASENOTES.md;>Release Notes -https://www.apache.org/dyn/closer.lua/hbase/2.3.3/hbase-2.3.3-src.tar.gz;>src (https://downloads.apache.org/hbase/2.3.3/hbase-2.3.3-src.tar.gz.sha512;>sha512 https://downloads.apache.org/hbase/2.3.3/hbase-2.3.3-src.tar.gz.asc;>asc) -https://www.apache.org/dyn/closer.lua/hbase/2.3.3/hbase-2.3.3-bin.tar.gz;>bin (https://downloads.apache.org/hbase/2.3.3/hbase-2.3.3-bin.tar.gz.sha512;>sha512 https://downloads.apache.org/hbase/2.3.3/hbase-2.3.3-bin.tar.gz.asc;>asc) -https://www.apache.org/dyn/closer.lua/hbase/2.3.3/hbase-2.3.3-client-bin.tar.gz;>client-bin (https://downloads.apache.org/hbase/2.3.3/hbase-2.3.3-client-bin.tar.gz.sha512;>sha512 https://downloads.apache.org/hbase/2.3.3/hbase-2.3.3-client-bin.tar.gz.asc;>asc) +https://www.apache.org/dyn/closer.lua/hbase/2.3.4/hbase-2.3.4-src.tar.gz;>src (https://downloads.apache.org/hbase/2.3.4/hbase-2.3.4-src.tar.gz.sha512;>sha512 https://downloads.apache.org/hbase/2.3.4/hbase-2.3.4-src.tar.gz.asc;>asc) +https://www.apache.org/dyn/closer.lua/hbase/2.3.4/hbase-2.3.4-bin.tar.gz;>bin (https://downloads.apache.org/hbase/2.3.4/hbase-2.3.4-bin.tar.gz.sha512;>sha512 https://downloads.apache.org/hbase/2.3.4/hbase-2.3.4-bin.tar.gz.asc;>asc) +https://www.apache.org/dyn/closer.lua/hbase/2.3.4/hbase-2.3.4-client-bin.tar.gz;>client-bin (https://downloads.apache.org/hbase/2.3.4/hbase-2.3.4-client-bin.tar.gz.sha512;>sha512 https://downloads.apache.org/hbase/2.3.4/hbase-2.3.4-client-bin.tar.gz.asc;>asc) - + stable release @@ -110,7 +110,6 @@ under the License. https://www.apache.org/dyn/closer.lua/hbase/2.2.6/hbase-2.2.6-bin.tar.gz;>bin (https://apache.org/dist/hbase/2.2.6/hbase-2.2.6-bin.tar.gz.sha512;>sha512 https://apache.org/dist/hbase/2.2.6/hbase-2.2.6-bin.tar.gz.asc;>asc) https://www.apache.org/dyn/closer.lua/hbase/2.2.6/hbase-2.2.6-client-bin.tar.gz;>client-bin (https://apache.org/dist/hbase/2.2.6/hbase-2.2.6-client-bin.tar.gz.sha512;>sha512 https://apache.org/dist/hbase/2.2.6/hbase-2.2.6-client-bin.tar.gz.asc;>asc) - stable release
[hbase] annotated tag rel/2.3.4 created (now 205e69e)
This is an automated email from the ASF dual-hosted git repository. huaxiangsun pushed a change to annotated tag rel/2.3.4 in repository https://gitbox.apache.org/repos/asf/hbase.git. at 205e69e (tag) tagging 86e8fd81f675cec84353419eebaa75c61961601a (tag) length 987 bytes by Huaxiang Sun on Fri Jan 22 14:41:24 2021 -0800 - Log - Tag 2.3.4 -BEGIN PGP SIGNATURE- iQIzBAABCAAdFiEEZQ5ukLH6mbYnhA++I6kA1BF8g14FAmALVJQACgkQI6kA1BF8 g17HBBAAptg+YwhrzXMzHQbHKMbz7YtePno0GRqfSi3hhVNcA/rjiK8vpxV1O8PI t0tLhNbqvYCIpQ1Z3EsDrYqpbdhDDF2Hudz28rwAqERymoAxbE3lpQJTSsPvYKkm 9CDhbfLvWo5DFt1KKijkJhp2Db9rWl9qldIaEIp4Gt7Mduxk7gM7KoGL2VLuTbgP /oHKWXTaN50TUfoJuo0KW8dav2Czo8DvWKhfMHwxfh0pAL+MrIkvDaZotX3F2sXB slc7Gtt/3wJJi7MZjqYHaRqtg1w67xaHzOGd6TSU6jwDkWAc16cE6JUSb73dv+id Q/NTB1sTG4fa29KjhHPQ1tD2/NC+eIoUmaue3gZO/RagmSUJTRFkTrew7ULmG6xv JXjT2aKN/n8k2IHQJaQ688CazcP+a4zyaz3h702ovr60zBsyW4GQn5CYLuq3/t0D Mo0SqwpW9OJqWPJ0WjGtWw+iiOym8Vu3hfifNxbJV4GRjLo9CsrMNQ3PVPI/nuFu ZYcib/v9wzASGZy9EdQdUpPtgsVM0onT11JVowlGhugWLwILfYZnDGp2CIlpT2Wa IO9aj0+5Xbz7Wf5eh/G5oXzNDQJy2nupnzT81sEp0oxnrofiWZ1g2rIh3HZah+7X 1mdOs1mA1sRxw1oY8g2Mfx68DPYguBgujkwBXyGeiAR9g1GFsro= =XRbt -END PGP SIGNATURE- --- No new revisions were added by this update.
svn commit: r45552 - /dev/hbase/2.3.4RC2/
Author: huaxiangsun Date: Fri Jan 22 21:06:37 2021 New Revision: 45552 Log: Remove stale 2.3.4RC2 Removed: dev/hbase/2.3.4RC2/
svn commit: r45551 - /dev/hbase/2.3.4RC0/
Author: huaxiangsun Date: Fri Jan 22 21:06:04 2021 New Revision: 45551 Log: Remove stale 2.3.4RC0 Removed: dev/hbase/2.3.4RC0/
[hbase] branch master updated: HBASE-25368 Filter out more invalid encoded name in isEncodedRegionNa… (#2868)
This is an automated email from the ASF dual-hosted git repository.
huaxiangsun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/master by this push:
new 5c13c68 HBASE-25368 Filter out more invalid encoded name in
isEncodedRegionNa… (#2868)
5c13c68 is described below
commit 5c13c684940ed6f7076b8f8240b33545058e4bdb
Author: huaxiangsun
AuthorDate: Wed Jan 20 09:04:50 2021 -0800
HBASE-25368 Filter out more invalid encoded name in isEncodedRegionNa…
(#2868)
HBASE-25368 Filter out more invalid encoded name in
isEncodedRegionName(byte[] regionName)
Signed-off-by: Duo Zhang
---
.../hadoop/hbase/client/RawAsyncHBaseAdmin.java| 87 --
.../org/apache/hadoop/hbase/client/RegionInfo.java | 18 -
.../org/apache/hadoop/hbase/client/TestAdmin1.java | 19 +
.../org/apache/hadoop/hbase/client/TestAdmin2.java | 8 +-
4 files changed, 85 insertions(+), 47 deletions(-)
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java
index 512e7a9..38bddde 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java
@@ -2388,51 +2388,56 @@ class RawAsyncHBaseAdmin implements AsyncAdmin {
if (regionNameOrEncodedRegionName == null) {
return failedFuture(new IllegalArgumentException("Passed region name
can't be null"));
}
-try {
- CompletableFuture> future;
- if (RegionInfo.isEncodedRegionName(regionNameOrEncodedRegionName)) {
-String encodedName = Bytes.toString(regionNameOrEncodedRegionName);
-if (encodedName.length() < RegionInfo.MD5_HEX_LENGTH) {
- // old format encodedName, should be meta region
- future = connection.registry.getMetaRegionLocations()
-.thenApply(locs -> Stream.of(locs.getRegionLocations())
- .filter(loc ->
loc.getRegion().getEncodedName().equals(encodedName)).findFirst());
-} else {
- future =
ClientMetaTableAccessor.getRegionLocationWithEncodedName(metaTable,
-regionNameOrEncodedRegionName);
-}
+
+CompletableFuture> future;
+if (RegionInfo.isEncodedRegionName(regionNameOrEncodedRegionName)) {
+ String encodedName = Bytes.toString(regionNameOrEncodedRegionName);
+ if (encodedName.length() < RegionInfo.MD5_HEX_LENGTH) {
+// old format encodedName, should be meta region
+future = connection.registry.getMetaRegionLocations()
+ .thenApply(locs -> Stream.of(locs.getRegionLocations())
+.filter(loc ->
loc.getRegion().getEncodedName().equals(encodedName)).findFirst());
} else {
-RegionInfo regionInfo =
-
CatalogFamilyFormat.parseRegionInfoFromRegionName(regionNameOrEncodedRegionName);
-if (regionInfo.isMetaRegion()) {
- future = connection.registry.getMetaRegionLocations()
-.thenApply(locs -> Stream.of(locs.getRegionLocations())
- .filter(loc -> loc.getRegion().getReplicaId() ==
regionInfo.getReplicaId())
- .findFirst());
-} else {
- future =
-ClientMetaTableAccessor.getRegionLocation(metaTable,
regionNameOrEncodedRegionName);
-}
+future =
ClientMetaTableAccessor.getRegionLocationWithEncodedName(metaTable,
+ regionNameOrEncodedRegionName);
+ }
+} else {
+ // Not all regionNameOrEncodedRegionName here is going to be a valid
region name,
+ // it needs to throw out IllegalArgumentException in case tableName is
passed in.
+ RegionInfo regionInfo;
+ try {
+regionInfo = CatalogFamilyFormat.parseRegionInfoFromRegionName(
+ regionNameOrEncodedRegionName);
+ } catch (IOException ioe) {
+return failedFuture(new IllegalArgumentException(ioe.getMessage()));
}
- CompletableFuture returnedFuture = new
CompletableFuture<>();
- addListener(future, (location, err) -> {
-if (err != null) {
- returnedFuture.completeExceptionally(err);
- return;
-}
-if (!location.isPresent() || location.get().getRegion() == null) {
- returnedFuture.completeExceptionally(
-new UnknownRegionException("Invalid region name or encoded region
name: " +
- Bytes.toStringBinary(regionNameOrEncodedRegionName)));
-} else {
- returnedFuture.complete(location.get());
-}
- });
- return returnedFuture;
-} catch (IOException e) {
- return failedFuture(e);
+ if (regionInfo.isMetaRegion()) {
+future = connection.registry.getMetaRegionLocations()
+ .thenApply(locs ->
svn commit: r45433 [2/4] - /dev/hbase/2.3.4RC4/
Added: dev/hbase/2.3.4RC4/CHANGES.md == --- dev/hbase/2.3.4RC4/CHANGES.md (added) +++ dev/hbase/2.3.4RC4/CHANGES.md Fri Jan 15 00:10:21 2021 @@ -0,0 +1,8616 @@ +# HBASE Changelog + + +## Release 2.3.4 - Unreleased (as of 2021-01-14) + + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | +|: |: | :--- |: | +| [HBASE-24620](https://issues.apache.org/jira/browse/HBASE-24620) | Add a ClusterManager which submits command to ZooKeeper and its Agent which picks and execute those Commands. | Major | integration tests | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | +|: |: | :--- |: | +| [HBASE-25449](https://issues.apache.org/jira/browse/HBASE-25449) | 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml | Major | conf | +| [HBASE-25476](https://issues.apache.org/jira/browse/HBASE-25476) | Enable error prone check in pre commit | Major | build | +| [HBASE-25483](https://issues.apache.org/jira/browse/HBASE-25483) | set the loadMeta log level to debug. | Major | MTTR, Region Assignment | +| [HBASE-25435](https://issues.apache.org/jira/browse/HBASE-25435) | Slow metric value can be configured | Minor | metrics | +| [HBASE-25318](https://issues.apache.org/jira/browse/HBASE-25318) | Configure where IntegrationTestImportTsv generates HFiles | Minor | integration tests | +| [HBASE-24850](https://issues.apache.org/jira/browse/HBASE-24850) | CellComparator perf improvement | Critical | Performance, scan | +| [HBASE-25425](https://issues.apache.org/jira/browse/HBASE-25425) | Some notes on RawCell | Trivial | . | +| [HBASE-25420](https://issues.apache.org/jira/browse/HBASE-25420) | Some minor improvements in rpc implementation | Minor | rpc | +| [HBASE-25246](https://issues.apache.org/jira/browse/HBASE-25246) | Backup/Restore hbase cell tags. | Major | backup | +| [HBASE-25328](https://issues.apache.org/jira/browse/HBASE-25328) | Add builder method to create Tags. | Minor | . | +| [HBASE-25339](https://issues.apache.org/jira/browse/HBASE-25339) | Method parameter and member variable are duplicated in checkSplittable() of SplitTableRegionProcedure | Minor | . | +| [HBASE-25237](https://issues.apache.org/jira/browse/HBASE-25237) | 'hbase master stop' shuts down the cluster, not the master only | Major | . | +| [HBASE-25323](https://issues.apache.org/jira/browse/HBASE-25323) | Fix potential NPE when the zookeeper path of RegionServerTracker does not exist when start | Minor | . | +| [HBASE-25187](https://issues.apache.org/jira/browse/HBASE-25187) | Improve SizeCachedKV variants initialization | Minor | . | +| [HBASE-25289](https://issues.apache.org/jira/browse/HBASE-25289) | [testing] Clean up resources after tests in rsgroup\_shell\_test.rb | Major | rsgroup, test | +| [HBASE-25261](https://issues.apache.org/jira/browse/HBASE-25261) | Upgrade Bootstrap to 3.4.1 | Major | security, UI | +| [HBASE-25267](https://issues.apache.org/jira/browse/HBASE-25267) | Add SSL keystore type and truststore related configs for HBase RESTServer | Major | REST | +| [HBASE-25003](https://issues.apache.org/jira/browse/HBASE-25003) | Backport HBASE-24350 and HBASE-24779 to branch-2.2 & branch-2.3 | Major | Replication | +| [HBASE-25240](https://issues.apache.org/jira/browse/HBASE-25240) | gson format of RpcServer.logResponse is abnormal | Minor | . | +| [HBASE-24859](https://issues.apache.org/jira/browse/HBASE-24859) | Optimize in-memory representation of mapreduce TableSplit objects | Major | mapreduce | +| [HBASE-25224](https://issues.apache.org/jira/browse/HBASE-25224) | Maximize sleep for checking meta and namespace regions availability | Major | master | +| [HBASE-24872](https://issues.apache.org/jira/browse/HBASE-24872) | refactor valueOf PoolType | Minor | Client | +| [HBASE-24827](https://issues.apache.org/jira/browse/HBASE-24827) | BackPort HBASE-11554 Remove Reusable poolmap Rpc client type. | Major | Client | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | +|: |: | :--- |: | +| [HBASE-25356](https://issues.apache.org/jira/browse/HBASE-25356) | HBaseAdmin#getRegion() needs to filter out non-regionName and non-encodedRegionName | Major | shell | +| [HBASE-25459](https://issues.apache.org/jira/browse/HBASE-25459) | WAL can't be cleaned in some scenes | Major | . | +| [HBASE-25434](https://issues.apache.org/jira/browse/HBASE-25434) | SlowDelete & SlowPut metric value should use updateDelete & updatePut | Major | regionserver | +| [HBASE-25441](https://issues.apache.org/jira/browse/HBASE-25441) | add security check for some APIs in RSRpcServices | Critical | . | +| [HBASE-25432](https://issues.apache.org/jira/browse/HBASE-25432) | we should add security checks for setTableStateInMeta and fixMeta | Blocker | . | +| [HBASE-25445](https://issues.apache.org/jira/browse/HBASE-25445) | Old WALs
svn commit: r45433 [1/4] - /dev/hbase/2.3.4RC4/
Author: huaxiangsun Date: Fri Jan 15 00:10:21 2021 New Revision: 45433 Log: Apache hbase 2.3.4RC4 Added: dev/hbase/2.3.4RC4/ dev/hbase/2.3.4RC4/CHANGES.md dev/hbase/2.3.4RC4/RELEASENOTES.md dev/hbase/2.3.4RC4/api_compare_2.3.3_to_2.3.4RC4.html dev/hbase/2.3.4RC4/hbase-2.3.4-bin.tar.gz (with props) dev/hbase/2.3.4RC4/hbase-2.3.4-bin.tar.gz.asc dev/hbase/2.3.4RC4/hbase-2.3.4-bin.tar.gz.sha512 dev/hbase/2.3.4RC4/hbase-2.3.4-client-bin.tar.gz (with props) dev/hbase/2.3.4RC4/hbase-2.3.4-client-bin.tar.gz.asc dev/hbase/2.3.4RC4/hbase-2.3.4-client-bin.tar.gz.sha512 dev/hbase/2.3.4RC4/hbase-2.3.4-src.tar.gz (with props) dev/hbase/2.3.4RC4/hbase-2.3.4-src.tar.gz.asc dev/hbase/2.3.4RC4/hbase-2.3.4-src.tar.gz.sha512
svn commit: r45433 [3/4] - /dev/hbase/2.3.4RC4/
Added: dev/hbase/2.3.4RC4/RELEASENOTES.md == --- dev/hbase/2.3.4RC4/RELEASENOTES.md (added) +++ dev/hbase/2.3.4RC4/RELEASENOTES.md Fri Jan 15 00:10:21 2021 @@ -0,0 +1,12453 @@ +# RELEASENOTES + + +# HBASE 2.3.4 Release Notes + +These release notes cover new developer and user-facing incompatibilities, important issues, features, and major improvements. + + +--- + +* [HBASE-25449](https://issues.apache.org/jira/browse/HBASE-25449) | *Major* | **'dfs.client.read.shortcircuit' should not be set in hbase-default.xml** + +The presence of HDFS short-circuit read configuration properties in hbase-default.xml inadvertently causes short-circuit reads to not happen inside of RegionServers, despite short-circuit reads being enabled in hdfs-site.xml. + + +--- + +* [HBASE-25441](https://issues.apache.org/jira/browse/HBASE-25441) | *Critical* | **add security check for some APIs in RSRpcServices** + +RsRpcServices APIs that can be accessed only through Admin rights: +- stopServer +- updateFavoredNodes +- updateConfiguration +- clearRegionBlockCache +- clearSlowLogsResponses + + +--- + +* [HBASE-25432](https://issues.apache.org/jira/browse/HBASE-25432) | *Blocker* | **we should add security checks for setTableStateInMeta and fixMeta** + +setTableStateInMeta and fixMeta can be accessed only through Admin rights + + +--- + +* [HBASE-25318](https://issues.apache.org/jira/browse/HBASE-25318) | *Minor* | **Configure where IntegrationTestImportTsv generates HFiles** + +Added IntegrationTestImportTsv.generatedHFileFolder configuration property to override the default location in IntegrationTestImportTsv. Useful for running the integration test when HDFS Transparent Encryption is enabled. + + +--- + +* [HBASE-25456](https://issues.apache.org/jira/browse/HBASE-25456) | *Critical* | **setRegionStateInMeta need security check** + +setRegionStateInMeta can be accessed only through Admin rights + + +--- + +* [HBASE-25237](https://issues.apache.org/jira/browse/HBASE-25237) | *Major* | **'hbase master stop' shuts down the cluster, not the master only** + +\`hbase master stop\` should shutdown only master by default. +1. Help added to \`hbase master stop\`: +To stop cluster, use \`stop-hbase.sh\` or \`hbase master stop --shutDownCluster\` + +2. Help added to \`stop-hbase.sh\`: +stop-hbase.sh can only be used for shutting down entire cluster. To shut down (HMaster\|HRegionServer) use hbase-daemon.sh stop (master\|regionserver) + + +--- + +* [HBASE-25238](https://issues.apache.org/jira/browse/HBASE-25238) | *Critical* | **Upgrading HBase from 2.2.0 to 2.3.x fails because of âMessage missing required fields: stateâ** + +Fixes master procedure store migration issues going from 2.0.x to 2.2.x and/or 2.3.x. Also fixes failed heartbeat parse during rolling upgrade from 2.0.x. to 2.3.x. + + +--- + +* [HBASE-25234](https://issues.apache.org/jira/browse/HBASE-25234) | *Major* | **[Upgrade]Incompatibility in reading RS report from 2.1 RS when Master is upgraded to a version containing HBASE-21406** + +Fixes so auto-migration of master procedure store works again going from 2.0.x =\> 2.2+. Also make it so heartbeats work when rolling upgrading from 2.0.x =\> 2.3+. + + +--- + +* [HBASE-25224](https://issues.apache.org/jira/browse/HBASE-25224) | *Major* | **Maximize sleep for checking meta and namespace regions availability** + +Changed the max sleep time during meta and namespace regions availability check to be 60 sec. Previously there was no such cap + + + +# HBASE 2.3.3 Release Notes + +These release notes cover new developer and user-facing incompatibilities, important issues, features, and major improvements. + + +--- + +* [HBASE-25163](https://issues.apache.org/jira/browse/HBASE-25163) | *Major* | **Increase the timeout value for nightly jobs** + +Increase timeout value for nightly jobs to 16 hours since the new build machines are dedicated to hbase project, so we are allowed to use it all the time. + + +--- + +* [HBASE-22976](https://issues.apache.org/jira/browse/HBASE-22976) | *Major* | **[HBCK2] Add RecoveredEditsPlayer** + +WALPlayer can replay the content of recovered.edits directories. + +Side-effect is that WAL filename timestamp is now factored when setting start/end times for WALInputFormat; i.e. wal.start.time and wal.end.time values on a job context. Previous we looked at wal.end.time only. Now we consider wal.start.time too. If a file has a name outside of wal.start.time\<-\>wal.end.time, it'll be by-passed. This change-in-behavior will make it easier on operator crafting timestamp filters processing WALs. + + +--- + +* [HBASE-25154](https://issues.apache.org/jira/browse/HBASE-25154) | *Major* | **Set java.io.tmpdir to project build directory to avoid writing std\*deferred files to /tmp** + +Change the java.io.tmpdir to project.build.directory in surefire-maven-plugin, to avoid writing std\*deferred files to /tmp
svn commit: r45433 [4/4] - /dev/hbase/2.3.4RC4/
Added: dev/hbase/2.3.4RC4/api_compare_2.3.3_to_2.3.4RC4.html
==
--- dev/hbase/2.3.4RC4/api_compare_2.3.3_to_2.3.4RC4.html (added)
+++ dev/hbase/2.3.4RC4/api_compare_2.3.3_to_2.3.4RC4.html Fri Jan 15 00:10:21
2021
@@ -0,0 +1,563 @@
+
+
+http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd;>
+http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
+
+
+
+
+
+hbase: rel/2.3.3 to 2.3.4RC4 compatibility report
+
+body {
+font-family:Arial, sans-serif;
+background-color:White;
+color:Black;
+}
+hr {
+color:Black;
+background-color:Black;
+height:1px;
+border:0;
+}
+h1 {
+margin-bottom:0px;
+padding-bottom:0px;
+font-size:1.625em;
+}
+h2 {
+margin-bottom:0px;
+padding-bottom:0px;
+font-size:1.25em;
+white-space:nowrap;
+}
+div.symbols {
+color:#003E69;
+}
+div.symbols i {
+color:Brown;
+}
+span.section {
+font-weight:bold;
+cursor:pointer;
+color:#003E69;
+white-space:nowrap;
+margin-left:0.3125em;
+}
+span:hover.section {
+color:#336699;
+}
+span.sect_aff {
+cursor:pointer;
+padding-left:1.55em;
+font-size:0.875em;
+color:#cc3300;
+}
+span.ext {
+font-weight:normal;
+}
+span.jar {
+color:#cc3300;
+font-size:0.875em;
+font-weight:bold;
+}
+div.jar_list {
+padding-left:0.4em;
+font-size:0.94em;
+}
+span.pkg_t {
+color:#408080;
+font-size:0.875em;
+}
+span.pkg {
+color:#408080;
+font-size:0.875em;
+font-weight:bold;
+}
+span.cname {
+color:Green;
+font-size:0.875em;
+font-weight:bold;
+}
+span.iname_b {
+font-weight:bold;
+}
+span.iname_a {
+color:#33;
+font-weight:bold;
+font-size:0.94em;
+}
+span.sym_p {
+font-weight:normal;
+white-space:normal;
+}
+span.sym_pd {
+white-space:normal;
+}
+span.sym_p span, span.sym_pd span {
+white-space:nowrap;
+}
+span.attr {
+color:Black;
+font-weight:normal;
+}
+span.deprecated {
+color:Red;
+font-weight:bold;
+font-family:Monaco, monospace;
+}
+div.affect {
+padding-left:1em;
+padding-bottom:10px;
+font-size:0.87em;
+font-style:italic;
+line-height:0.9em;
+}
+div.affected {
+padding-left:2em;
+padding-top:10px;
+}
+table.ptable {
+border-collapse:collapse;
+border:1px outset black;
+margin-left:0.95em;
+margin-top:3px;
+margin-bottom:3px;
+width:56.25em;
+}
+table.ptable td {
+border:1px solid Gray;
+padding:3px;
+font-size:0.875em;
+text-align:left;
+vertical-align:top;
+max-width:28em;
+word-wrap:break-word;
+}
+table.ptable th {
+background-color:#ee;
+font-weight:bold;
+color:#33;
+font-family:Verdana, Arial;
+font-size:0.875em;
+border:1px solid Gray;
+text-align:center;
+vertical-align:top;
+white-space:nowrap;
+padding:3px;
+}
+table.summary {
+border-collapse:collapse;
+border:1px outset black;
+}
+table.summary th {
+background-color:#ee;
+font-weight:normal;
+text-align:left;
+font-size:0.94em;
+white-space:nowrap;
+border:1px inset Gray;
+padding:3px;
+}
+table.summary td {
+text-align:right;
+white-space:nowrap;
+border:1px inset Gray;
+padding:3px 5px 3px 10px;
+}
+span.mngl {
+padding-left:1em;
+font-size:0.875em;
+cursor:text;
+color:#44;
+font-weight:bold;
+}
+span.pleft {
+padding-left:2.5em;
+}
+span.color_p {
+font-style:italic;
+color:Brown;
+}
+span.param {
+font-style:italic;
+}
+span.focus_p {
+font-style:italic;
+background-color:#DCDCDC;
+}
+span.ttype {
+font-weight:normal;
+}
+span.nowrap {
+white-space:nowrap;
+}
+span.value {
+white-space:nowrap;
+font-weight:bold;
+}
+.passed {
+background-color:#CCFFCC;
+font-weight:normal;
+}
+.warning {
+background-color:#F4F4AF;
+font-weight:normal;
+}
+.failed {
+background-color:#FF;
+font-weight:normal;
+}
+.new {
+background-color:#C6DEFF;
+font-weight:normal;
+}
+
+.compatible {
+background-color:#CCFFCC;
+font-weight:normal;
+}
+.almost_compatible {
+background-color:#FFDAA3;
+font-weight:normal;
+}
+.incompatible {
+background-color:#FF;
+font-weight:normal;
+}
+.gray {
+background-color:#DCDCDC;
+font-weight:normal;
+}
+
+.top_ref {
+font-size:0.69em;
+}
+.footer {
+font-size:0.8125em;
+}
+.tabset {
+float:left;
+}
+a.tab {
+border:1px solid Black;
+float:left;
+margin:0px 5px -1px 0px;
+padding:3px 5px 3px 5px;
+position:relative;
+font-size:0.875em;
+background-color:#DDD;
+text-decoration:none;
+color:Black;
+}
+a.disabled:hover
+{
+color:Black;
+background:#EEE;
+}
+a.active:hover
+{
+color:Black;
+background:White;
+}
+a.active {
+border-bottom-color:White;
+background-color:White;
+}
+div.tab {
+border-top:1px solid
[hbase] 01/01: Preparing development version 2.3.5-SNAPSHOT
This is an automated email from the ASF dual-hosted git repository. huaxiangsun pushed a commit to branch branch-2.3 in repository https://gitbox.apache.org/repos/asf/hbase.git commit 995157e345bda5237f4070e125ada25ee07e351d Author: huaxiangsun AuthorDate: Thu Jan 14 21:17:33 2021 + Preparing development version 2.3.5-SNAPSHOT Signed-off-by: huaxiangsun --- hbase-annotations/pom.xml | 2 +- hbase-archetypes/hbase-archetype-builder/pom.xml | 2 +- hbase-archetypes/hbase-client-project/pom.xml | 2 +- hbase-archetypes/hbase-shaded-client-project/pom.xml | 2 +- hbase-archetypes/pom.xml | 2 +- hbase-assembly/pom.xml | 2 +- hbase-asyncfs/pom.xml | 2 +- hbase-build-configuration/pom.xml | 2 +- hbase-checkstyle/pom.xml | 4 ++-- hbase-client/pom.xml | 2 +- hbase-common/pom.xml | 2 +- hbase-endpoint/pom.xml | 2 +- hbase-examples/pom.xml | 2 +- hbase-external-blockcache/pom.xml | 2 +- hbase-hadoop-compat/pom.xml| 2 +- hbase-hadoop2-compat/pom.xml | 2 +- hbase-hbtop/pom.xml| 2 +- hbase-http/pom.xml | 2 +- hbase-it/pom.xml | 2 +- hbase-logging/pom.xml | 2 +- hbase-mapreduce/pom.xml| 2 +- hbase-metrics-api/pom.xml | 2 +- hbase-metrics/pom.xml | 2 +- hbase-procedure/pom.xml| 2 +- hbase-protocol-shaded/pom.xml | 2 +- hbase-protocol/pom.xml | 2 +- hbase-replication/pom.xml | 2 +- hbase-resource-bundle/pom.xml | 2 +- hbase-rest/pom.xml | 2 +- hbase-rsgroup/pom.xml | 2 +- hbase-server/pom.xml | 2 +- hbase-shaded/hbase-shaded-check-invariants/pom.xml | 2 +- hbase-shaded/hbase-shaded-client-byo-hadoop/pom.xml| 2 +- hbase-shaded/hbase-shaded-client/pom.xml | 2 +- hbase-shaded/hbase-shaded-mapreduce/pom.xml| 2 +- hbase-shaded/hbase-shaded-testing-util-tester/pom.xml | 2 +- hbase-shaded/hbase-shaded-testing-util/pom.xml | 2 +- hbase-shaded/hbase-shaded-with-hadoop-check-invariants/pom.xml | 2 +- hbase-shaded/pom.xml | 2 +- hbase-shell/pom.xml| 2 +- hbase-testing-util/pom.xml | 2 +- hbase-thrift/pom.xml | 2 +- hbase-zookeeper/pom.xml| 2 +- pom.xml| 2 +- 44 files changed, 45 insertions(+), 45 deletions(-) diff --git a/hbase-annotations/pom.xml b/hbase-annotations/pom.xml index 40ea816..9a046e5 100644 --- a/hbase-annotations/pom.xml +++ b/hbase-annotations/pom.xml @@ -23,7 +23,7 @@ hbase org.apache.hbase -2.3.4 +2.3.5-SNAPSHOT .. diff --git a/hbase-archetypes/hbase-archetype-builder/pom.xml b/hbase-archetypes/hbase-archetype-builder/pom.xml index c94b38d..b964c65 100644 --- a/hbase-archetypes/hbase-archetype-builder/pom.xml +++ b/hbase-archetypes/hbase-archetype-builder/pom.xml @@ -25,7 +25,7 @@ hbase-archetypes org.apache.hbase -2.3.4 +2.3.5-SNAPSHOT .. diff --git a/hbase-archetypes/hbase-client-project/pom.xml b/hbase-archetypes/hbase-client-project/pom.xml index de6e2da..6e6394c 100644 --- a/hbase-archetypes/hbase-client-project/pom.xml +++ b/hbase-archetypes/hbase-client-project/pom.xml @@ -26,7 +26,7 @@ hbase-archetypes org.apache.hbase -2.3.4 +2.3.5-SNAPSHOT .. hbase-client-project diff --git a/hbase-archetypes/hbase-shaded-client-project/pom.xml b/hbase-archetypes/hbase-shaded-client-project/pom.xml index 0619442..3496e85 100644 --- a/hbase-archetypes/hbase-shaded-client-project/pom.xml +++ b/hbase-archetypes/hbase-shaded-client-project/pom.xml @@ -26,7 +26,7 @@ hbase-archetypes org.apache.hbase -2.3.4 +2.3.5-SNAPSHOT .. hbase-shaded-client-project diff --git a/hbase
