date:20201029

[GitHub] [hbase] Apache-HBase commented on pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…

2020-10-29 Thread GitBox



Apache-HBase commented on pull request #2584:
URL: https://github.com/apache/hbase/pull/2584#issuecomment-718427635


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m  6s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 12s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   3m 46s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   1m 36s |  master passed  |
   | +1 :green_heart: |  spotbugs  |   3m  5s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 11s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   3m 48s |  the patch passed  |
   | -0 :warning: |  checkstyle  |   0m 28s |  hbase-client: The patch 
generated 3 new + 2 unchanged - 0 fixed = 5 total (was 2)  |
   | -0 :warning: |  checkstyle  |   1m 10s |  hbase-server: The patch 
generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  hadoopcheck  |  18m 46s |  Patch does not cause any 
errors with Hadoop 3.1.2 3.2.1 3.3.0.  |
   | +1 :green_heart: |  spotbugs  |   3m 29s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 21s |  The patch does not generate 
ASF License warnings.  |
   |  |   |  45m 45s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.13 Server=19.03.13 base: 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/6/artifact/yetus-general-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/2584 |
   | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti 
checkstyle |
   | uname | Linux 98ff7fb50def 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / bb4a9d335f |
   | checkstyle | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/6/artifact/yetus-general-check/output/diff-checkstyle-hbase-client.txt
 |
   | checkstyle | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/6/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
 |
   | Max. process+thread count | 84 (vs. ulimit of 3) |
   | modules | C: hbase-client hbase-server U: . |
   | Console output | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/6/console
 |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) 
spotbugs=3.1.12 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] anoopsjohn commented on pull request #2594: HBASE-25227 [branch-1] Cast in UnsafeAccess to avoid Java 11 runtime issue

2020-10-29 Thread GitBox



anoopsjohn commented on pull request #2594:
URL: https://github.com/apache/hbase/pull/2594#issuecomment-718503775


   > +1, what about directly changing BYTE_ARRAY_BASE_OFFSET's type to long
   
   Ya that single change will be enough then and can avoid all cast



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Reidddddd commented on a change in pull request #2591: HBASE-24859: Optimize in-memory representation of HBase map reduce table splits

2020-10-29 Thread GitBox



Reidd commented on a change in pull request #2591:
URL: https://github.com/apache/hbase/pull/2591#discussion_r514092739



##
File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSplit.java
##
@@ -184,9 +184,24 @@ public TableSplit(final byte [] tableName, byte[] 
startRow, byte[] endRow,
* @param startRow The start row of the split.
* @param endRow The end row of the split.
* @param location The location of the region.
+   * @param encodedRegionName The region ID.
+   * @param length Size of region in bytes
*/
   public TableSplit(TableName tableName, byte[] startRow, byte[] endRow,
-  final String location) {
+final String location, final String encodedRegionName, long length) {
+this(tableName, null, startRow, endRow, location, encodedRegionName, 
length);

Review comment:
   can't we ignore the scan assignment, just using `this.scan = ""` anyway, 
instead of the following:
   ```
   try {
 this.scan =
   (null == scan) ? "" : TableMapReduceUtil.convertScanToString(scan);
   } catch (IOException e) {
 LOG.warn("Failed to convert Scan to String", e);
   }
   ```
   I also prefer not to add a new constructor.

##
File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSplit.java
##
@@ -184,9 +184,24 @@ public TableSplit(final byte [] tableName, byte[] 
startRow, byte[] endRow,
* @param startRow The start row of the split.
* @param endRow The end row of the split.
* @param location The location of the region.
+   * @param encodedRegionName The region ID.
+   * @param length Size of region in bytes
*/
   public TableSplit(TableName tableName, byte[] startRow, byte[] endRow,
-  final String location) {
+final String location, final String encodedRegionName, long length) {
+this(tableName, null, startRow, endRow, location, encodedRegionName, 
length);

Review comment:
   iff we make sure the scan is useless here 





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…

2020-10-29 Thread GitBox



Apache-HBase commented on pull request #2584:
URL: https://github.com/apache/hbase/pull/2584#issuecomment-718552316


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   3m 59s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): 
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 22s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   4m  9s |  master passed  |
   | +1 :green_heart: |  compile  |   1m 35s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   6m 38s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   1m  7s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 16s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   4m  6s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 41s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 41s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   6m 41s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   1m  6s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   1m 11s |  hbase-client in the patch passed.  
|
   | +1 :green_heart: |  unit  | 131m 22s |  hbase-server in the patch passed.  
|
   |  |   | 166m 30s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.13 Server=19.03.13 base: 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/6/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/2584 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 2107d6099071 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 
16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / bb4a9d335f |
   | Default Java | 2020-01-14 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/6/testReport/
 |
   | Max. process+thread count | 3961 (vs. ulimit of 3) |
   | modules | C: hbase-client hbase-server U: . |
   | Console output | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/6/console
 |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] GeorryHuang opened a new pull request #2595: HBASE-25090 CompactionConfiguration logs unrealistic store file sizes

2020-10-29 Thread GitBox



GeorryHuang opened a new pull request #2595:
URL: https://github.com/apache/hbase/pull/2595


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…

2020-10-29 Thread GitBox



Apache-HBase commented on pull request #2584:
URL: https://github.com/apache/hbase/pull/2584#issuecomment-718639479


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m  7s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): 
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 12s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   3m 44s |  master passed  |
   | +1 :green_heart: |  compile  |   1m 21s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   7m  9s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 56s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 13s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   3m 49s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 23s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 23s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   7m  5s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 58s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   1m 11s |  hbase-client in the patch passed.  
|
   | +1 :green_heart: |  unit  | 206m 12s |  hbase-server in the patch passed.  
|
   |  |   | 237m 10s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.13 Server=19.03.13 base: 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/6/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/2584 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 4d4984b72b8d 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / bb4a9d335f |
   | Default Java | 1.8.0_232 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/6/testReport/
 |
   | Max. process+thread count | 3536 (vs. ulimit of 3) |
   | modules | C: hbase-client hbase-server U: . |
   | Console output | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/6/console
 |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #2595: HBASE-25090 CompactionConfiguration logs unrealistic store file sizes

2020-10-29 Thread GitBox



Apache-HBase commented on pull request #2595:
URL: https://github.com/apache/hbase/pull/2595#issuecomment-718684251


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m  8s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   ||| _ branch-2 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   3m 58s |  branch-2 passed  |
   | +1 :green_heart: |  checkstyle  |   1m 16s |  branch-2 passed  |
   | +1 :green_heart: |  spotbugs  |   2m  5s |  branch-2 passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   3m 38s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   1m 13s |  the patch passed  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  hadoopcheck  |  12m 41s |  Patch does not cause any 
errors with Hadoop 3.1.2 3.2.1.  |
   | +1 :green_heart: |  spotbugs  |   2m 15s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 12s |  The patch does not generate 
ASF License warnings.  |
   |  |   |  36m  7s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.13 Server=19.03.13 base: 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2595/1/artifact/yetus-general-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/2595 |
   | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti 
checkstyle |
   | uname | Linux f655fd5ddec1 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | branch-2 / b155c62f38 |
   | Max. process+thread count | 84 (vs. ulimit of 12500) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2595/1/console
 |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) 
spotbugs=3.1.12 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #2582: HBASE-25187 Improve SizeCachedKV variants initialization

2020-10-29 Thread GitBox



Apache-HBase commented on pull request #2582:
URL: https://github.com/apache/hbase/pull/2582#issuecomment-718706220


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m 15s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): 
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 21s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   4m 19s |  master passed  |
   | +1 :green_heart: |  compile  |   1m 35s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   6m 52s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   1m  7s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 15s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   4m  6s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 33s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 33s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   6m 44s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   1m  5s |  the patch passed  |
   ||| _ Other Tests _ |
   | -1 :x: |  unit  |   0m 58s |  hbase-common in the patch failed.  |
   | -1 :x: |  unit  |   7m 27s |  hbase-server in the patch failed.  |
   |  |   |  38m 57s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.13 Server=19.03.13 base: 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/2582 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux af9bfdf2e913 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 
17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / bb4a9d335f |
   | Default Java | 2020-01-14 |
   | unit | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-common.txt
 |
   | unit | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/2/testReport/
 |
   | Max. process+thread count | 996 (vs. ulimit of 3) |
   | modules | C: hbase-common hbase-server U: . |
   | Console output | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/2/console
 |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #2582: HBASE-25187 Improve SizeCachedKV variants initialization

2020-10-29 Thread GitBox



Apache-HBase commented on pull request #2582:
URL: https://github.com/apache/hbase/pull/2582#issuecomment-718705999


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   4m  0s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): 
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 22s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   3m 36s |  master passed  |
   | +1 :green_heart: |  compile  |   1m 19s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   6m 34s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   1m  1s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 16s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   3m 32s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 19s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 19s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   6m 28s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 58s |  the patch passed  |
   ||| _ Other Tests _ |
   | -1 :x: |  unit  |   0m 49s |  hbase-common in the patch failed.  |
   | -1 :x: |  unit  |   6m 48s |  hbase-server in the patch failed.  |
   |  |   |  38m 31s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.13 Server=19.03.13 base: 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/2582 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 3b84bb189870 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / bb4a9d335f |
   | Default Java | 1.8.0_232 |
   | unit | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-common.txt
 |
   | unit | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/2/testReport/
 |
   | Max. process+thread count | 837 (vs. ulimit of 3) |
   | modules | C: hbase-common hbase-server U: . |
   | Console output | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/2/console
 |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #2582: HBASE-25187 Improve SizeCachedKV variants initialization

2020-10-29 Thread GitBox



Apache-HBase commented on pull request #2582:
URL: https://github.com/apache/hbase/pull/2582#issuecomment-718709754


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m 16s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 28s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   3m 59s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   1m 34s |  master passed  |
   | +1 :green_heart: |  spotbugs  |   2m 54s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 12s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   3m 48s |  the patch passed  |
   | -0 :warning: |  checkstyle  |   0m 24s |  hbase-common: The patch 
generated 7 new + 3 unchanged - 0 fixed = 10 total (was 3)  |
   | -0 :warning: |  checkstyle  |   1m 13s |  hbase-server: The patch 
generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  hadoopcheck  |  19m 16s |  Patch does not cause any 
errors with Hadoop 3.1.2 3.2.1 3.3.0.  |
   | -1 :x: |  spotbugs  |   0m 53s |  hbase-common generated 3 new + 0 
unchanged - 0 fixed = 3 total (was 0)  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 21s |  The patch does not generate 
ASF License warnings.  |
   |  |   |  46m 29s |   |
   
   
   | Reason | Tests |
   |---:|:--|
   | FindBugs | module:hbase-common |
   |  |  org.apache.hadoop.hbase.SizeCachedByteBufferKeyValue doesn't override 
ByteBufferKeyValue.equals(Object)  At SizeCachedByteBufferKeyValue.java:At 
SizeCachedByteBufferKeyValue.java:[line 1] |
   |  |  Dead store to keyLen rather than field with same name in new 
org.apache.hadoop.hbase.SizeCachedKeyValue(byte[], int, int, long, int)  At 
SizeCachedKeyValue.java:than field with same name in new 
org.apache.hadoop.hbase.SizeCachedKeyValue(byte[], int, int, long, int)  At 
SizeCachedKeyValue.java:[line 47] |
   |  |  org.apache.hadoop.hbase.SizeCachedNoTagsByteBufferKeyValue doesn't 
override ByteBufferKeyValue.equals(Object)  At 
SizeCachedNoTagsByteBufferKeyValue.java:At 
SizeCachedNoTagsByteBufferKeyValue.java:[line 1] |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.13 Server=19.03.13 base: 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/2/artifact/yetus-general-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/2582 |
   | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti 
checkstyle |
   | uname | Linux 41263c094b4b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / bb4a9d335f |
   | checkstyle | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-common.txt
 |
   | checkstyle | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
 |
   | spotbugs | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/2/artifact/yetus-general-check/output/new-spotbugs-hbase-common.html
 |
   | Max. process+thread count | 84 (vs. ulimit of 3) |
   | modules | C: hbase-common hbase-server U: . |
   | Console output | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/2/console
 |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) 
spotbugs=3.1.12 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] virajjasani commented on a change in pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close

2020-10-29 Thread GitBox



virajjasani commented on a change in pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#discussion_r514160584



##
File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -9000,6 +9173,51 @@ public long getReadPoint() {
 return getReadPoint(IsolationLevel.READ_COMMITTED);
   }
 
+  /**
+   * Interrupt any region options that have acquired the region lock via
+   * {@link 
#startRegionOperation(org.apache.hadoop.hbase.regionserver.Region.Operation)},
+   * or {@link #startBulkRegionOperation(boolean)}.
+   */
+  private void interruptRegionOperations() {
+for (Map.Entry entry: regionLockHolders.entrySet()) {
+  // An entry in this map will have a boolean value indicating if it is 
currently
+  // eligible for interrupt; if so, we should interrupt it.
+  if (entry.getValue().booleanValue()) {

Review comment:
   nit: `entry.getValue()` is enough





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache9 commented on a change in pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close

2020-10-29 Thread GitBox



Apache9 commented on a change in pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#discussion_r514222462



##
File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -4569,13 +4665,29 @@ private void doMiniBatchMutate(BatchOperation 
batchOp) throws IOException {
   // We should record the timestamp only after we have acquired the 
rowLock,
   // otherwise, newer puts/deletes/increment/append are not guaranteed to 
have a newer
   // timestamp
+
+  // Check for thread interrupt status in case we have been signaled from
+  // #interruptRegionOperation.
+  checkInterrupt();
+
   long now = EnvironmentEdgeManager.currentTime();
   batchOp.prepareMiniBatchOperations(miniBatchOp, now, acquiredRowLocks);
 
   // STEP 3. Build WAL edit
+
+  // Check for thread interrupt status in case we have been signaled from
+  // #interruptRegionOperation.
+  checkInterrupt();
+
   List> walEdits = 
batchOp.buildWALEdits(miniBatchOp);
 
   // STEP 4. Append the WALEdits to WAL and sync.
+
+  // Check for thread interrupt status in case we have been signaled from
+  // #interruptRegionOperation. This is the last place we can do it 
"safely" before
+  // WAL appends.
+  checkInterrupt();
+

Review comment:
   After reading this discussion several times, I think the reason we do 
not want to niterrupte a WAL sync is that it may lead to a region server abort?
   I would say this is not the case here. I checked the code again, the actual 
sync is done in the disruptor thread, in the rpc thread we just block on a 
SyncFuture(as Andrew mentioned above), the interruption on the rpc thread will 
just lead to an IOException tp client, the actual sync operation will not be 
interrupted so we are safe.
   So I do not think we need to disable interrupts here?

##
File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -8777,23 +8936,36 @@ public void closeRegionOperation(Operation operation) 
throws IOException {
 if (operation == Operation.SNAPSHOT) {
   stores.values().forEach(HStore::postSnapshotOperation);
 }
+regionLockHolders.remove(Thread.currentThread());
 lock.readLock().unlock();
 if (coprocessorHost != null) {
   coprocessorHost.postCloseRegionOperation(operation);
 }
   }
 
+  /**
+   * If a handler thread is eligible for interrupt, make it ineligible. Should 
be paired
+   * with {{@link #enableInterrupts()}.
+   */
+  protected void disableInterrupts() {
+regionLockHolders.computeIfPresent(Thread.currentThread(), (t,b) -> false);
+  }
+
+  /**
+   * If a handler thread was made ineligible for interrupt via {{@link 
#disableInterrupts()},
+   * make it eligible again. No-op if interrupts are already enabled.
+   */
+  protected void enableInterrupts() {
+regionLockHolders.computeIfPresent(Thread.currentThread(), (t,b) -> true);
+  }
+
   /**
* This method needs to be called before any public call that reads or
* modifies stores in bulk. It has to be called just before a try.
* #closeBulkRegionOperation needs to be called in the try's finally block
* Acquires a writelock and checks if the region is closing or closed.
-   * @throws NotServingRegionException when the region is closing or closed

Review comment:
   Why remove these comments? They are not the cases for now?

##
File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -6588,8 +6677,10 @@ protected RowLock getRowLockInternal(byte[] row, boolean 
readLock, final RowLock
   success = true;
   return result;
 } catch (InterruptedException ie) {
-  LOG.warn("Thread interrupted waiting for lock on row: {}, in region {}", 
rowKey,
-getRegionInfo().getRegionNameAsString());
+  if (LOG.isDebugEnabled()) {
+LOG.debug("Thread interrupted waiting for lock on row: {}, in region 
{}", rowKey,

Review comment:
   I mean the log message. Waiting on row lock is not the only case now?





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache9 commented on a change in pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…

2020-10-29 Thread GitBox



Apache9 commented on a change in pull request #2584:
URL: https://github.com/apache/hbase/pull/2584#discussion_r514224110



##
File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/CatalogReplicaLoadBalanceReplicaSelector.java
##
@@ -0,0 +1,46 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.client;
+
+import org.apache.hadoop.hbase.HRegionLocation;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.yetus.audience.InterfaceAudience;
+
+/**
+ * A Catalog replica selector decides which catalog replica to go for read 
requests.
+ */
+@InterfaceAudience.Private
+public interface CatalogReplicaLoadBalanceReplicaSelector {
+
+  /**
+   *  This method is called upon when input location is stale.
+   * @param loc location which is stale.
+   * @param fromReplicaId which replica the stale location comes from.
+   */
+  void notifyOnError(HRegionLocation loc, int fromReplicaId);

Review comment:
   NotServingRegionException? Anyway, this is an IA.Private class so it can 
be changed at any time. I think we could move the filtering logic down to the 
implementation class.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache9 commented on a change in pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…

2020-10-29 Thread GitBox



Apache9 commented on a change in pull request #2584:
URL: https://github.com/apache/hbase/pull/2584#discussion_r514228087



##
File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncNonMetaRegionLocator.java
##
@@ -577,6 +635,15 @@ private void removeLocationFromCache(HRegionLocation loc) {
   if (!canUpdateOnError(loc, oldLoc)) {
 return;
   }
+  // Tell metaReplicaSelector that the location is stale. It will create a 
stale entry
+  // with timestamp internally. Next time the client looks up the same 
location,
+  // it will pick a different meta replica region. For the current 
implementation,
+  // the metaReplicaId is not used, so the primary one is passed in.
+  if (this.metaReplicaMode == CatalogReplicaMode.LoadBalance) {
+// metaReplicaId is not used in simpleSelector, default to the primary 
one.
+metaReplicaSelector.onError(loc, RegionInfo.DEFAULT_REPLICA_ID);

Review comment:
   This is a bit strange? We are using a CatalogReplicaLoadBalanceSelector, 
not a CatalogReplicaLoadBalanceSimpleSelector, so I do not think we should 
assume it is a simple selector and pass default replica id here? It may spend a 
bunch of time for later developers who implement new selectors to find out why 
the selector does not work as expected.  If you want to do this please just 
declare a CatalogReplicaLoadBalanceSimpleSelector above.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache9 commented on a change in pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…

2020-10-29 Thread GitBox



Apache9 commented on a change in pull request #2584:
URL: https://github.com/apache/hbase/pull/2584#discussion_r514232329



##
File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/CatalogReplicaLoadBalanceReplicaSimpleSelector.java
##
@@ -0,0 +1,273 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.client;
+
+import static org.apache.hadoop.hbase.client.ConnectionUtils.isEmptyStopRow;
+import static org.apache.hadoop.hbase.util.Bytes.BYTES_COMPARATOR;
+import static org.apache.hadoop.hbase.util.ConcurrentMapUtils.computeIfAbsent;
+import java.io.IOException;
+import java.util.Iterator;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ConcurrentMap;
+import java.util.concurrent.ConcurrentNavigableMap;
+import java.util.concurrent.ConcurrentSkipListMap;
+import java.util.concurrent.ThreadLocalRandom;
+import org.apache.commons.lang3.builder.ToStringBuilder;
+import org.apache.hadoop.hbase.HRegionLocation;
+import org.apache.hadoop.hbase.ScheduledChore;
+import org.apache.hadoop.hbase.Stoppable;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * CatalogReplicaLoadBalanceReplicaSimpleSelector implements a simple catalog 
replica load balancing
+ * algorithm. It maintains a stale location cache for each table. Whenever 
client looks up location,
+ * it first check if the row is the stale location cache. If yes, the location 
from
+ * catalog replica is stale, it will go to the primary region to look up 
update-to-date location;
+ * otherwise, it will randomly pick up a replica region for lookup. When 
clients receive
+ * RegionNotServedException from region servers, it will add these region 
locations to the stale
+ * location cache. The stale cache will be cleaned up periodically by a chore.
+ *
+ * It follows a simple algorithm to choose a replica to go:
+ *
+ *  1. If there is no stale location entry for rows it looks up, it will 
randomly
+ * pick a replica region to do lookup.
+ *  2. If the location from the replica region is stale, client gets 
RegionNotServedException
+ * from region server, in this case, it will create 
StaleLocationCacheEntry in
+ * CatalogReplicaLoadBalanceReplicaSimpleSelector.
+ *  3. When client tries to do location lookup, it checks StaleLocationCache 
first for rows it
+ * tries to lookup, if entry exists, it will go with primary meta region 
to do lookup;
+ * otherwise, it will follow step 1.
+ *  4. A chore will periodically run to clean up cache entries in the 
StaleLocationCache.
+ */
+class CatalogReplicaLoadBalanceReplicaSimpleSelector implements
+  CatalogReplicaLoadBalanceReplicaSelector {
+  private static final Logger LOG =
+
LoggerFactory.getLogger(CatalogReplicaLoadBalanceReplicaSimpleSelector.class);
+  private final long STALE_CACHE_TIMEOUT_IN_MILLISECONDS = 3000; // 3 seconds
+  private final int STALE_CACHE_CLEAN_CHORE_INTERVAL = 1500; // 1.5 seconds
+
+  /**
+   * StaleLocationCacheEntry is the entry when a stale location is reported by 
an client.
+   */
+  private static final class StaleLocationCacheEntry {
+// replica id where the stale location comes from.
+private int fromReplicaId;
+
+// timestamp in milliseconds
+private long timestamp;
+
+private byte[] endKey;
+
+StaleLocationCacheEntry(final int metaReplicaId, final byte[] endKey) {
+  this.fromReplicaId = metaReplicaId;
+  this.endKey = endKey;
+  timestamp = System.currentTimeMillis();
+}
+
+public byte[] getEndKey() {
+  return this.endKey;
+}
+
+public int getFromReplicaId() {
+  return this.fromReplicaId;
+}
+public long getTimestamp() {
+  return this.timestamp;
+}
+
+@Override
+public String toString() {
+  return new ToStringBuilder(this)
+.append("endKey", endKey)
+.append("fromReplicaId", fromReplicaId)
+.append("timestamp", timestamp)
+.toString();
+}
+  }
+
+  private static final class StaleTableCache {
+private final ConcurrentNavigableMap 
cache =
+

[jira] [Commented] (HBASE-25223) Use try-with-resources statement in snapshot package

2020-10-29 Thread Hudson (Jira)



[ 
https://issues.apache.org/jira/browse/HBASE-25223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17222882#comment-17222882
 ] 

Hudson commented on HBASE-25223:


Results for branch master
[build #109 on 
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/109/]:
 (x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/109/General_20Nightly_20Build_20Report/]






(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/109/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11 
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/109/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Use try-with-resources statement in snapshot package
> 
>
> Key: HBASE-25223
> URL: https://issues.apache.org/jira/browse/HBASE-25223
> Project: HBase
>  Issue Type: Improvement
>Reporter: minjikim
>Assignee: minjikim
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
> Using try-with-resources statement makes the code simpler.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-25176) MasterStoppedException should be moved to hbase-client module

2020-10-29 Thread Hudson (Jira)



[ 
https://issues.apache.org/jira/browse/HBASE-25176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17222880#comment-17222880
 ] 

Hudson commented on HBASE-25176:


Results for branch master
[build #109 on 
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/109/]:
 (x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/109/General_20Nightly_20Build_20Report/]






(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/109/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11 
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/109/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> MasterStoppedException should be moved to hbase-client module
> -
>
> Key: HBASE-25176
> URL: https://issues.apache.org/jira/browse/HBASE-25176
> Project: HBase
>  Issue Type: Bug
>  Components: Client
>Reporter: Duo Zhang
>Assignee: niuyulin
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
> Since it will be thrown from the HMaster.checkInitialized which will be 
> called in a rpc call method, which means it will be propagated to client, we 
> should move this class to hbase-client.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-25201) YouAreDeadException should be moved to hbase-server module

2020-10-29 Thread Hudson (Jira)



[ 
https://issues.apache.org/jira/browse/HBASE-25201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17222881#comment-17222881
 ] 

Hudson commented on HBASE-25201:


Results for branch master
[build #109 on 
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/109/]:
 (x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/109/General_20Nightly_20Build_20Report/]






(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/109/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11 
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/109/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> YouAreDeadException should be moved to hbase-server module
> --
>
> Key: HBASE-25201
> URL: https://issues.apache.org/jira/browse/HBASE-25201
> Project: HBase
>  Issue Type: Improvement
>  Components: Client
>Reporter: niuyulin
>Assignee: niuyulin
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
> Since it will be thrown from the region server reports, it will not be 
> propagated to client, we should move this class to hbase-server.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-25224) Maximize sleep for checking meta and namespace regions availability

2020-10-29 Thread Hudson (Jira)



[ 
https://issues.apache.org/jira/browse/HBASE-25224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17222883#comment-17222883
 ] 

Hudson commented on HBASE-25224:


Results for branch master
[build #109 on 
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/109/]:
 (x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/109/General_20Nightly_20Build_20Report/]






(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/109/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11 
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/109/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Maximize sleep for checking meta and namespace regions availability
> ---
>
> Key: HBASE-25224
> URL: https://issues.apache.org/jira/browse/HBASE-25224
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.0, 2.2.7, 2.3.4
>
>
> The isRegionOnline method in HMaster is used on Master startup to check the 
> availability of hbase:meta and hbase:namespace tables.
> I've run into an issue when namespace was not online and Master was just 
> waiting there. I've used HBCK2 to fix the cluster but the initialization was 
> not completed because RetryCounterFactory was already waiting for 10+ hours.
> Since Master is waiting in an idle state it makes no harm to check the region 
> availability more frequently and limit the maximum sleep time.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] Apache-HBase commented on pull request #2595: HBASE-25090 CompactionConfiguration logs unrealistic store file sizes

2020-10-29 Thread GitBox



Apache-HBase commented on pull request #2595:
URL: https://github.com/apache/hbase/pull/2595#issuecomment-718749861


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 33s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  7s |  Unprocessed flag(s): 
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ branch-2 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   4m 14s |  branch-2 passed  |
   | +1 :green_heart: |  compile  |   1m  5s |  branch-2 passed  |
   | +1 :green_heart: |  shadedjars  |   6m 41s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 42s |  branch-2 passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   3m 57s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m  5s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m  5s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   6m 36s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 39s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  | 131m  5s |  hbase-server in the patch passed.  
|
   |  |   | 158m 56s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.13 Server=19.03.13 base: 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2595/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/2595 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux ff81262c5806 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | branch-2 / b155c62f38 |
   | Default Java | 2020-01-14 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2595/1/testReport/
 |
   | Max. process+thread count | 4198 (vs. ulimit of 12500) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2595/1/console
 |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #2595: HBASE-25090 CompactionConfiguration logs unrealistic store file sizes

2020-10-29 Thread GitBox



Apache-HBase commented on pull request #2595:
URL: https://github.com/apache/hbase/pull/2595#issuecomment-718754483


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   4m  5s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  7s |  Unprocessed flag(s): 
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ branch-2 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   3m 41s |  branch-2 passed  |
   | +1 :green_heart: |  compile  |   0m 59s |  branch-2 passed  |
   | +1 :green_heart: |  shadedjars  |   5m 57s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 39s |  branch-2 passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   3m 19s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 58s |  the patch passed  |
   | +1 :green_heart: |  javac  |   0m 58s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   5m 57s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 37s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  | 137m 49s |  hbase-server in the patch passed.  
|
   |  |   | 166m 18s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.13 Server=19.03.13 base: 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2595/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/2595 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux dcd4b73f4762 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 
16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | branch-2 / b155c62f38 |
   | Default Java | 1.8.0_232 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2595/1/testReport/
 |
   | Max. process+thread count | 4114 (vs. ulimit of 12500) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2595/1/console
 |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Commented] (HBASE-25221) Backport HBASE-24368 "Let HBCKSCP clear 'Unknown Servers', even if RegionStateNode has RegionLocation == null"

2020-10-29 Thread Peter Somogyi (Jira)



[ 
https://issues.apache.org/jira/browse/HBASE-25221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17222911#comment-17222911
 ] 

Peter Somogyi commented on HBASE-25221:
---

The original commit (HBASE-24368) was merged to branch-2.3, branch-2 and master 
but it was not pushed to branch-2.2 at that time. I've created this backport 
ticket for branch-2.2 because the original was resolved a long time ago. So now 
all active HBase 2 based active releases have the text on hbck report page.

> Backport HBASE-24368 "Let HBCKSCP clear 'Unknown Servers', even if 
> RegionStateNode has RegionLocation == null"
> --
>
> Key: HBASE-25221
> URL: https://issues.apache.org/jira/browse/HBASE-25221
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Fix For: 2.2.7
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] Apache-HBase commented on pull request #2582: HBASE-25187 Improve SizeCachedKV variants initialization

2020-10-29 Thread GitBox



Apache-HBase commented on pull request #2582:
URL: https://github.com/apache/hbase/pull/2582#issuecomment-718775649


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m 13s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  1s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 20s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   4m  0s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   1m 35s |  master passed  |
   | +1 :green_heart: |  spotbugs  |   2m 50s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 11s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   3m 48s |  the patch passed  |
   | -0 :warning: |  checkstyle  |   0m 23s |  hbase-common: The patch 
generated 7 new + 3 unchanged - 0 fixed = 10 total (was 3)  |
   | -0 :warning: |  checkstyle  |   1m 10s |  hbase-server: The patch 
generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  hadoopcheck  |  19m 34s |  Patch does not cause any 
errors with Hadoop 3.1.2 3.2.1 3.3.0.  |
   | -1 :x: |  spotbugs  |   1m  5s |  hbase-common generated 2 new + 0 
unchanged - 0 fixed = 2 total (was 0)  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 22s |  The patch does not generate 
ASF License warnings.  |
   |  |   |  47m  0s |   |
   
   
   | Reason | Tests |
   |---:|:--|
   | FindBugs | module:hbase-common |
   |  |  org.apache.hadoop.hbase.SizeCachedByteBufferKeyValue doesn't override 
ByteBufferKeyValue.equals(Object)  At SizeCachedByteBufferKeyValue.java:At 
SizeCachedByteBufferKeyValue.java:[line 1] |
   |  |  org.apache.hadoop.hbase.SizeCachedNoTagsByteBufferKeyValue doesn't 
override ByteBufferKeyValue.equals(Object)  At 
SizeCachedNoTagsByteBufferKeyValue.java:At 
SizeCachedNoTagsByteBufferKeyValue.java:[line 1] |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.13 Server=19.03.13 base: 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/3/artifact/yetus-general-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/2582 |
   | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti 
checkstyle |
   | uname | Linux 2751c6219819 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / bb4a9d335f |
   | checkstyle | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-common.txt
 |
   | checkstyle | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
 |
   | spotbugs | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/3/artifact/yetus-general-check/output/new-spotbugs-hbase-common.html
 |
   | Max. process+thread count | 84 (vs. ulimit of 3) |
   | modules | C: hbase-common hbase-server U: . |
   | Console output | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/3/console
 |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) 
spotbugs=3.1.12 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #2595: HBASE-25090 CompactionConfiguration logs unrealistic store file sizes

2020-10-29 Thread GitBox



Apache-HBase commented on pull request #2595:
URL: https://github.com/apache/hbase/pull/2595#issuecomment-718779796


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m 36s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   ||| _ branch-2 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   4m 21s |  branch-2 passed  |
   | +1 :green_heart: |  checkstyle  |   1m 22s |  branch-2 passed  |
   | +1 :green_heart: |  spotbugs  |   2m 28s |  branch-2 passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   3m 55s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   1m 15s |  the patch passed  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  hadoopcheck  |  12m 47s |  Patch does not cause any 
errors with Hadoop 3.1.2 3.2.1.  |
   | +1 :green_heart: |  spotbugs  |   2m 34s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 12s |  The patch does not generate 
ASF License warnings.  |
   |  |   |  38m 51s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.13 Server=19.03.13 base: 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2595/2/artifact/yetus-general-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/2595 |
   | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti 
checkstyle |
   | uname | Linux a0f019176aa2 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | branch-2 / b155c62f38 |
   | Max. process+thread count | 84 (vs. ulimit of 12500) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2595/2/console
 |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) 
spotbugs=3.1.12 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #2385: HBASE-24950 Another solution for Splittable Meta

2020-10-29 Thread GitBox



Apache-HBase commented on pull request #2385:
URL: https://github.com/apache/hbase/pull/2385#issuecomment-718800705


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m 10s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  2s |  No case conflicting files 
found.  |
   | +0 :ok: |  prototool  |   0m  2s |  prototool was not available.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 22s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   3m 59s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   3m  5s |  master passed  |
   | +1 :green_heart: |  spotbugs  |   8m 29s |  master passed  |
   | -0 :warning: |  patch  |   1m 18s |  Used diff version of patch file. 
Binary files and potentially other changes not applied. Please rebase and 
squash commits if necessary.  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 11s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   3m 49s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   0m  9s |  The patch passed checkstyle 
in hbase-protocol-shaded  |
   | +1 :green_heart: |  checkstyle  |   0m 24s |  The patch passed checkstyle 
in hbase-common  |
   | +1 :green_heart: |  checkstyle  |   0m 31s |  hbase-client: The patch 
generated 0 new + 207 unchanged - 1 fixed = 207 total (was 208)  |
   | +1 :green_heart: |  checkstyle  |   0m 11s |  The patch passed checkstyle 
in hbase-zookeeper  |
   | +1 :green_heart: |  checkstyle  |   0m 11s |  The patch passed checkstyle 
in hbase-balancer  |
   | +1 :green_heart: |  checkstyle  |   1m 23s |  hbase-server: The patch 
generated 0 new + 673 unchanged - 8 fixed = 673 total (was 681)  |
   | +1 :green_heart: |  checkstyle  |   0m 16s |  The patch passed checkstyle 
in hbase-it  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  hadoopcheck  |  18m 50s |  Patch does not cause any 
errors with Hadoop 3.1.2 3.2.1 3.3.0.  |
   | +1 :green_heart: |  hbaseprotoc  |   3m 23s |  the patch passed  |
   | +1 :green_heart: |  spotbugs  |   9m 20s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   1m 12s |  The patch does not generate 
ASF License warnings.  |
   |  |   |  66m 22s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.13 Server=19.03.13 base: 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2385/14/artifact/yetus-general-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/2385 |
   | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti 
checkstyle cc hbaseprotoc prototool |
   | uname | Linux e684c3c6cf36 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / bb4a9d335f |
   | Max. process+thread count | 84 (vs. ulimit of 3) |
   | modules | C: hbase-protocol-shaded hbase-common hbase-client 
hbase-zookeeper hbase-balancer hbase-server hbase-it U: . |
   | Console output | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2385/14/console
 |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) 
spotbugs=3.1.12 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] apurtell commented on a change in pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close

2020-10-29 Thread GitBox



apurtell commented on a change in pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#discussion_r514337192



##
File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -8730,12 +8868,22 @@ public void startRegionOperation() throws IOException {
 
   @Override
   public void startRegionOperation(Operation op) throws IOException {
+boolean isInterruptableOp = false;
 switch (op) {
-  case GET:  // read operations
+  case GET:  // interruptible read operations
   case SCAN:
+isInterruptableOp = true;

Review comment:
   There are some checkInterrupts placed inside the scanner inner loops. 





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] apurtell commented on a change in pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close

2020-10-29 Thread GitBox



apurtell commented on a change in pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#discussion_r514337192



##
File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -8730,12 +8868,22 @@ public void startRegionOperation() throws IOException {
 
   @Override
   public void startRegionOperation(Operation op) throws IOException {
+boolean isInterruptableOp = false;
 switch (op) {
-  case GET:  // read operations
+  case GET:  // interruptible read operations
   case SCAN:
+isInterruptableOp = true;

Review comment:
   There are some checkInterrupts placed inside the scanner inner loops. 
Get, Exist, and Scan operations all share this path. Row mutation processor 
also checks for interrupts. I did not put checkInterrupt into the checkAndXXX 
operations because these are point ops that will complete quickly, but could if 
you feel this represents missing coverage. @Reidd 





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Commented] (HBASE-23369) Auto-close 'unknown' Regions reported as OPEN on RegionServers

2020-10-29 Thread Peter Somogyi (Jira)



[ 
https://issues.apache.org/jira/browse/HBASE-23369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17222961#comment-17222961
 ] 

Peter Somogyi commented on HBASE-23369:
---

[~zghao], [~stack]: What do you think about backporting this issue to 
branch-2.2? This Jira is marked as "Incompatible change" but this bugfix does 
not show any incompatibilities to me.

> Auto-close 'unknown' Regions reported as OPEN on RegionServers
> --
>
> Key: HBASE-23369
> URL: https://issues.apache.org/jira/browse/HBASE-23369
> Project: HBase
>  Issue Type: Bug
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.0
>
>
> In old days, if a RegionServer reported a variance that didn't agree w/ 
> Master view of the cluster, we'd kill the RegionServer.
> Lately, in tests that overrun a cluster, after a sustained high-load, Master 
> can start failing its updates against Meta (CallQueueTooBigException <= More 
> on this later). It then can lose proper accounting of all Region members. One 
> variant has a RegionServer reporting its list of open Regions to the Master 
> and the Master doesn't 'know' of a particular Region or the Master may know 
> the Region but expects it open on another RegionServer.
> Here is an example of how it looks each time RS reports:
> {code}
>  2019-12-03 07:07:00,757 WARN 
> org.apache.hadoop.hbase.master.assignment.AssignmentManager: No 
> t1,08f5c285,1573094375485.ee78a0c951c1c902d8f3f3912394a0e5. RegionStateNode 
> but reported ONLINE at server.example.org,16020,1575354666245 
> (inServerRegionList=false).
>  2019-12-03 07:07:03,793 WARN 
> org.apache.hadoop.hbase.master.assignment.AssignmentManager: No 
> t1,08f5c285,1573094375485.ee78a0c951c1c902d8f3f3912394a0e5. RegionStateNode 
> but reported ONLINE at server.example.org,16020,1575354666245 
> (inServerRegionList=false).
> {code}
> Will also show as an 'inconsistency' in the 'HBCK' tab on the Master UI.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] Apache-HBase commented on pull request #2582: HBASE-25187 Improve SizeCachedKV variants initialization

2020-10-29 Thread GitBox



Apache-HBase commented on pull request #2582:
URL: https://github.com/apache/hbase/pull/2582#issuecomment-718855909


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 31s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  5s |  Unprocessed flag(s): 
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 22s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   3m 35s |  master passed  |
   | +1 :green_heart: |  compile  |   1m 19s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   6m 32s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 59s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 17s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   3m 23s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 19s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 19s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   6m 34s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 59s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   1m 30s |  hbase-common in the patch passed.  
|
   | +1 :green_heart: |  unit  | 138m 57s |  hbase-server in the patch passed.  
|
   |  |   | 168m 34s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.13 Server=19.03.13 base: 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/2582 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux caa288a24d48 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / bb4a9d335f |
   | Default Java | 1.8.0_232 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/3/testReport/
 |
   | Max. process+thread count | 4640 (vs. ulimit of 3) |
   | modules | C: hbase-common hbase-server U: . |
   | Console output | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/3/console
 |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #2582: HBASE-25187 Improve SizeCachedKV variants initialization

2020-10-29 Thread GitBox



Apache-HBase commented on pull request #2582:
URL: https://github.com/apache/hbase/pull/2582#issuecomment-718857112


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m 17s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  5s |  Unprocessed flag(s): 
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 13s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   4m 16s |  master passed  |
   | +1 :green_heart: |  compile  |   1m 33s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   6m 49s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   1m  3s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 15s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   4m  3s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 31s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 31s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   6m 33s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   1m  2s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   1m 42s |  hbase-common in the patch passed.  
|
   | +1 :green_heart: |  unit  | 137m 32s |  hbase-server in the patch passed.  
|
   |  |   | 170m 10s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.13 Server=19.03.13 base: 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/2582 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux ae716252a73f 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 
17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / bb4a9d335f |
   | Default Java | 2020-01-14 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/3/testReport/
 |
   | Max. process+thread count | 3929 (vs. ulimit of 3) |
   | modules | C: hbase-common hbase-server U: . |
   | Console output | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2582/3/console
 |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #2595: HBASE-25090 CompactionConfiguration logs unrealistic store file sizes

2020-10-29 Thread GitBox



Apache-HBase commented on pull request #2595:
URL: https://github.com/apache/hbase/pull/2595#issuecomment-718860254


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 35s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  9s |  Unprocessed flag(s): 
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ branch-2 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   3m 51s |  branch-2 passed  |
   | +1 :green_heart: |  compile  |   1m  5s |  branch-2 passed  |
   | +1 :green_heart: |  shadedjars  |   6m 37s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 39s |  branch-2 passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   3m 56s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m  6s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m  6s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   6m 38s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 38s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  | 132m 34s |  hbase-server in the patch passed.  
|
   |  |   | 160m  3s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.13 Server=19.03.13 base: 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2595/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/2595 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 46077d83df30 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | branch-2 / b155c62f38 |
   | Default Java | 2020-01-14 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2595/2/testReport/
 |
   | Max. process+thread count | 4163 (vs. ulimit of 12500) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2595/2/console
 |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #2595: HBASE-25090 CompactionConfiguration logs unrealistic store file sizes

2020-10-29 Thread GitBox



Apache-HBase commented on pull request #2595:
URL: https://github.com/apache/hbase/pull/2595#issuecomment-718861846


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 36s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  7s |  Unprocessed flag(s): 
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ branch-2 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   3m 13s |  branch-2 passed  |
   | +1 :green_heart: |  compile  |   0m 57s |  branch-2 passed  |
   | +1 :green_heart: |  shadedjars  |   5m 53s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 36s |  branch-2 passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   3m 13s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 57s |  the patch passed  |
   | +1 :green_heart: |  javac  |   0m 57s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   6m  2s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 37s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  | 138m  2s |  hbase-server in the patch passed.  
|
   |  |   | 162m 30s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.13 Server=19.03.13 base: 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2595/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/2595 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux f1e9a0557611 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 
16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | branch-2 / b155c62f38 |
   | Default Java | 1.8.0_232 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2595/2/testReport/
 |
   | Max. process+thread count | 4289 (vs. ulimit of 12500) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2595/2/console
 |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Updated] (HBASE-24845) Git/Jira Release Audit: limit branches when building audit db

2020-10-29 Thread Nick Dimiduk (Jira)



 [ 
https://issues.apache.org/jira/browse/HBASE-24845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nick Dimiduk updated HBASE-24845:
-
Fix Version/s: 3.0.0-alpha-1

> Git/Jira Release Audit: limit branches when building audit db
> -
>
> Key: HBASE-24845
> URL: https://issues.apache.org/jira/browse/HBASE-24845
> Project: HBase
>  Issue Type: Task
>  Components: tooling
>Affects Versions: 3.0.0-alpha-1
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
> Fix For: 3.0.0-alpha-1
>
>
> Populating the audit database with release tag information from git is time 
> consuming. Until that's sorted out, give the user a flag for limiting which 
> branches they want to be reviewed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] ndimiduk merged pull request #2238: HBASE-24845 Git/Jira Release Audit: limit branches when building audit db

2020-10-29 Thread GitBox



ndimiduk merged pull request #2238:
URL: https://github.com/apache/hbase/pull/2238


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Updated] (HBASE-24845) Git/Jira Release Audit: limit branches when building audit db

2020-10-29 Thread Nick Dimiduk (Jira)



 [ 
https://issues.apache.org/jira/browse/HBASE-24845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nick Dimiduk updated HBASE-24845:
-
Component/s: community

> Git/Jira Release Audit: limit branches when building audit db
> -
>
> Key: HBASE-24845
> URL: https://issues.apache.org/jira/browse/HBASE-24845
> Project: HBase
>  Issue Type: Task
>  Components: community, tooling
>Affects Versions: 3.0.0-alpha-1
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
> Fix For: 3.0.0-alpha-1
>
>
> Populating the audit database with release tag information from git is time 
> consuming. Until that's sorted out, give the user a flag for limiting which 
> branches they want to be reviewed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (HBASE-24845) Git/Jira Release Audit: limit branches when building audit db

2020-10-29 Thread Nick Dimiduk (Jira)



 [ 
https://issues.apache.org/jira/browse/HBASE-24845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nick Dimiduk resolved HBASE-24845.
--
Resolution: Fixed

Thanks for the review [~apurtell]

> Git/Jira Release Audit: limit branches when building audit db
> -
>
> Key: HBASE-24845
> URL: https://issues.apache.org/jira/browse/HBASE-24845
> Project: HBase
>  Issue Type: Task
>  Components: community, tooling
>Affects Versions: 3.0.0-alpha-1
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
> Fix For: 3.0.0-alpha-1
>
>
> Populating the audit database with release tag information from git is time 
> consuming. Until that's sorted out, give the user a flag for limiting which 
> branches they want to be reviewed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] apurtell commented on a change in pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close

2020-10-29 Thread GitBox



apurtell commented on a change in pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#discussion_r514337192



##
File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -8730,12 +8868,22 @@ public void startRegionOperation() throws IOException {
 
   @Override
   public void startRegionOperation(Operation op) throws IOException {
+boolean isInterruptableOp = false;
 switch (op) {
-  case GET:  // read operations
+  case GET:  // interruptible read operations
   case SCAN:
+isInterruptableOp = true;

Review comment:
   There are some checkInterrupts placed inside the scanner inner loops. 
Get, Exist, and Scan operations all share this path. I did not put 
checkInterrupt into the checkAndXXX operations because these are point ops that 
will complete quickly, but could if you feel this represents missing coverage. 
@Reidd 
   
   On the write path there are checkInterrupts placed at points in 
doMiniBatchMutation to catch the cases where we are likely to run a long time, 
and the row mutation processor also checks for interrupts. Again not every 
operation has a check, like Increment or Append, where the unit of work is 
small, but we could add them there too if you feel this represents missing 
coverage. 
   
   My opinion is the long running cases are where the check is definitely worth 
it -- scanners on the read side, batch mutations on the write side -- and 
others are marginal (and therefore not included).





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] huaxiangsun commented on a change in pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…

2020-10-29 Thread GitBox



huaxiangsun commented on a change in pull request #2584:
URL: https://github.com/apache/hbase/pull/2584#discussion_r514396843



##
File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncNonMetaRegionLocator.java
##
@@ -577,6 +635,15 @@ private void removeLocationFromCache(HRegionLocation loc) {
   if (!canUpdateOnError(loc, oldLoc)) {
 return;
   }
+  // Tell metaReplicaSelector that the location is stale. It will create a 
stale entry
+  // with timestamp internally. Next time the client looks up the same 
location,
+  // it will pick a different meta replica region. For the current 
implementation,
+  // the metaReplicaId is not used, so the primary one is passed in.
+  if (this.metaReplicaMode == CatalogReplicaMode.LoadBalance) {
+// metaReplicaId is not used in simpleSelector, default to the primary 
one.
+metaReplicaSelector.onError(loc, RegionInfo.DEFAULT_REPLICA_ID);

Review comment:
   In my initial patch, I do keep "fromReplicaId" in TableCache to indicate 
which meta replica id the location comes from.
   However, this is not being used at this moment so I do not think it is a 
good idea to add more bytes which is not being used at this moment. How about 
passing an invalid replica id (say -1) to indicate it is not being filled at 
this moment?





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] apurtell commented on pull request #2576: HBASE-25212 [branch-1] Optionally abort requests in progress after deciding a region should close

2020-10-29 Thread GitBox



apurtell commented on pull request #2576:
URL: https://github.com/apache/hbase/pull/2576#issuecomment-718871541


   Unit test failure seems unrelated, probable flaky test
   
   [ERROR]   TestDurability.testDurability:132->verifyWALCount:242 
expected:<10> but was:<9>



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] huaxiangsun commented on a change in pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…

2020-10-29 Thread GitBox



huaxiangsun commented on a change in pull request #2584:
URL: https://github.com/apache/hbase/pull/2584#discussion_r514400512



##
File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/CatalogReplicaLoadBalanceReplicaSimpleSelector.java
##
@@ -0,0 +1,273 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.client;
+
+import static org.apache.hadoop.hbase.client.ConnectionUtils.isEmptyStopRow;
+import static org.apache.hadoop.hbase.util.Bytes.BYTES_COMPARATOR;
+import static org.apache.hadoop.hbase.util.ConcurrentMapUtils.computeIfAbsent;
+import java.io.IOException;
+import java.util.Iterator;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ConcurrentMap;
+import java.util.concurrent.ConcurrentNavigableMap;
+import java.util.concurrent.ConcurrentSkipListMap;
+import java.util.concurrent.ThreadLocalRandom;
+import org.apache.commons.lang3.builder.ToStringBuilder;
+import org.apache.hadoop.hbase.HRegionLocation;
+import org.apache.hadoop.hbase.ScheduledChore;
+import org.apache.hadoop.hbase.Stoppable;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * CatalogReplicaLoadBalanceReplicaSimpleSelector implements a simple catalog 
replica load balancing
+ * algorithm. It maintains a stale location cache for each table. Whenever 
client looks up location,
+ * it first check if the row is the stale location cache. If yes, the location 
from
+ * catalog replica is stale, it will go to the primary region to look up 
update-to-date location;
+ * otherwise, it will randomly pick up a replica region for lookup. When 
clients receive
+ * RegionNotServedException from region servers, it will add these region 
locations to the stale
+ * location cache. The stale cache will be cleaned up periodically by a chore.
+ *
+ * It follows a simple algorithm to choose a replica to go:
+ *
+ *  1. If there is no stale location entry for rows it looks up, it will 
randomly
+ * pick a replica region to do lookup.
+ *  2. If the location from the replica region is stale, client gets 
RegionNotServedException
+ * from region server, in this case, it will create 
StaleLocationCacheEntry in
+ * CatalogReplicaLoadBalanceReplicaSimpleSelector.
+ *  3. When client tries to do location lookup, it checks StaleLocationCache 
first for rows it
+ * tries to lookup, if entry exists, it will go with primary meta region 
to do lookup;
+ * otherwise, it will follow step 1.
+ *  4. A chore will periodically run to clean up cache entries in the 
StaleLocationCache.
+ */
+class CatalogReplicaLoadBalanceReplicaSimpleSelector implements
+  CatalogReplicaLoadBalanceReplicaSelector {
+  private static final Logger LOG =
+
LoggerFactory.getLogger(CatalogReplicaLoadBalanceReplicaSimpleSelector.class);
+  private final long STALE_CACHE_TIMEOUT_IN_MILLISECONDS = 3000; // 3 seconds
+  private final int STALE_CACHE_CLEAN_CHORE_INTERVAL = 1500; // 1.5 seconds
+
+  /**
+   * StaleLocationCacheEntry is the entry when a stale location is reported by 
an client.
+   */
+  private static final class StaleLocationCacheEntry {
+// replica id where the stale location comes from.
+private int fromReplicaId;
+
+// timestamp in milliseconds
+private long timestamp;
+
+private byte[] endKey;
+
+StaleLocationCacheEntry(final int metaReplicaId, final byte[] endKey) {
+  this.fromReplicaId = metaReplicaId;
+  this.endKey = endKey;
+  timestamp = System.currentTimeMillis();
+}
+
+public byte[] getEndKey() {
+  return this.endKey;
+}
+
+public int getFromReplicaId() {
+  return this.fromReplicaId;
+}
+public long getTimestamp() {
+  return this.timestamp;
+}
+
+@Override
+public String toString() {
+  return new ToStringBuilder(this)
+.append("endKey", endKey)
+.append("fromReplicaId", fromReplicaId)
+.append("timestamp", timestamp)
+.toString();
+}
+  }
+
+  private static final class StaleTableCache {
+private final ConcurrentNavigableMap 
cache =
+

[GitHub] [hbase] huaxiangsun commented on a change in pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…

2020-10-29 Thread GitBox



huaxiangsun commented on a change in pull request #2584:
URL: https://github.com/apache/hbase/pull/2584#discussion_r514400318



##
File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/CatalogReplicaLoadBalanceReplicaSelector.java
##
@@ -0,0 +1,46 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.client;
+
+import org.apache.hadoop.hbase.HRegionLocation;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.yetus.audience.InterfaceAudience;
+
+/**
+ * A Catalog replica selector decides which catalog replica to go for read 
requests.
+ */
+@InterfaceAudience.Private
+public interface CatalogReplicaLoadBalanceReplicaSelector {
+
+  /**
+   *  This method is called upon when input location is stale.
+   * @param loc location which is stale.
+   * @param fromReplicaId which replica the stale location comes from.
+   */
+  void notifyOnError(HRegionLocation loc, int fromReplicaId);

Review comment:
   My thoughts was that since filtering logic is done already, it is not 
necessary to do for a second time in selector implementation. Some of the meta 
data such as "fromMetaReplicaId" needs to come from TableCache (in the future 
implementation), so somehow, upper layer is already aware of 
RegionMovedException and NotServingRegionException. 
   Will keep it as it is for now.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] apurtell commented on a change in pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close

2020-10-29 Thread GitBox



apurtell commented on a change in pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#discussion_r514400489



##
File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -8777,23 +8936,36 @@ public void closeRegionOperation(Operation operation) 
throws IOException {
 if (operation == Operation.SNAPSHOT) {
   stores.values().forEach(HStore::postSnapshotOperation);
 }
+regionLockHolders.remove(Thread.currentThread());
 lock.readLock().unlock();
 if (coprocessorHost != null) {
   coprocessorHost.postCloseRegionOperation(operation);
 }
   }
 
+  /**
+   * If a handler thread is eligible for interrupt, make it ineligible. Should 
be paired
+   * with {{@link #enableInterrupts()}.
+   */
+  protected void disableInterrupts() {
+regionLockHolders.computeIfPresent(Thread.currentThread(), (t,b) -> false);
+  }
+
+  /**
+   * If a handler thread was made ineligible for interrupt via {{@link 
#disableInterrupts()},
+   * make it eligible again. No-op if interrupts are already enabled.
+   */
+  protected void enableInterrupts() {
+regionLockHolders.computeIfPresent(Thread.currentThread(), (t,b) -> true);
+  }
+
   /**
* This method needs to be called before any public call that reads or
* modifies stores in bulk. It has to be called just before a try.
* #closeBulkRegionOperation needs to be called in the try's finally block
* Acquires a writelock and checks if the region is closing or closed.
-   * @throws NotServingRegionException when the region is closing or closed

Review comment:
   I can put them back but I believe this was done in response to feedback 
from another reviewer





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] apurtell commented on a change in pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close

2020-10-29 Thread GitBox



apurtell commented on a change in pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#discussion_r514400950



##
File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -9000,6 +9173,51 @@ public long getReadPoint() {
 return getReadPoint(IsolationLevel.READ_COMMITTED);
   }
 
+  /**
+   * Interrupt any region options that have acquired the region lock via
+   * {@link 
#startRegionOperation(org.apache.hadoop.hbase.regionserver.Region.Operation)},
+   * or {@link #startBulkRegionOperation(boolean)}.
+   */
+  private void interruptRegionOperations() {
+for (Map.Entry entry: regionLockHolders.entrySet()) {
+  // An entry in this map will have a boolean value indicating if it is 
currently
+  // eligible for interrupt; if so, we should interrupt it.
+  if (entry.getValue().booleanValue()) {

Review comment:
   The generated code will be the same. I opted for this to be pedantic in 
the code. It should be fine, right?





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] huaxiangsun commented on a change in pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…

2020-10-29 Thread GitBox



huaxiangsun commented on a change in pull request #2584:
URL: https://github.com/apache/hbase/pull/2584#discussion_r514401694



##
File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncNonMetaRegionLocator.java
##
@@ -577,6 +635,15 @@ private void removeLocationFromCache(HRegionLocation loc) {
   if (!canUpdateOnError(loc, oldLoc)) {
 return;
   }
+  // Tell metaReplicaSelector that the location is stale. It will create a 
stale entry
+  // with timestamp internally. Next time the client looks up the same 
location,
+  // it will pick a different meta replica region. For the current 
implementation,
+  // the metaReplicaId is not used, so the primary one is passed in.
+  if (this.metaReplicaMode == CatalogReplicaMode.LoadBalance) {
+// metaReplicaId is not used in simpleSelector, default to the primary 
one.
+metaReplicaSelector.onError(loc, RegionInfo.DEFAULT_REPLICA_ID);

Review comment:
   What do you think to bring my initial implementation (fromReplicaId) 
back so fromReplicaId is going to be a valid one (not faked one)?





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] apurtell commented on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close

2020-10-29 Thread GitBox



apurtell commented on pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#issuecomment-718875966


   @Apache9 I am having trouble responding to your comment above. 
   
   > After reading this discussion several times, I think the reason we do not 
want to niterrupte a WAL sync is that it may lead to a region server abort?
   > I would say this is not the case here. I checked the code again, the 
actual sync is done in the disruptor thread, in the rpc thread we just block on 
a SyncFuture(as Andrew mentioned above), the interruption on the rpc thread 
will just lead to an IOException tp client, the actual sync operation will not 
be interrupted so we are safe.
   > So I do not think we need to disable interrupts here?
   
   You are correct about the SyncFuture.
   
   My concern is different. I am concerned about this case:
   - We start the WAL append
   - SyncFuture is interrupted, client gets an exception, client thinks the 
mutation failed
   - WAL sync completes, mutation is in WAL, mutation is applied (e.g. during 
WAL replay for some reason)
   
   So we have a case where the client's understanding of what happened is 
incorrect. I would prefer to prevent interrupts from messing with our ability 
to return success indication to the client once we are far enough along the 
mutation code path that we are writing to the WAL. At that point it is a point 
of no return, what happens in the WAL and the client's understanding of what 
happend should be in sync. If the WAL sync fails the client should get an 
exception. If the WAL sync succeeds the client should see success. Exception 
would be the case where the server is shutting down, that we can't control. We 
can control when interrupts happen so can be considerate. 



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] apurtell edited a comment on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close

2020-10-29 Thread GitBox



apurtell edited a comment on pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#issuecomment-718875966


   @Apache9 I am having trouble responding to your comment above. 
   
   > After reading this discussion several times, I think the reason we do not 
want to niterrupte a WAL sync is that it may lead to a region server abort?
   > I would say this is not the case here. I checked the code again, the 
actual sync is done in the disruptor thread, in the rpc thread we just block on 
a SyncFuture(as Andrew mentioned above), the interruption on the rpc thread 
will just lead to an IOException tp client, the actual sync operation will not 
be interrupted so we are safe.
   > So I do not think we need to disable interrupts here?
   
   You are correct about the SyncFuture.
   
   My concern is different. I am concerned about this case:
   - We start the WAL append
   - SyncFuture is interrupted, client gets an exception, client thinks the 
mutation failed
   - WAL sync completes, mutation is in WAL, mutation is applied (e.g. during 
WAL replay for some reason)
   
   So we have a case where the client's understanding of what happened is 
incorrect. I would prefer to prevent interrupts from messing with our ability 
to return success indication to the client once we are far enough along the 
mutation code path that we are writing to the WAL. At that point it is a point 
of no return, what happens in the WAL and the client's understanding of what 
happend should be in sync. If the WAL sync fails the client should get an 
exception. If the WAL sync succeeds the client should see success. Exception 
would be the case where the server is shutting down, that we can't control. We 
can control when interrupts happen so can be considerate. 
   
   I can add the above as a code comment next to the disableInterrupt() call, 
would that help?



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] apurtell edited a comment on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close

2020-10-29 Thread GitBox



apurtell edited a comment on pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#issuecomment-718875966


   @Apache9 I am having trouble responding to your comment above. 
   
   > After reading this discussion several times, I think the reason we do not 
want to niterrupte a WAL sync is that it may lead to a region server abort?
   > I would say this is not the case here. I checked the code again, the 
actual sync is done in the disruptor thread, in the rpc thread we just block on 
a SyncFuture(as Andrew mentioned above), the interruption on the rpc thread 
will just lead to an IOException tp client, the actual sync operation will not 
be interrupted so we are safe.
   > So I do not think we need to disable interrupts here?
   
   You are correct about the SyncFuture.
   
   My concern is different. I am concerned about this case:
   - We start the WAL append
   - SyncFuture is interrupted, client gets an exception, client thinks the 
mutation failed
   - WAL sync completes, mutation is in WAL, mutation is applied (e.g. during 
WAL replay for some reason)
   
   So we have a case where the client's understanding of what happened is 
incorrect. I would prefer to prevent interrupts from messing with our ability 
to return success indication to the client once we are far enough along the 
mutation code path that we are writing to the WAL. At that point it is a point 
of no return, what happens in the WAL and the client's understanding of what 
happend should be in sync. If the WAL sync fails the client should get an 
exception. If the WAL sync succeeds the client should see success. Exception 
would be the case where the server is shutting down, that we can't control. We 
can control when interrupts happen so can be considerate. 
   
   I can add the above as a code comment next to the disableInterrupt() call, 
would that help?
   
   Here is another case of concern:
   
   - We start the WAL append
   - SyncFuture is interrupted, memstore is not updated, client gets an 
exception. Unless the WAL is replayed the local cluster will not have the 
mutation applied. 
   - WAL was written, though, so change will be replicated to a remote cluster. 
Remote cluster applies the edit. Remote cluster is out of sync with local 
cluster.
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] apurtell edited a comment on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close

2020-10-29 Thread GitBox



apurtell edited a comment on pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#issuecomment-718875966


   @Apache9 I am having trouble responding to your comment above. 
   
   > After reading this discussion several times, I think the reason we do not 
want to niterrupte a WAL sync is that it may lead to a region server abort?
   > I would say this is not the case here. I checked the code again, the 
actual sync is done in the disruptor thread, in the rpc thread we just block on 
a SyncFuture(as Andrew mentioned above), the interruption on the rpc thread 
will just lead to an IOException tp client, the actual sync operation will not 
be interrupted so we are safe.
   > So I do not think we need to disable interrupts here?
   
   You are correct about the SyncFuture.
   
   My concern is different. I am concerned about this case:
   - We start the WAL append
   - SyncFuture is interrupted, client gets an exception, client thinks the 
mutation failed
   - WAL sync completes, mutation is in WAL, mutation is applied (e.g. during 
WAL replay for some reason)
   
   So we have a case where the client's understanding of what happened is 
incorrect. I would prefer to prevent interrupts from messing with our ability 
to return success indication to the client once we are far enough along the 
mutation code path that we are writing to the WAL. At that point it is a point 
of no return, what happens in the WAL and the client's understanding of what 
happend should be in sync. If the WAL sync fails the client should get an 
exception. If the WAL sync succeeds the client should see success. Exception 
would be the case where the server is shutting down, that we can't control. We 
can control when interrupts happen so can be considerate. 
   
   I can add the above as a code comment next to the disableInterrupt() call, 
would that help?
   
   Here is another case of concern:
   
   - We start the WAL append
   - SyncFuture is interrupted, memstore is not updated, client gets an 
exception. Unless the WAL is replayed the local cluster will not have the 
mutation applied. 
   - WAL was written, though, so change will be replicated to a remote cluster. 
Remote cluster applies the edit. Remote cluster is out of sync with local 
cluster.
   
   **The disableInterrupt/enableInterrupt pair puts both the WAL append and 
memstore update into a critical section** with respect to this proposed change 
to close handling. 
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] apurtell edited a comment on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close

2020-10-29 Thread GitBox



apurtell edited a comment on pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#issuecomment-718875966


   @Apache9 I am having trouble responding to your comment above. 
   
   > After reading this discussion several times, I think the reason we do not 
want to niterrupte a WAL sync is that it may lead to a region server abort?
   > I would say this is not the case here. I checked the code again, the 
actual sync is done in the disruptor thread, in the rpc thread we just block on 
a SyncFuture(as Andrew mentioned above), the interruption on the rpc thread 
will just lead to an IOException tp client, the actual sync operation will not 
be interrupted so we are safe.
   > So I do not think we need to disable interrupts here?
   
   You are correct about the SyncFuture.
   
   My concern is different. I am concerned about this case:
   - We start the WAL append
   - SyncFuture is interrupted, client gets an exception, client thinks the 
mutation failed
   - WAL sync completes, mutation is in WAL, mutation is applied (e.g. during 
WAL replay for some reason)
   
   So we have a case where the client's understanding of what happened is 
incorrect. I would prefer to prevent interrupts from messing with our ability 
to return success indication to the client once we are far enough along the 
mutation code path that we are writing to the WAL. At that point it is a point 
of no return, what happens in the WAL and the client's understanding of what 
happend should be in sync. If the WAL sync fails the client should get an 
exception. If the WAL sync succeeds the client should see success. Exception 
would be the case where the server is shutting down, that we can't control. We 
can control when interrupts happen so can be considerate. 
   
   I can add the above as a code comment next to the disableInterrupt() call, 
would that help?
   
   Here is another case of concern:
   
   - We start the WAL append
   - SyncFuture is interrupted, memstore is not updated, client gets an 
exception. Unless the WAL is replayed the local cluster will not have the 
mutation applied. 
   - WAL was written, though, so change will be replicated to a remote cluster. 
Remote cluster applies the edit. Remote cluster is out of sync with local 
cluster.
   
   The disableInterrupt/enableInterrupt pair combines the WAL append and 
memstore update into a single protected operation with respect to this proposed 
change.
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #2385: HBASE-24950 Another solution for Splittable Meta

2020-10-29 Thread GitBox



Apache-HBase commented on pull request #2385:
URL: https://github.com/apache/hbase/pull/2385#issuecomment-718880483


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 33s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  6s |  Unprocessed flag(s): 
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 30s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   3m 32s |  master passed  |
   | +1 :green_heart: |  compile  |   3m 32s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   6m 29s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   2m 20s |  master passed  |
   | -0 :warning: |  patch  |  10m  2s |  Used diff version of patch file. 
Binary files and potentially other changes not applied. Please rebase and 
squash commits if necessary.  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 15s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   3m 32s |  the patch passed  |
   | +1 :green_heart: |  compile  |   3m 34s |  the patch passed  |
   | +1 :green_heart: |  javac  |   3m 34s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   6m 34s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 14s |  hbase-protocol-shaded in the 
patch passed.  |
   | +1 :green_heart: |  javadoc  |   0m 21s |  hbase-common in the patch 
passed.  |
   | +1 :green_heart: |  javadoc  |   0m 24s |  hbase-client in the patch 
passed.  |
   | +1 :green_heart: |  javadoc  |   0m 16s |  hbase-zookeeper in the patch 
passed.  |
   | +1 :green_heart: |  javadoc  |   0m 15s |  hbase-balancer generated 0 new 
+ 0 unchanged - 1 fixed = 0 total (was 1)  |
   | +1 :green_heart: |  javadoc  |   0m 34s |  hbase-server in the patch 
passed.  |
   | +1 :green_heart: |  javadoc  |   0m 14s |  hbase-it in the patch passed.  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   0m 46s |  hbase-protocol-shaded in the patch 
passed.  |
   | +1 :green_heart: |  unit  |   1m 30s |  hbase-common in the patch passed.  
|
   | +1 :green_heart: |  unit  |   1m  2s |  hbase-client in the patch passed.  
|
   | +1 :green_heart: |  unit  |   0m 42s |  hbase-zookeeper in the patch 
passed.  |
   | +1 :green_heart: |  unit  |   0m 22s |  hbase-balancer in the patch 
passed.  |
   | -1 :x: |  unit  | 145m 10s |  hbase-server in the patch failed.  |
   | +1 :green_heart: |  unit  |   1m 13s |  hbase-it in the patch passed.  |
   |  |   | 187m 47s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.13 Server=19.03.13 base: 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2385/14/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/2385 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux f82ec1c629fb 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 
16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / bb4a9d335f |
   | Default Java | 1.8.0_232 |
   | unit | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2385/14/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2385/14/testReport/
 |
   | Max. process+thread count | 4802 (vs. ulimit of 3) |
   | modules | C: hbase-protocol-shaded hbase-common hbase-client 
hbase-zookeeper hbase-balancer hbase-server hbase-it U: . |
   | Console output | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2385/14/console
 |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] apurtell edited a comment on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close

2020-10-29 Thread GitBox



apurtell edited a comment on pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#issuecomment-718875966


   @Apache9 I am having trouble responding to your comment above. 
   
   > After reading this discussion several times, I think the reason we do not 
want to niterrupte a WAL sync is that it may lead to a region server abort?
   > I would say this is not the case here. I checked the code again, the 
actual sync is done in the disruptor thread, in the rpc thread we just block on 
a SyncFuture(as Andrew mentioned above), the interruption on the rpc thread 
will just lead to an IOException tp client, the actual sync operation will not 
be interrupted so we are safe.
   > So I do not think we need to disable interrupts here?
   
   You are correct about the SyncFuture.
   
   My concern is different. I am concerned about this case:
   - We start the WAL append
   - SyncFuture is interrupted, client gets an exception, client thinks the 
mutation failed
   - WAL sync completes, mutation is in WAL, mutation is applied (e.g. during 
WAL replay for some reason)
   
   So we have a case where the client's understanding of what happened is 
incorrect. I would prefer to prevent interrupts from messing with our ability 
to return success indication to the client once we are far enough along the 
mutation code path that we are writing to the WAL. At that point it is a point 
of no return, what happens in the WAL and the client's understanding of what 
happend should be in sync. If the WAL sync fails the client should get an 
exception. If the WAL sync succeeds the client should see success. Exception 
would be the case where the server is shutting down, or the existing timeout 
case, those we can't control. We can control when interrupts happen, so we can 
be considerate. disableInterrupt/enableInterrupt is consideration for the 
client.
   
   I can add the above as a code comment next to the disableInterrupt() call, 
would that help?
   
   Here is another case of concern:
   
   - We start the WAL append
   - SyncFuture is interrupted, memstore is not updated, client gets an 
exception. Unless the WAL is replayed the local cluster will not have the 
mutation applied. 
   - WAL was written, though, so change will be replicated to a remote cluster. 
Remote cluster applies the edit. Remote cluster is out of sync with local 
cluster.
   
   The disableInterrupt/enableInterrupt pair combines the WAL append and 
memstore update into a single protected operation with respect to this proposed 
change.
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] apurtell edited a comment on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close

2020-10-29 Thread GitBox



apurtell edited a comment on pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#issuecomment-718875966


   @Apache9 I am having trouble responding to your comment above. 
   
   > After reading this discussion several times, I think the reason we do not 
want to niterrupte a WAL sync is that it may lead to a region server abort?
   > I would say this is not the case here. I checked the code again, the 
actual sync is done in the disruptor thread, in the rpc thread we just block on 
a SyncFuture(as Andrew mentioned above), the interruption on the rpc thread 
will just lead to an IOException tp client, the actual sync operation will not 
be interrupted so we are safe.
   > So I do not think we need to disable interrupts here?
   
   You are correct about the SyncFuture.
   
   Initially my thinking was the same as yours. 
   
   At some point I became concerned about this case, though:
   - We start the WAL append
   - SyncFuture is interrupted, client gets an exception, client thinks the 
mutation failed
   - WAL sync completes, mutation is in WAL, mutation is applied (e.g. during 
WAL replay for some reason)
   
   So we have a case where the client's understanding of what happened is 
incorrect. I would prefer to prevent interrupts from messing with our ability 
to return success indication to the client once we are far enough along the 
mutation code path that we are writing to the WAL. At that point it is a point 
of no return, what happens in the WAL and the client's understanding of what 
happend should be in sync. If the WAL sync fails the client should get an 
exception. If the WAL sync succeeds the client should see success. Exception 
would be the case where the server is shutting down, or the existing timeout 
case, those we can't control. We can control when interrupts happen, so we can 
be considerate. disableInterrupt/enableInterrupt is consideration for the 
client.
   
   I can add the above as a code comment next to the disableInterrupt() call, 
would that help?
   
   Here is another case of concern:
   
   - We start the WAL append
   - SyncFuture is interrupted, memstore is not updated, client gets an 
exception. Unless the WAL is replayed the local cluster will not have the 
mutation applied. 
   - WAL was written, though, so change will be replicated to a remote cluster. 
Remote cluster applies the edit. Remote cluster is out of sync with local 
cluster.
   
   The disableInterrupt/enableInterrupt pair combines the WAL append and 
memstore update into a single protected operation with respect to this proposed 
change.
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] apurtell edited a comment on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close

2020-10-29 Thread GitBox



apurtell edited a comment on pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#issuecomment-718875966


   @Apache9 I am having trouble responding to your comment above. 
   
   > After reading this discussion several times, I think the reason we do not 
want to niterrupte a WAL sync is that it may lead to a region server abort?
   > I would say this is not the case here. I checked the code again, the 
actual sync is done in the disruptor thread, in the rpc thread we just block on 
a SyncFuture(as Andrew mentioned above), the interruption on the rpc thread 
will just lead to an IOException tp client, the actual sync operation will not 
be interrupted so we are safe.
   > So I do not think we need to disable interrupts here?
   
   You are correct about the SyncFuture.
   
   Initially my thinking was the same as yours. 
   
   At some point I became concerned about this case, though:
   - We start the WAL append
   - SyncFuture is interrupted, client gets an exception, client thinks the 
mutation failed
   - WALedit is actually applied to the WAL by the disruptor, so the mutation 
is included in the WAL, and so we are at risk of the failed from client 
perspective mutation being applied applied during WAL replay for some reason
   
   So we have a case where the client's understanding of what happened is 
incorrect. I would prefer to prevent interrupts from messing with our ability 
to return success indication to the client once we are far enough along the 
mutation code path that we are writing to the WAL. At that point it is a point 
of no return, what happens in the WAL and the client's understanding of what 
happend should be in sync. If the WAL sync fails the client should get an 
exception. If the WAL sync succeeds the client should see success. Exception 
would be the case where the server is shutting down, or the existing timeout 
case, those we can't control. We can control when interrupts happen, so we can 
be considerate. disableInterrupt/enableInterrupt is consideration for the 
client.
   
   I can add the above as a code comment next to the disableInterrupt() call, 
would that help?
   
   Here is another case of concern:
   
   - We start the WAL append
   - SyncFuture is interrupted, memstore is not updated, client gets an 
exception. Unless the WAL is replayed the local cluster A will not have the 
mutation applied. 
   - WALedit is actually applied to the WAL by the disruptor, so the mutation 
is included in the WAL, and so it is shipped to the remote cluster, and is 
applied at the remote cluster B. 
   - Now the data in cluster A and B are out of sync.
   
   The disableInterrupt/enableInterrupt pair combines the WAL append and 
memstore update into a single protected operation with respect to this proposed 
change.
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] apurtell edited a comment on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close

2020-10-29 Thread GitBox



apurtell edited a comment on pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#issuecomment-718875966


   @Apache9 I am having trouble responding to your comment above. 
   
   > After reading this discussion several times, I think the reason we do not 
want to niterrupte a WAL sync is that it may lead to a region server abort?
   > I would say this is not the case here. I checked the code again, the 
actual sync is done in the disruptor thread, in the rpc thread we just block on 
a SyncFuture(as Andrew mentioned above), the interruption on the rpc thread 
will just lead to an IOException tp client, the actual sync operation will not 
be interrupted so we are safe.
   > So I do not think we need to disable interrupts here?
   
   You are correct about the SyncFuture.
   
   Initially my thinking was the same as yours. 
   
   At some point I became concerned about this case, though:
   - We start the WAL append
   - SyncFuture is interrupted, client gets an exception, client thinks the 
mutation failed
   - WALedit is actually applied to the WAL by the disruptor, so the mutation 
is included in the WAL, and so we are at risk of the failed from client 
perspective mutation being applied during WAL replay for some reason
   
   So we have a case where the client's understanding of what happened is 
incorrect. I would prefer to prevent interrupts from messing with our ability 
to return success indication to the client once we are far enough along the 
mutation code path that we are writing to the WAL. At that point it is a point 
of no return, what happens in the WAL and the client's understanding of what 
happend should be in sync. If the WAL sync fails the client should get an 
exception. If the WAL sync succeeds the client should see success. Exception 
would be the case where the server is shutting down, or the existing timeout 
case, those we can't control. We can control when interrupts happen, so we can 
be considerate. disableInterrupt/enableInterrupt is consideration for the 
client.
   
   I can add the above as a code comment next to the disableInterrupt() call, 
would that help?
   
   Here is another case of concern:
   
   - We start the WAL append
   - SyncFuture is interrupted, memstore is not updated, client gets an 
exception. Unless the WAL is replayed the local cluster A will not have the 
mutation applied. 
   - WALedit is actually applied to the WAL by the disruptor, so the mutation 
is included in the WAL, and so it is shipped to the remote cluster, and is 
applied at the remote cluster B. 
   - Now the data in cluster A and B are out of sync.
   
   The disableInterrupt/enableInterrupt pair combines the WAL append and 
memstore update into a single protected operation with respect to this proposed 
change.
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] apurtell edited a comment on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close

2020-10-29 Thread GitBox



apurtell edited a comment on pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#issuecomment-718875966


   @Apache9 I am having trouble responding to your comment above. 
   
   > After reading this discussion several times, I think the reason we do not 
want to niterrupte a WAL sync is that it may lead to a region server abort?
   > I would say this is not the case here. I checked the code again, the 
actual sync is done in the disruptor thread, in the rpc thread we just block on 
a SyncFuture(as Andrew mentioned above), the interruption on the rpc thread 
will just lead to an IOException tp client, the actual sync operation will not 
be interrupted so we are safe.
   > So I do not think we need to disable interrupts here?
   
   You are correct about the SyncFuture.
   
   Initially my thinking was the same as yours. 
   
   At some point I became concerned about this case, though:
   - We start the WAL append
   - SyncFuture is interrupted, client gets an exception, client thinks the 
mutation failed
   - WALedit is actually applied to the WAL by the disruptor, so the mutation 
is included in the WAL, and so we are at risk of the failed from client 
perspective mutation being applied during WAL replay for some reason
   
   So we have a case where the client's understanding of what happened is 
incorrect. What happens in the WAL and the client's understanding of what 
happend should be in sync. If the WAL commit fails the client should get an 
exception. If the WAL commit succeeds the client should see success. Whether we 
interrupt or not is our option. disableInterrupt/enableInterrupt here is 
consideration for the client.
   
   I can add the above as a code comment next to the disableInterrupt() call, 
would that help?
   
   Here is another case of concern:
   
   - We start the WAL append
   - SyncFuture is interrupted, memstore is not updated, client gets an 
exception. Unless the WAL is replayed the local cluster A will not have the 
mutation applied. 
   - WALedit is actually applied to the WAL by the disruptor, so the mutation 
is included in the WAL, and so it is shipped to the remote cluster, and is 
applied at the remote cluster B. 
   - Now the data in cluster A and B are out of sync.
   
   The disableInterrupt/enableInterrupt pair combines the WAL append and 
memstore update into a single protected operation with respect to this proposed 
change.
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] apurtell edited a comment on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close

2020-10-29 Thread GitBox



apurtell edited a comment on pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#issuecomment-718875966


   @Apache9 I am having trouble responding to your comment above. 
   
   > After reading this discussion several times, I think the reason we do not 
want to niterrupte a WAL sync is that it may lead to a region server abort?
   > I would say this is not the case here. I checked the code again, the 
actual sync is done in the disruptor thread, in the rpc thread we just block on 
a SyncFuture(as Andrew mentioned above), the interruption on the rpc thread 
will just lead to an IOException tp client, the actual sync operation will not 
be interrupted so we are safe.
   > So I do not think we need to disable interrupts here?
   
   You are correct about the SyncFuture.
   
   Initially my thinking was the same as yours. 
   
   At some point I became concerned about this case, though:
   - We start the WAL append
   - SyncFuture is interrupted, client gets an exception, client thinks the 
mutation failed
   - WALedit is actually applied to the WAL by the disruptor, so the mutation 
is included in the WAL, and so we are at risk of the failed from client 
perspective mutation being applied during WAL replay for some reason
   
   So we have a case where the client's understanding of what happened is 
incorrect. What happens in the WAL and the client's understanding of what 
happend should be in sync. If the WAL commit fails the client should get an 
exception. If the WAL commit succeeds the client should see success. Whether we 
interrupt or not is our option. disableInterrupt/enableInterrupt here is 
consideration for the client.
   
   I can add the above as a code comment next to the disableInterrupt() call, 
would that help?
   
   Here is another case of concern:
   
   - We start the WAL append
   - SyncFuture is interrupted, memstore is not updated, client gets an 
exception. Unless the WAL is replayed the local cluster A will not have the 
mutation applied. 
   - WALedit is actually applied to the WAL by the disruptor, so the mutation 
is included in the WAL, and so it is shipped to the remote cluster, and is 
applied at the remote cluster B. 
   - Now the data in cluster A and B are out of sync.
   
   The disableInterrupt/enableInterrupt pair combines the WAL append and 
memstore update into a single protected operation , so this error case cannot 
happen.
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] apurtell commented on pull request #2594: HBASE-25227 [branch-1] Cast in UnsafeAccess to avoid Java 11 runtime issue

2020-10-29 Thread GitBox



apurtell commented on pull request #2594:
URL: https://github.com/apache/hbase/pull/2594#issuecomment-718889835


   > directly changing BYTE_ARRAY_BASE_OFFSET's type to long
   
   Ok



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] apurtell edited a comment on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close

2020-10-29 Thread GitBox



apurtell edited a comment on pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#issuecomment-718875966


   @Apache9 I am having trouble responding to your comment above. 
   
   > After reading this discussion several times, I think the reason we do not 
want to niterrupte a WAL sync is that it may lead to a region server abort?
   > I would say this is not the case here. I checked the code again, the 
actual sync is done in the disruptor thread, in the rpc thread we just block on 
a SyncFuture(as Andrew mentioned above), the interruption on the rpc thread 
will just lead to an IOException tp client, the actual sync operation will not 
be interrupted so we are safe.
   > So I do not think we need to disable interrupts here?
   
   You are correct about the SyncFuture.
   
   Initially my thinking was the same as yours. 
   
   At some point I became concerned about this case, though:
   - We start the WAL append
   - SyncFuture is interrupted, client gets an exception, client thinks the 
mutation failed
   - WALedit is actually applied to the WAL by the disruptor, so the mutation 
is included in the WAL, and so we are at risk of the failed from client 
perspective mutation being applied during WAL replay for some reason
   
   So we have a case where the client's understanding of what happened is 
incorrect. What happens in the WAL and the client's understanding of what 
happend should be in sync. If the WAL commit fails the client should get an 
exception. If the WAL commit succeeds the client should see success. Whether we 
interrupt or not is our option. disableInterrupt/enableInterrupt here is 
consideration for the client.
   
   I can add the above as a code comment next to the disableInterrupt() call, 
would that help?
   
   Here is another case of concern:
   
   - We start the WAL append
   - SyncFuture is interrupted, memstore is not updated, client gets an 
exception. Unless the WAL is replayed the local cluster A will not have the 
mutation applied. 
   - WALedit is actually applied to the WAL by the disruptor, so the mutation 
is included in the WAL, and so it is shipped to the remote cluster, and is 
applied at the remote cluster B. 
   - Now the data in cluster A and B are out of sync.
   
   The disableInterrupt/enableInterrupt pair combines the WAL append and 
memstore update into a single protected operation , so this error case will not 
happen because of the interrupts I am adding with this patch.
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] bharathv commented on a change in pull request #2591: HBASE-24859: Optimize in-memory representation of HBase map reduce table splits

2020-10-29 Thread GitBox



bharathv commented on a change in pull request #2591:
URL: https://github.com/apache/hbase/pull/2591#discussion_r514426736



##
File path: 
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScanBase.java
##
@@ -271,6 +272,11 @@ public void testNumOfSplits(int splitsPerRegion, int 
expectedNumOfSplits) throws
 TableInputFormat tif = new TableInputFormat();
 tif.setConf(job.getConfiguration());
 List splits = tif.getSplits(job);
+for (InputSplit split : splits) {
+  TableSplit tableSplit = (TableSplit) split;
+  Assert.assertEquals(tableSplit.getScan().getStartRow(), 
HConstants.EMPTY_START_ROW);

Review comment:
   Ya my point is, that is a weak check. You wouldn't know if that scan is 
an empty scan or a default scan. Change the code to something like this and you 
can see there is a bug ...
   
```
   for (InputSplit split : splits) {
 TableSplit tableSplit = (TableSplit) split;
 assertTrue(tableSplit.getScanAsString().isEmpty());
 }
   
   In table split code, add this...
   
public String getScanAsString() {
   return scan;
 }
   ```
   
   Run this test
   
   ```
 @Test
 public void testGetSplits() throws IOException, InterruptedException, 
ClassNotFoundException {
   testNumOfSplits(1, 26);
   testNumOfSplits(3, 78);  <
 }
   ```
   
   If number of splits per region is > 1, your patch didn't handle the 
TableSplit constructor in this call createNInputSplitsUniform(). Your test 
still passes because you are asserting on a default Scan object and not the 
fact that it shouldn't be set in the first place.

##
File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSplit.java
##
@@ -184,9 +184,24 @@ public TableSplit(final byte [] tableName, byte[] 
startRow, byte[] endRow,
* @param startRow The start row of the split.
* @param endRow The end row of the split.
* @param location The location of the region.
+   * @param encodedRegionName The region ID.
+   * @param length Size of region in bytes
*/
   public TableSplit(TableName tableName, byte[] startRow, byte[] endRow,
-  final String location) {
+final String location, final String encodedRegionName, long length) {
+this(tableName, null, startRow, endRow, location, encodedRegionName, 
length);

Review comment:
   > I think the separate constructor is more intuitive that it is okay to 
not have scan object? What do you think?
   
   IMHO more constructors is less readability. In this case we have 8 
constructors for TableSplit. If I were to use it, I'd confused unless I see the 
usages of each of these. Either we should have a [fluent 
interface](https://en.wikipedia.org/wiki/Fluent_interface) or fewer 
constructors is what I think. Subjective of course.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Commented] (HBASE-25224) Maximize sleep for checking meta and namespace regions availability

2020-10-29 Thread Hudson (Jira)



[ 
https://issues.apache.org/jira/browse/HBASE-25224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17223065#comment-17223065
 ] 

Hudson commented on HBASE-25224:


Results for branch branch-2.3
[build #93 on 
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/93/]:
 (x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/93/General_20Nightly_20Build_20Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/93/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]


(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/93/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11 
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/93/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Maximize sleep for checking meta and namespace regions availability
> ---
>
> Key: HBASE-25224
> URL: https://issues.apache.org/jira/browse/HBASE-25224
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.0, 2.2.7, 2.3.4
>
>
> The isRegionOnline method in HMaster is used on Master startup to check the 
> availability of hbase:meta and hbase:namespace tables.
> I've run into an issue when namespace was not online and Master was just 
> waiting there. I've used HBCK2 to fix the cluster but the initialization was 
> not completed because RetryCounterFactory was already waiting for 10+ hours.
> Since Master is waiting in an idle state it makes no harm to check the region 
> availability more frequently and limit the maximum sleep time.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] nkalmar edited a comment on pull request #2308: HBASE-20598 - Upgrade to JRuby 9.2

2020-10-29 Thread GitBox



nkalmar edited a comment on pull request #2308:
URL: https://github.com/apache/hbase/pull/2308#issuecomment-718901775


   Why is this so flaky? Different 5 tests this time, only 1 test last time :(
   All of them seems to be timeout issues.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] nkalmar commented on pull request #2308: HBASE-20598 - Upgrade to JRuby 9.2

2020-10-29 Thread GitBox



nkalmar commented on pull request #2308:
URL: https://github.com/apache/hbase/pull/2308#issuecomment-718901775


   Why is this so flaky? Different 5 tests this time, only 1 test last time :(



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] apurtell commented on pull request #2594: HBASE-25227 [branch-1] Cast in UnsafeAccess to avoid Java 11 runtime issue

2020-10-29 Thread GitBox



apurtell commented on pull request #2594:
URL: https://github.com/apache/hbase/pull/2594#issuecomment-718905126


   Change uses @Reidd and @anoopsjohn suggestion instead of casts



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Updated] (HBASE-25227) [branch-1] Fix Java 11 runtime issue in UnsafeAccess

2020-10-29 Thread Andrew Kyle Purtell (Jira)



 [ 
https://issues.apache.org/jira/browse/HBASE-25227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Kyle Purtell updated HBASE-25227:

Description: When running tests with Java 11 UnsafeAccess is observed to 
throw NoSuchMethodErrors. Some of our methods accept 'int' parameters and use 
them as parameters to Unsafe methods which should take 'long'. Not an issue on 
branch-2 and up.  (was: When running tests with Java 11 UnsafeAccess is 
observed to throw NoSuchMethodErrors. Some of our methods accept 'int' 
parameters and use them as parameters to Unsafe methods which should take 
'long'. The Java 8 compiler does the implicit conversion but the Java 11 
compiler does not. Add casts to fix. Not an issue on branch-2 and up.)

> [branch-1] Fix Java 11 runtime issue in UnsafeAccess
> 
>
> Key: HBASE-25227
> URL: https://issues.apache.org/jira/browse/HBASE-25227
> Project: HBase
>  Issue Type: Bug
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Minor
> Fix For: 1.7.0
>
>
> When running tests with Java 11 UnsafeAccess is observed to throw 
> NoSuchMethodErrors. Some of our methods accept 'int' parameters and use them 
> as parameters to Unsafe methods which should take 'long'. Not an issue on 
> branch-2 and up.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HBASE-25227) [branch-1] Fix Java 11 runtime issue in UnsafeAccess

2020-10-29 Thread Andrew Kyle Purtell (Jira)



 [ 
https://issues.apache.org/jira/browse/HBASE-25227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Kyle Purtell updated HBASE-25227:

Summary: [branch-1] Fix Java 11 runtime issue in UnsafeAccess  (was: 
[branch-1] Cast in UnsafeAccess to avoid Java 11 runtime issue)

> [branch-1] Fix Java 11 runtime issue in UnsafeAccess
> 
>
> Key: HBASE-25227
> URL: https://issues.apache.org/jira/browse/HBASE-25227
> Project: HBase
>  Issue Type: Bug
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Minor
> Fix For: 1.7.0
>
>
> When running tests with Java 11 UnsafeAccess is observed to throw 
> NoSuchMethodErrors. Some of our methods accept 'int' parameters and use them 
> as parameters to Unsafe methods which should take 'long'. The Java 8 compiler 
> does the implicit conversion but the Java 11 compiler does not. Add casts to 
> fix. Not an issue on branch-2 and up.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] Apache-HBase commented on pull request #2490: HBASE-24419 Normalizer merge plans should consider more than 2 regions when possible

2020-10-29 Thread GitBox



Apache-HBase commented on pull request #2490:
URL: https://github.com/apache/hbase/pull/2490#issuecomment-718913446


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m  8s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 28s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   4m  4s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   1m 42s |  master passed  |
   | +1 :green_heart: |  spotbugs  |   3m 17s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 13s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   4m 16s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   1m 40s |  the patch passed  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  hadoopcheck  |  19m 33s |  Patch does not cause any 
errors with Hadoop 3.1.2 3.2.1 3.3.0.  |
   | +1 :green_heart: |  spotbugs  |   3m 13s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 21s |  The patch does not generate 
ASF License warnings.  |
   |  |   |  48m 11s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.13 Server=19.03.13 base: 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/7/artifact/yetus-general-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/2490 |
   | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti 
checkstyle |
   | uname | Linux a7990961c13f 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 259fe1984a |
   | Max. process+thread count | 84 (vs. ulimit of 3) |
   | modules | C: hbase-common hbase-server U: . |
   | Console output | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/7/console
 |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) 
spotbugs=3.1.12 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #2385: HBASE-24950 Another solution for Splittable Meta

2020-10-29 Thread GitBox



Apache-HBase commented on pull request #2385:
URL: https://github.com/apache/hbase/pull/2385#issuecomment-718914945


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m 19s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  5s |  Unprocessed flag(s): 
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 20s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   4m 51s |  master passed  |
   | +1 :green_heart: |  compile  |   4m 29s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   7m 27s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   2m 31s |  master passed  |
   | -0 :warning: |  patch  |  11m  0s |  Used diff version of patch file. 
Binary files and potentially other changes not applied. Please rebase and 
squash commits if necessary.  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 14s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   4m 57s |  the patch passed  |
   | +1 :green_heart: |  compile  |   4m 13s |  the patch passed  |
   | +1 :green_heart: |  javac  |   4m 13s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   7m 19s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 12s |  hbase-protocol-shaded in the 
patch passed.  |
   | +1 :green_heart: |  javadoc  |   0m 24s |  hbase-common in the patch 
passed.  |
   | +1 :green_heart: |  javadoc  |   0m 25s |  hbase-client in the patch 
passed.  |
   | +1 :green_heart: |  javadoc  |   0m 16s |  hbase-zookeeper in the patch 
passed.  |
   | +1 :green_heart: |  javadoc  |   0m 16s |  hbase-balancer generated 0 new 
+ 1 unchanged - 1 fixed = 1 total (was 2)  |
   | +1 :green_heart: |  javadoc  |   0m 41s |  hbase-server in the patch 
passed.  |
   | +1 :green_heart: |  javadoc  |   0m 14s |  hbase-it in the patch passed.  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   1m  2s |  hbase-protocol-shaded in the patch 
passed.  |
   | +1 :green_heart: |  unit  |   2m 10s |  hbase-common in the patch passed.  
|
   | +1 :green_heart: |  unit  |   1m 16s |  hbase-client in the patch passed.  
|
   | +1 :green_heart: |  unit  |   0m 44s |  hbase-zookeeper in the patch 
passed.  |
   | +1 :green_heart: |  unit  |   0m 22s |  hbase-balancer in the patch 
passed.  |
   | +1 :green_heart: |  unit  | 195m 30s |  hbase-server in the patch passed.  
|
   | +1 :green_heart: |  unit  |   1m  5s |  hbase-it in the patch passed.  |
   |  |   | 245m 22s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.13 Server=19.03.13 base: 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2385/14/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/2385 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux d7f20d92ccca 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / bb4a9d335f |
   | Default Java | 2020-01-14 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2385/14/testReport/
 |
   | Max. process+thread count | 3447 (vs. ulimit of 3) |
   | modules | C: hbase-protocol-shaded hbase-common hbase-client 
hbase-zookeeper hbase-balancer hbase-server hbase-it U: . |
   | Console output | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2385/14/console
 |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #2308: HBASE-20598 - Upgrade to JRuby 9.2

2020-10-29 Thread GitBox



Apache-HBase commented on pull request #2308:
URL: https://github.com/apache/hbase/pull/2308#issuecomment-718918341


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m  7s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 32s |  Maven dependency ordering for branch  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 14s |  Maven dependency ordering for patch  |
   | -0 :warning: |  rubocop  |   0m  4s |  The patch generated 8 new + 24 
unchanged - 1 fixed = 32 total (was 25)  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  xml  |   0m  2s |  The patch has no ill-formed XML 
file.  |
   | +1 :green_heart: |  hadoopcheck  |  17m 26s |  Patch does not cause any 
errors with Hadoop 3.1.2 3.2.1 3.3.0.  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 28s |  The patch does not generate 
ASF License warnings.  |
   |  |   |  26m 33s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.13 Server=19.03.13 base: 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/18/artifact/yetus-general-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/2308 |
   | Optional Tests | dupname asflicense rubocop hadoopcheck xml |
   | uname | Linux d65df110ab6c 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 259fe1984a |
   | rubocop | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/18/artifact/yetus-general-check/output/diff-patch-rubocop.txt
 |
   | Max. process+thread count | 76 (vs. ulimit of 3) |
   | modules | C: . hbase-shell U: . |
   | Console output | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/18/console
 |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) 
rubocop=0.80.0 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #2594: HBASE-25227 [branch-1] Fix Java 11 runtime issue in UnsafeAccess

2020-10-29 Thread GitBox



Apache-HBase commented on pull request #2594:
URL: https://github.com/apache/hbase/pull/2594#issuecomment-718933092


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |  11m 39s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   | -0 :warning: |  test4tests  |   0m  0s |  The patch doesn't appear to 
include any new or modified tests. Please justify why no new tests are needed 
for this patch. Also please list what manual steps were performed to verify 
this patch.  |
   ||| _ branch-1 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   9m 58s |  branch-1 passed  |
   | +1 :green_heart: |  compile  |   0m 19s |  branch-1 passed with JDK 
v1.8.0_262  |
   | +1 :green_heart: |  compile  |   0m 23s |  branch-1 passed with JDK 
v1.7.0_272  |
   | +1 :green_heart: |  checkstyle  |   0m 41s |  branch-1 passed  |
   | +1 :green_heart: |  shadedjars  |   3m 18s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 35s |  branch-1 passed with JDK 
v1.8.0_262  |
   | +1 :green_heart: |  javadoc  |   0m 23s |  branch-1 passed with JDK 
v1.7.0_272  |
   | +0 :ok: |  spotbugs  |   1m 27s |  Used deprecated FindBugs config; 
considering switching to SpotBugs.  |
   | +1 :green_heart: |  findbugs  |   1m 25s |  branch-1 passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   2m  7s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 19s |  the patch passed with JDK 
v1.8.0_262  |
   | +1 :green_heart: |  javac  |   0m 19s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 23s |  the patch passed with JDK 
v1.7.0_272  |
   | +1 :green_heart: |  javac  |   0m 23s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   0m 30s |  the patch passed  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  shadedjars  |   3m  6s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  hadoopcheck  |   4m 56s |  Patch does not cause any 
errors with Hadoop 2.8.5 2.9.2.  |
   | +1 :green_heart: |  javadoc  |   0m 18s |  the patch passed with JDK 
v1.8.0_262  |
   | +1 :green_heart: |  javadoc  |   0m 23s |  the patch passed with JDK 
v1.7.0_272  |
   | +1 :green_heart: |  findbugs  |   1m 18s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   2m 37s |  hbase-common in the patch passed.  
|
   | +1 :green_heart: |  asflicense  |   0m 21s |  The patch does not generate 
ASF License warnings.  |
   |  |   |  48m 28s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.13 Server=19.03.13 base: 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2594/3/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/2594 |
   | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs 
shadedjars hadoopcheck hbaseanti checkstyle compile |
   | uname | Linux c43bf1b1ee01 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | 
/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2594/out/precommit/personality/provided.sh
 |
   | git revision | branch-1 / b30d1d1 |
   | Default Java | 1.7.0_272 |
   | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_262 
/usr/lib/jvm/zulu-7-amd64:1.7.0_272 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2594/3/testReport/
 |
   | Max. process+thread count | 165 (vs. ulimit of 1) |
   | modules | C: hbase-common U: hbase-common |
   | Console output | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2594/3/console
 |
   | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] ndimiduk commented on a change in pull request #2523: HBASE-25167 Normalizer support for hot config reloading

2020-10-29 Thread GitBox



ndimiduk commented on a change in pull request #2523:
URL: https://github.com/apache/hbase/pull/2523#discussion_r514493750



##
File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SimpleRegionNormalizer.java
##
@@ -71,38 +76,78 @@
   static final String MERGE_MIN_REGION_SIZE_MB_KEY = 
"hbase.normalizer.merge.min_region_size.mb";
   static final int DEFAULT_MERGE_MIN_REGION_SIZE_MB = 1;
 
-  private Configuration conf;
   private MasterServices masterServices;
-  private boolean splitEnabled;
-  private boolean mergeEnabled;
-  private int minRegionCount;
-  private Period mergeMinRegionAge;
-  private int mergeMinRegionSizeMb;
+
+  /** Ensure configuration changes are applied atomically. */
+  private final ReadWriteLock configUpdateLock = new ReentrantReadWriteLock();
+  @GuardedBy("configUpdateLock") private Configuration conf;
+  @GuardedBy("configUpdateLock") private boolean splitEnabled;
+  @GuardedBy("configUpdateLock") private boolean mergeEnabled;
+  @GuardedBy("configUpdateLock") private int minRegionCount;
+  @GuardedBy("configUpdateLock") private Period mergeMinRegionAge;
+  @GuardedBy("configUpdateLock") private int mergeMinRegionSizeMb;
 
   public SimpleRegionNormalizer() {
-splitEnabled = DEFAULT_SPLIT_ENABLED;
-mergeEnabled = DEFAULT_MERGE_ENABLED;
-minRegionCount = DEFAULT_MIN_REGION_COUNT;
-mergeMinRegionAge = Period.ofDays(DEFAULT_MERGE_MIN_REGION_AGE_DAYS);
-mergeMinRegionSizeMb = DEFAULT_MERGE_MIN_REGION_SIZE_MB;
+final Lock writeLock = configUpdateLock.writeLock();
+writeLock.lock();
+try {
+  splitEnabled = DEFAULT_SPLIT_ENABLED;
+  mergeEnabled = DEFAULT_MERGE_ENABLED;
+  minRegionCount = DEFAULT_MIN_REGION_COUNT;
+  mergeMinRegionAge = Period.ofDays(DEFAULT_MERGE_MIN_REGION_AGE_DAYS);
+  mergeMinRegionSizeMb = DEFAULT_MERGE_MIN_REGION_SIZE_MB;
+} finally {
+  writeLock.unlock();
+}
   }
 
   @Override
   public Configuration getConf() {
-return conf;
+final Lock readLock = configUpdateLock.readLock();
+readLock.lock();
+try {
+  return conf;
+} finally {
+  readLock.unlock();
+}
   }
 
   @Override
   public void setConf(final Configuration conf) {
 if (conf == null) {
   return;
 }
-this.conf = conf;
-splitEnabled = conf.getBoolean(SPLIT_ENABLED_KEY, DEFAULT_SPLIT_ENABLED);
-mergeEnabled = conf.getBoolean(MERGE_ENABLED_KEY, DEFAULT_MERGE_ENABLED);
-minRegionCount = parseMinRegionCount(conf);
-mergeMinRegionAge = parseMergeMinRegionAge(conf);
-mergeMinRegionSizeMb = parseMergeMinRegionSizeMb(conf);
+
+final Lock writeLock = configUpdateLock.writeLock();
+writeLock.lock();

Review comment:
   Sure @bharathv , I think that's a nice suggestion. I agree that the 
strict ordering guarantees provided by explicit locking are not needed here. 
I've pushed a new commit that unwinds the locking and uses atomic instance 
assignment as you suggest. Let me know what you think.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Commented] (HBASE-25224) Maximize sleep for checking meta and namespace regions availability

2020-10-29 Thread Hudson (Jira)



[ 
https://issues.apache.org/jira/browse/HBASE-25224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17223157#comment-17223157
 ] 

Hudson commented on HBASE-25224:


Results for branch branch-2
[build #87 on 
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/87/]:
 (/) *{color:green}+1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/87/General_20Nightly_20Build_20Report/]




(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/87/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]


(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/87/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11 
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/87/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Maximize sleep for checking meta and namespace regions availability
> ---
>
> Key: HBASE-25224
> URL: https://issues.apache.org/jira/browse/HBASE-25224
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.0, 2.2.7, 2.3.4
>
>
> The isRegionOnline method in HMaster is used on Master startup to check the 
> availability of hbase:meta and hbase:namespace tables.
> I've run into an issue when namespace was not online and Master was just 
> waiting there. I've used HBCK2 to fix the cluster but the initialization was 
> not completed because RetryCounterFactory was already waiting for 10+ hours.
> Since Master is waiting in an idle state it makes no harm to check the region 
> availability more frequently and limit the maximum sleep time.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-25223) Use try-with-resources statement in snapshot package

2020-10-29 Thread Hudson (Jira)



[ 
https://issues.apache.org/jira/browse/HBASE-25223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17223156#comment-17223156
 ] 

Hudson commented on HBASE-25223:


Results for branch branch-2
[build #87 on 
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/87/]:
 (/) *{color:green}+1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/87/General_20Nightly_20Build_20Report/]




(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/87/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]


(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/87/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11 
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/87/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Use try-with-resources statement in snapshot package
> 
>
> Key: HBASE-25223
> URL: https://issues.apache.org/jira/browse/HBASE-25223
> Project: HBase
>  Issue Type: Improvement
>Reporter: minjikim
>Assignee: minjikim
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
> Using try-with-resources statement makes the code simpler.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] huaxiangsun commented on a change in pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…

2020-10-29 Thread GitBox



huaxiangsun commented on a change in pull request #2584:
URL: https://github.com/apache/hbase/pull/2584#discussion_r514512363



##
File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncNonMetaRegionLocator.java
##
@@ -577,6 +635,15 @@ private void removeLocationFromCache(HRegionLocation loc) {
   if (!canUpdateOnError(loc, oldLoc)) {
 return;
   }
+  // Tell metaReplicaSelector that the location is stale. It will create a 
stale entry
+  // with timestamp internally. Next time the client looks up the same 
location,
+  // it will pick a different meta replica region. For the current 
implementation,
+  // the metaReplicaId is not used, so the primary one is passed in.
+  if (this.metaReplicaMode == CatalogReplicaMode.LoadBalance) {
+// metaReplicaId is not used in simpleSelector, default to the primary 
one.
+metaReplicaSelector.onError(loc, RegionInfo.DEFAULT_REPLICA_ID);

Review comment:
   I pushed a patch with fromReplicaId set to -1. Work on an add-on patch 
which will fill in this metaReplicaId.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #2490: HBASE-24419 Normalizer merge plans should consider more than 2 regions when possible

2020-10-29 Thread GitBox



Apache-HBase commented on pull request #2490:
URL: https://github.com/apache/hbase/pull/2490#issuecomment-718978987


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m 22s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): 
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 27s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   4m 34s |  master passed  |
   | +1 :green_heart: |  compile  |   1m 47s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   6m 48s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   1m  2s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 17s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   3m 59s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 31s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 31s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   6m 39s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   1m  5s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   1m 33s |  hbase-common in the patch passed.  
|
   | +1 :green_heart: |  unit  | 135m 36s |  hbase-server in the patch passed.  
|
   |  |   | 169m  9s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.13 Server=19.03.13 base: 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/7/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/2490 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux f16918938d8d 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 259fe1984a |
   | Default Java | 2020-01-14 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/7/testReport/
 |
   | Max. process+thread count | 4807 (vs. ulimit of 3) |
   | modules | C: hbase-common hbase-server U: . |
   | Console output | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/7/console
 |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #2523: HBASE-25167 Normalizer support for hot config reloading

2020-10-29 Thread GitBox



Apache-HBase commented on pull request #2523:
URL: https://github.com/apache/hbase/pull/2523#issuecomment-718979616


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 44s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 32s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   4m  9s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   1m 46s |  master passed  |
   | +1 :green_heart: |  spotbugs  |   3m  6s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 13s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   3m 51s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   0m 24s |  hbase-common: The patch 
generated 0 new + 0 unchanged - 1 fixed = 0 total (was 1)  |
   | +1 :green_heart: |  checkstyle  |   1m 15s |  The patch passed checkstyle 
in hbase-server  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  hadoopcheck  |  19m 48s |  Patch does not cause any 
errors with Hadoop 3.1.2 3.2.1 3.3.0.  |
   | +1 :green_heart: |  spotbugs  |   3m 35s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 23s |  The patch does not generate 
ASF License warnings.  |
   |  |   |  48m 30s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.13 Server=19.03.13 base: 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2523/7/artifact/yetus-general-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/2523 |
   | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti 
checkstyle |
   | uname | Linux db187a2cc5e7 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 
16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 259fe1984a |
   | Max. process+thread count | 94 (vs. ulimit of 3) |
   | modules | C: hbase-common hbase-server U: . |
   | Console output | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2523/7/console
 |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) 
spotbugs=3.1.12 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #2490: HBASE-24419 Normalizer merge plans should consider more than 2 regions when possible

2020-10-29 Thread GitBox



Apache-HBase commented on pull request #2490:
URL: https://github.com/apache/hbase/pull/2490#issuecomment-718982958


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m  9s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): 
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 22s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   3m 39s |  master passed  |
   | +1 :green_heart: |  compile  |   1m 19s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   6m 35s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   1m  0s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 16s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   3m 30s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 20s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 20s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   6m 35s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 58s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   1m 31s |  hbase-common in the patch passed.  
|
   | +1 :green_heart: |  unit  | 146m 58s |  hbase-server in the patch passed.  
|
   |  |   | 177m 28s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.13 Server=19.03.13 base: 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/7/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/2490 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 1eaa5f30ffd7 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 
17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 259fe1984a |
   | Default Java | 1.8.0_232 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/7/testReport/
 |
   | Max. process+thread count | 4348 (vs. ulimit of 3) |
   | modules | C: hbase-common hbase-server U: . |
   | Console output | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/7/console
 |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] sandeepvinayak commented on a change in pull request #2591: HBASE-24859: Optimize in-memory representation of HBase map reduce table splits

2020-10-29 Thread GitBox



sandeepvinayak commented on a change in pull request #2591:
URL: https://github.com/apache/hbase/pull/2591#discussion_r514525613



##
File path: 
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScanBase.java
##
@@ -271,6 +272,11 @@ public void testNumOfSplits(int splitsPerRegion, int 
expectedNumOfSplits) throws
 TableInputFormat tif = new TableInputFormat();
 tif.setConf(job.getConfiguration());
 List splits = tif.getSplits(job);
+for (InputSplit split : splits) {
+  TableSplit tableSplit = (TableSplit) split;
+  Assert.assertEquals(tableSplit.getScan().getStartRow(), 
HConstants.EMPTY_START_ROW);

Review comment:
   I see your point, thanks !





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] sandeepvinayak commented on a change in pull request #2591: HBASE-24859: Optimize in-memory representation of HBase map reduce table splits

2020-10-29 Thread GitBox



sandeepvinayak commented on a change in pull request #2591:
URL: https://github.com/apache/hbase/pull/2591#discussion_r514528418



##
File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSplit.java
##
@@ -184,9 +184,24 @@ public TableSplit(final byte [] tableName, byte[] 
startRow, byte[] endRow,
* @param startRow The start row of the split.
* @param endRow The end row of the split.
* @param location The location of the region.
+   * @param encodedRegionName The region ID.
+   * @param length Size of region in bytes
*/
   public TableSplit(TableName tableName, byte[] startRow, byte[] endRow,
-  final String location) {
+final String location, final String encodedRegionName, long length) {
+this(tableName, null, startRow, endRow, location, encodedRegionName, 
length);

Review comment:
   @Reidd Scan is not useless for all the input formats of map-reduce, 
it is for the `TableInputFormat`.
   We do need the scan object for `MultiTableInputFormat`





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] saintstack merged pull request #2595: HBASE-25090 CompactionConfiguration logs unrealistic store file sizes

2020-10-29 Thread GitBox



saintstack merged pull request #2595:
URL: https://github.com/apache/hbase/pull/2595


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Resolved] (HBASE-25090) CompactionConfiguration logs unrealistic store file sizes

2020-10-29 Thread Michael Stack (Jira)



 [ 
https://issues.apache.org/jira/browse/HBASE-25090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Stack resolved HBASE-25090.
---
Fix Version/s: 2.4.0
   3.0.0-alpha-1
 Hadoop Flags: Reviewed
   Resolution: Fixed

Merged to master and branch-2. Thanks for the PR [~GeorryHuang]

> CompactionConfiguration logs unrealistic store file sizes
> -
>
> Key: HBASE-25090
> URL: https://issues.apache.org/jira/browse/HBASE-25090
> Project: HBase
>  Issue Type: Bug
>  Components: Compaction
>Affects Versions: 2.3.2
>Reporter: Nick Dimiduk
>Assignee: Zhuoyue Huang
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
> I noticed this in the debug logging of a unit test that makes use of a 
> MiniCluster. I'm pretty sure {{8.00 EB}} -- exabytes?? -- is not an accurate 
> reporting of the size of these HFiles.
> {noformat}
> 2020-09-22 16:20:57,996 INFO  
> [StoreOpener-5e0b426a952230abb65623fc54869833-1] 
> compactions.CompactionConfiguration(177): size [128 MB, 8.00 EB, 8.00 EB); 
> files [3, 10); ...
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] Apache-HBase commented on pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…

2020-10-29 Thread GitBox



Apache-HBase commented on pull request #2584:
URL: https://github.com/apache/hbase/pull/2584#issuecomment-719002752


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m 31s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 34s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   5m  5s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   2m 11s |  master passed  |
   | +1 :green_heart: |  spotbugs  |   4m  8s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 17s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   5m  0s |  the patch passed  |
   | -0 :warning: |  checkstyle  |   0m 33s |  hbase-client: The patch 
generated 3 new + 2 unchanged - 0 fixed = 5 total (was 2)  |
   | -0 :warning: |  checkstyle  |   1m 23s |  hbase-server: The patch 
generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  hadoopcheck  |  24m 33s |  Patch does not cause any 
errors with Hadoop 3.1.2 3.2.1 3.3.0.  |
   | +1 :green_heart: |  spotbugs  |   4m  9s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 27s |  The patch does not generate 
ASF License warnings.  |
   |  |   |  60m 23s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.13 Server=19.03.13 base: 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/7/artifact/yetus-general-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/2584 |
   | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti 
checkstyle |
   | uname | Linux 8c8f9a206e7a 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 259fe1984a |
   | checkstyle | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/7/artifact/yetus-general-check/output/diff-checkstyle-hbase-client.txt
 |
   | checkstyle | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/7/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
 |
   | Max. process+thread count | 85 (vs. ulimit of 3) |
   | modules | C: hbase-client hbase-server U: . |
   | Console output | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/7/console
 |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) 
spotbugs=3.1.12 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] ndimiduk merged pull request #2490: HBASE-24419 Normalizer merge plans should consider more than 2 regions when possible

2020-10-29 Thread GitBox



ndimiduk merged pull request #2490:
URL: https://github.com/apache/hbase/pull/2490


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] ndimiduk opened a new pull request #2596: Backport "HBASE-24419 Normalizer merge plans should consider more than 2 region…" to branch-2

2020-10-29 Thread GitBox



ndimiduk opened a new pull request #2596:
URL: https://github.com/apache/hbase/pull/2596


   …s when possible
   
   The core change here is to the loop in
   `SimpleRegionNormalizer#computeMergeNormalizationPlans`. It's a nested
   loop that walks the table's region chain once, looking for contiguous
   sequences of regions that meet the criteria for merge. The outer loop
   tracks the starting point of the next sequence, the inner loop looks
   for the end of that sequence. A single sequence becomes an instance of
   `MergeNormalizationPlan`.
   
   Signed-off-by: Huaxiang Sun 



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #2308: HBASE-20598 - Upgrade to JRuby 9.2

2020-10-29 Thread GitBox



Apache-HBase commented on pull request #2308:
URL: https://github.com/apache/hbase/pull/2308#issuecomment-719012161


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 29s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  4s |  Unprocessed flag(s): 
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 31s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   4m  9s |  master passed  |
   | +1 :green_heart: |  compile  |   2m 47s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   6m 34s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   2m 56s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 22s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   3m 54s |  the patch passed  |
   | +1 :green_heart: |  compile  |   2m 46s |  the patch passed  |
   | +1 :green_heart: |  javac  |   2m 46s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   6m 39s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   2m 56s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  | 167m 33s |  root in the patch passed.  |
   |  |   | 204m 44s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.13 Server=19.03.13 base: 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/18/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/2308 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux bf321ea0cd9e 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 259fe1984a |
   | Default Java | 2020-01-14 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/18/testReport/
 |
   | Max. process+thread count | 7441 (vs. ulimit of 3) |
   | modules | C: hbase-shell . U: . |
   | Console output | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/18/console
 |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] saintstack commented on a change in pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…

2020-10-29 Thread GitBox



saintstack commented on a change in pull request #2584:
URL: https://github.com/apache/hbase/pull/2584#discussion_r514543049



##
File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnectionImpl.java
##
@@ -182,8 +183,20 @@ public void newDead(ServerName sn) {
   }
 
   private void spawnRenewalChore(final UserGroupInformation user) {
-authService = new ChoreService("Relogin service");
-authService.scheduleChore(AuthUtil.getAuthRenewalChore(user));
+ChoreService service = getChoreService();
+service.scheduleChore(AuthUtil.getAuthRenewalChore(user));
+  }
+
+  /**
+   * If choreService has not been created yet, create the ChoreService.
+   * It is not thread safe.
+   * @return ChoreService
+   */
+  ChoreService getChoreService() {

Review comment:
   Do we need this?
   
   It is called from the constructor only. If so, just create choreservice in 
the constructor.
   
   If getChoreService can be called from elsewhere, then does this need to be 
synchronized so we don't make multiple chore service instances?

##
File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncNonMetaRegionLocator.java
##
@@ -196,8 +202,43 @@ private boolean tryComplete(LocateRequest req, 
CompletableFuture {
+int numOfReplicas = 1;
+try {
+  RegionLocations metaLocations = 
conn.registry.getMetaRegionLocations().get(
+GET_META_LOCATIONS_TIMEOUT, TimeUnit.MILLISECONDS);
+  numOfReplicas = metaLocations.size();
+} catch (Exception e) {
+  LOG.error("Failed to get table {}'s region replication, ", 
META_TABLE_NAME, e);
+}
+return numOfReplicas;
+  });
+break;
+  case None:
+// If user does not configure LOCATOR_META_REPLICAS_MODE, let's check 
the legacy config.
+if (this.metaReplicaMode == CatalogReplicaMode.None) {
+  boolean useMetaReplicas = 
conn.getConfiguration().getBoolean(USE_META_REPLICAS,
+DEFAULT_USE_META_REPLICAS);
+  if (useMetaReplicas) {
+this.metaReplicaMode = CatalogReplicaMode.HedgedRead;

Review comment:
   None is not the same as hedged read, right?
   
   If user configures hedged read as metaReplicaMode, they fall through to 
default which 'does nothing'. That don't seem right. Shouldn't it have a case 
here?

##
File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionLocator.java
##
@@ -37,6 +37,12 @@
  */
 @InterfaceAudience.Public
 public interface RegionLocator extends Closeable {
+
+  String LOCATOR_META_REPLICAS_MODE = "hbase.locator.meta.replicas.mode";

Review comment:
   Needs javadoc to at least point at what these are about.

##
File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncNonMetaRegionLocator.java
##
@@ -433,9 +474,24 @@ private void locateInMeta(TableName tableName, 
LocateRequest req) {
 Scan scan = new Scan().withStartRow(metaStartKey).withStopRow(metaStopKey, 
true)
   
.addFamily(HConstants.CATALOG_FAMILY).setReversed(true).setCaching(locatePrefetchLimit)
   .setReadType(ReadType.PREAD);
-if (useMetaReplicas) {
-  scan.setConsistency(Consistency.TIMELINE);
+
+switch (this.metaReplicaMode) {
+  case LoadBalance:
+int metaReplicaId = this.metaReplicaSelector.select(tableName, 
req.row, req.locateType);
+if (metaReplicaId != RegionInfo.DEFAULT_REPLICA_ID) {
+  // If the selector gives a non-primary meta replica region, then go 
with it.
+  // Otherwise, just go to primary in non-hedgedRead mode.
+  scan.setConsistency(Consistency.TIMELINE);
+  scan.setReplicaId(metaReplicaId);
+}
+break;
+  case HedgedRead:
+scan.setConsistency(Consistency.TIMELINE);
+break;
+  default:
+// do nothing

Review comment:
   Yeah, said this before but in follow-on, would be good to shove all this 
stuff into a CatalogReplicaMode class. Internally this class would figure which 
policy to run. It would have a method that took a Scan that allowed decorating 
the Scan w/ whatever the mode needed to implement its policy. Later.

##
File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncNonMetaRegionLocator.java
##
@@ -577,6 +635,15 @@ private void removeLocationFromCache(HRegionLocation loc) {
   if (!canUpdateOnError(loc, oldLoc)) {
 return;
   }
+  // Tell metaReplicaSelector that the location is stale. It will create a 
stale entry
+  // with timestamp internally. Next time the client looks up the same 
location,
+  // it will pick a different meta replica region. For the current 
implementation,
+  // the metaReplicaId is not used, so the primary one is passed in.
+  if (this.metaReplicaMode == CatalogReplicaMode.LoadBalance) {
+// TODO: pass i

[GitHub] [hbase] apurtell merged pull request #2578: HBASE-24768 Clear cached service kerberos ticket in case of SASL fail…

2020-10-29 Thread GitBox



apurtell merged pull request #2578:
URL: https://github.com/apache/hbase/pull/2578


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #2596: Backport "HBASE-24419 Normalizer merge plans should consider more than 2 region…" to branch-2

2020-10-29 Thread GitBox



Apache-HBase commented on pull request #2596:
URL: https://github.com/apache/hbase/pull/2596#issuecomment-719035664


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   2m 17s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  1s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   ||| _ branch-2 Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 16s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   4m  4s |  branch-2 passed  |
   | +1 :green_heart: |  checkstyle  |   1m 39s |  branch-2 passed  |
   | +1 :green_heart: |  spotbugs  |   3m  0s |  branch-2 passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 15s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   3m 45s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   1m 40s |  the patch passed  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  hadoopcheck  |  13m 56s |  Patch does not cause any 
errors with Hadoop 3.1.2 3.2.1.  |
   | +1 :green_heart: |  spotbugs  |   3m 38s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 25s |  The patch does not generate 
ASF License warnings.  |
   |  |   |  44m 14s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.13 Server=19.03.13 base: 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2596/1/artifact/yetus-general-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/2596 |
   | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti 
checkstyle |
   | uname | Linux b59c48d6f54b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | branch-2 / 69f282e21a |
   | Max. process+thread count | 94 (vs. ulimit of 12500) |
   | modules | C: hbase-common hbase-server U: . |
   | Console output | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2596/1/console
 |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) 
spotbugs=3.1.12 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] huaxiangsun commented on a change in pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…

2020-10-29 Thread GitBox



huaxiangsun commented on a change in pull request #2584:
URL: https://github.com/apache/hbase/pull/2584#discussion_r514580314



##
File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnectionImpl.java
##
@@ -182,8 +183,20 @@ public void newDead(ServerName sn) {
   }
 
   private void spawnRenewalChore(final UserGroupInformation user) {
-authService = new ChoreService("Relogin service");
-authService.scheduleChore(AuthUtil.getAuthRenewalChore(user));
+ChoreService service = getChoreService();
+service.scheduleChore(AuthUtil.getAuthRenewalChore(user));
+  }
+
+  /**
+   * If choreService has not been created yet, create the ChoreService.
+   * It is not thread safe.
+   * @return ChoreService
+   */
+  ChoreService getChoreService() {

Review comment:
   Yeah, it is only called in the context of constructor. To start the 
service in the constructor, it needs to check if meta replica load balancer is 
enabled. Duo has a comment to avoid checking multiple configs in 
AsyncConnectionImpl and start the service on demand. Tough there is no 
multithread requirement for now, I think it makes sense to add synchronized. 





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] apurtell merged pull request #2594: HBASE-25227 [branch-1] Fix Java 11 runtime issue in UnsafeAccess

2020-10-29 Thread GitBox



apurtell merged pull request #2594:
URL: https://github.com/apache/hbase/pull/2594


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] apurtell commented on pull request #2594: HBASE-25227 [branch-1] Fix Java 11 runtime issue in UnsafeAccess

2020-10-29 Thread GitBox



apurtell commented on pull request #2594:
URL: https://github.com/apache/hbase/pull/2594#issuecomment-719038355


   I'm not sure what happened here. Applying the commit locally. 



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Commented] (HBASE-25032) Wait for region server to become online before adding it to online servers in Master

2020-10-29 Thread Sandeep Guggilam (Jira)



[ 
https://issues.apache.org/jira/browse/HBASE-25032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17223221#comment-17223221
 ] 

Sandeep Guggilam commented on HBASE-25032:
--

[~anoop.hbase] [~apurtell]  I spent some time looking at the code today. One 
thing I noticed is that we abort the RS by throwing exception in case of any 
issues with replication setup with the peer during the 
[startup|https://github.com/apache/hbase/blob/branch-1/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java#L1964]
 of RS : 
[https://github.com/apache/hbase/blob/branch-1/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/Replication.java#L241]

So looks like the current design already treats some aspects of setting up the 
replication as important and aborts the RS if not setup properly as opposed to 
our thought of letting RS accept requests even if replication fails in an async 
thread

Should we consider the thought of delaying adding this RS to availableServers 
on Master till it is actually ready to accept requests ?

Thoughts ?

> Wait for region server to become online before adding it to online servers in 
> Master
> 
>
> Key: HBASE-25032
> URL: https://issues.apache.org/jira/browse/HBASE-25032
> Project: HBase
>  Issue Type: Bug
>Reporter: Sandeep Guggilam
>Assignee: Sandeep Guggilam
>Priority: Major
>
> As part of RS start up, RS reports for duty to Master . Master acknowledges 
> the request and adds it to the onlineServers list for further assigning any 
> regions to the RS
> Once Master acknowledges the reportForDuty and sends back the response, RS 
> does a bunch of stuff like initializing replication sources etc before 
> becoming online. However, sometimes there could be an issue with initializing 
> replication sources when it is unable to connect to peer clusters because of 
> some kerberos configuration and there would be a delay of around 20 mins in 
> becoming online.
>  
> Since master considers it online, it tries to assign regions and which fails 
> with ServerNotRunningYet exception, then the master tries to unassign which 
> again fails with the same exception leading the region to FAILED_CLOSE state.
>  
> It would be good to have a check to see if the RS is ready to accept the 
> assignment requests before adding it to online servers list which would 
> account for any such delays as described above



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] huaxiangsun commented on a change in pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…

2020-10-29 Thread GitBox



huaxiangsun commented on a change in pull request #2584:
URL: https://github.com/apache/hbase/pull/2584#discussion_r514583501



##
File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncNonMetaRegionLocator.java
##
@@ -196,8 +202,43 @@ private boolean tryComplete(LocateRequest req, 
CompletableFuture {
+int numOfReplicas = 1;
+try {
+  RegionLocations metaLocations = 
conn.registry.getMetaRegionLocations().get(
+GET_META_LOCATIONS_TIMEOUT, TimeUnit.MILLISECONDS);
+  numOfReplicas = metaLocations.size();
+} catch (Exception e) {
+  LOG.error("Failed to get table {}'s region replication, ", 
META_TABLE_NAME, e);
+}
+return numOfReplicas;
+  });
+break;
+  case None:
+// If user does not configure LOCATOR_META_REPLICAS_MODE, let's check 
the legacy config.
+if (this.metaReplicaMode == CatalogReplicaMode.None) {
+  boolean useMetaReplicas = 
conn.getConfiguration().getBoolean(USE_META_REPLICAS,
+DEFAULT_USE_META_REPLICAS);
+  if (useMetaReplicas) {
+this.metaReplicaMode = CatalogReplicaMode.HedgedRead;

Review comment:
   The whole purpose is to set up the metaReplicaMode. In case of 
LoadBalance mode, it needs to create CatalogSelector. In case of HedgedRead, 
there is nothing to be done. None is not HedgedRead, it means that there is no 
metaReplicaMode configured. In this case, it still needs to check if 
USE_META_REPLICAS is configured or not, if USE_META_REPLICAS is configured, it 
needs to set metaReplicaMode to HedgedRead. HedgedRead is the previous 
USE_META_REPLICAS. default branch is added to avoid warning from some java 
checking.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] huaxiangsun commented on a change in pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…

2020-10-29 Thread GitBox



huaxiangsun commented on a change in pull request #2584:
URL: https://github.com/apache/hbase/pull/2584#discussion_r514584611



##
File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncNonMetaRegionLocator.java
##
@@ -196,8 +202,43 @@ private boolean tryComplete(LocateRequest req, 
CompletableFuture {
+int numOfReplicas = 1;
+try {
+  RegionLocations metaLocations = 
conn.registry.getMetaRegionLocations().get(
+GET_META_LOCATIONS_TIMEOUT, TimeUnit.MILLISECONDS);
+  numOfReplicas = metaLocations.size();
+} catch (Exception e) {
+  LOG.error("Failed to get table {}'s region replication, ", 
META_TABLE_NAME, e);
+}
+return numOfReplicas;
+  });
+break;
+  case None:
+// If user does not configure LOCATOR_META_REPLICAS_MODE, let's check 
the legacy config.
+if (this.metaReplicaMode == CatalogReplicaMode.None) {
+  boolean useMetaReplicas = 
conn.getConfiguration().getBoolean(USE_META_REPLICAS,
+DEFAULT_USE_META_REPLICAS);
+  if (useMetaReplicas) {
+this.metaReplicaMode = CatalogReplicaMode.HedgedRead;
+  }
+}
+break;
+  default:
+// Doing nothing

Review comment:
   There are three modes:
   1. HedgedRead
   2. LoadBalance
   3. None (no mode is configured).
   
   It will get here when HedgedRead is configured. I assume that there is no 
need to log HedgedRead mode, otherwise, it needs to log at other two branches. 
wdyt?





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Updated] (HBASE-25227) [branch-1] Fix Java 11 runtime issue in UnsafeAccess

2020-10-29 Thread Andrew Kyle Purtell (Jira)



 [ 
https://issues.apache.org/jira/browse/HBASE-25227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Kyle Purtell updated HBASE-25227:

Hadoop Flags: Reviewed
  Resolution: Fixed
  Status: Resolved  (was: Patch Available)

> [branch-1] Fix Java 11 runtime issue in UnsafeAccess
> 
>
> Key: HBASE-25227
> URL: https://issues.apache.org/jira/browse/HBASE-25227
> Project: HBase
>  Issue Type: Bug
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Minor
> Fix For: 1.7.0
>
>
> When running tests with Java 11 UnsafeAccess is observed to throw 
> NoSuchMethodErrors. Some of our methods accept 'int' parameters and use them 
> as parameters to Unsafe methods which should take 'long'. Not an issue on 
> branch-2 and up.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] Apache-HBase commented on pull request #2523: HBASE-25167 Normalizer support for hot config reloading

2020-10-29 Thread GitBox



Apache-HBase commented on pull request #2523:
URL: https://github.com/apache/hbase/pull/2523#issuecomment-719041537


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 26s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): 
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 21s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   4m 12s |  master passed  |
   | +1 :green_heart: |  compile  |   1m 33s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   6m 41s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   1m  7s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 16s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   4m  4s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 30s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 30s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   6m 39s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   1m  5s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   1m 46s |  hbase-common in the patch passed.  
|
   | +1 :green_heart: |  unit  | 136m  6s |  hbase-server in the patch passed.  
|
   |  |   | 168m 10s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.13 Server=19.03.13 base: 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2523/7/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/2523 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux a2a9b079 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 259fe1984a |
   | Default Java | 2020-01-14 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2523/7/testReport/
 |
   | Max. process+thread count | 3733 (vs. ulimit of 3) |
   | modules | C: hbase-common hbase-server U: . |
   | Console output | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2523/7/console
 |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #2523: HBASE-25167 Normalizer support for hot config reloading

2020-10-29 Thread GitBox



Apache-HBase commented on pull request #2523:
URL: https://github.com/apache/hbase/pull/2523#issuecomment-719042209


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 27s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  4s |  Unprocessed flag(s): 
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 23s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   3m 44s |  master passed  |
   | +1 :green_heart: |  compile  |   1m 19s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   6m 28s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   1m  1s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 17s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   3m 28s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 24s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 24s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   6m 31s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   1m  0s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   1m 32s |  hbase-common in the patch passed.  
|
   | +1 :green_heart: |  unit  | 139m 42s |  hbase-server in the patch passed.  
|
   |  |   | 169m 47s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.13 Server=19.03.13 base: 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2523/7/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/2523 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux d3193a0f0a64 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 
16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 259fe1984a |
   | Default Java | 1.8.0_232 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2523/7/testReport/
 |
   | Max. process+thread count | 4393 (vs. ulimit of 3) |
   | modules | C: hbase-common hbase-server U: . |
   | Console output | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2523/7/console
 |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] huaxiangsun commented on a change in pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…

2020-10-29 Thread GitBox



huaxiangsun commented on a change in pull request #2584:
URL: https://github.com/apache/hbase/pull/2584#discussion_r514587250



##
File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncNonMetaRegionLocator.java
##
@@ -433,9 +474,24 @@ private void locateInMeta(TableName tableName, 
LocateRequest req) {
 Scan scan = new Scan().withStartRow(metaStartKey).withStopRow(metaStopKey, 
true)
   
.addFamily(HConstants.CATALOG_FAMILY).setReversed(true).setCaching(locatePrefetchLimit)
   .setReadType(ReadType.PREAD);
-if (useMetaReplicas) {
-  scan.setConsistency(Consistency.TIMELINE);
+
+switch (this.metaReplicaMode) {
+  case LoadBalance:
+int metaReplicaId = this.metaReplicaSelector.select(tableName, 
req.row, req.locateType);
+if (metaReplicaId != RegionInfo.DEFAULT_REPLICA_ID) {
+  // If the selector gives a non-primary meta replica region, then go 
with it.
+  // Otherwise, just go to primary in non-hedgedRead mode.
+  scan.setConsistency(Consistency.TIMELINE);
+  scan.setReplicaId(metaReplicaId);
+}
+break;
+  case HedgedRead:
+scan.setConsistency(Consistency.TIMELINE);
+break;
+  default:
+// do nothing

Review comment:
   Now I thought about it, it makes sense. Maybe a 
CatalogReplicaModeManager class which encaps mode and selector?
   Let me create a followup jira after this is merged.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] huaxiangsun commented on a change in pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…

2020-10-29 Thread GitBox



huaxiangsun commented on a change in pull request #2584:
URL: https://github.com/apache/hbase/pull/2584#discussion_r514588062



##
File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncNonMetaRegionLocator.java
##
@@ -577,6 +635,15 @@ private void removeLocationFromCache(HRegionLocation loc) {
   if (!canUpdateOnError(loc, oldLoc)) {
 return;
   }
+  // Tell metaReplicaSelector that the location is stale. It will create a 
stale entry
+  // with timestamp internally. Next time the client looks up the same 
location,
+  // it will pick a different meta replica region. For the current 
implementation,
+  // the metaReplicaId is not used, so the primary one is passed in.
+  if (this.metaReplicaMode == CatalogReplicaMode.LoadBalance) {
+// TODO: pass in -1 as currently fromReplicaId is not being used.

Review comment:
   Good catch, missed the comment. I am working on a follow-up patch which 
fills in the read fromMetaReplicaId. Going to post it as a new jira or within 
this patch depends on Duo's feedback.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] huaxiangsun commented on pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…

2020-10-29 Thread GitBox



huaxiangsun commented on pull request #2584:
URL: https://github.com/apache/hbase/pull/2584#issuecomment-719052727


   Update to address some of comments from Stack.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…

2020-10-29 Thread GitBox



Apache-HBase commented on pull request #2584:
URL: https://github.com/apache/hbase/pull/2584#issuecomment-719054811


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   4m  1s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  4s |  Unprocessed flag(s): 
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 24s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   4m  8s |  master passed  |
   | +1 :green_heart: |  compile  |   1m 34s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   6m 46s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   1m 10s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 15s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   4m  4s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 32s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 32s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   6m 42s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   1m  7s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   1m  5s |  hbase-client in the patch passed.  
|
   | -1 :x: |  unit  | 131m 35s |  hbase-server in the patch failed.  |
   |  |   | 166m 54s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.13 Server=19.03.13 base: 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/7/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/2584 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 6a121e4aa929 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 259fe1984a |
   | Default Java | 2020-01-14 |
   | unit | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/7/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/7/testReport/
 |
   | Max. process+thread count | 3941 (vs. ulimit of 3) |
   | modules | C: hbase-client hbase-server U: . |
   | Console output | 
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/7/console
 |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

1 2 >

1 - 100 of 160 matches

Mail list logo