[GitHub] [hbase] saintstack commented on issue #995: HBASE-23055 Alter hbase:meta
saintstack commented on issue #995: HBASE-23055 Alter hbase:meta URL: https://github.com/apache/hbase/pull/995#issuecomment-571901659 Thanks for the review @bharathv . Helps. I just pushed some more checkstyle and test fixes. Will address your helpful comments in the morning. Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #995: HBASE-23055 Alter hbase:meta
saintstack commented on a change in pull request #995: HBASE-23055 Alter
hbase:meta
URL: https://github.com/apache/hbase/pull/995#discussion_r364072745
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKAsyncRegistry.java
##
@@ -229,6 +244,44 @@ private void
getMetaRegionLocation(CompletableFuture future,
});
}
+ @Override
+ public CompletableFuture getMetaTableState() {
Review comment:
Yes. Let me add. Thanks.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #995: HBASE-23055 Alter hbase:meta
saintstack commented on a change in pull request #995: HBASE-23055 Alter
hbase:meta
URL: https://github.com/apache/hbase/pull/995#discussion_r364072688
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java
##
@@ -661,42 +662,54 @@ public void run(PRESP resp) {
new DisableTableProcedureBiConsumer(tableName));
}
+ /**
+ * Utility for completing passed {@link CompletableFuture}
future
+ * using passed parameters.
+ */
+ private static CompletableFuture
completeBoolean(CompletableFuture future,
Review comment:
Let me fix in next push.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-23645) Fix remaining Checkstyle violations in tests of hbase-common
[
https://issues.apache.org/jira/browse/HBASE-23645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010383#comment-17010383
]
Hudson commented on HBASE-23645:
Results for branch branch-2.1
[build #1765 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1765/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1765//console].
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1765//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1765//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Fix remaining Checkstyle violations in tests of hbase-common
>
>
> Key: HBASE-23645
> URL: https://issues.apache.org/jira/browse/HBASE-23645
> Project: HBase
> Issue Type: Improvement
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9
>
>
> In {{hbase-common}} Checkstyle reports a lot of violations. The remaining
> violations in the tests should be fixed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] bharathv commented on a change in pull request #995: HBASE-23055 Alter hbase:meta
bharathv commented on a change in pull request #995: HBASE-23055 Alter
hbase:meta
URL: https://github.com/apache/hbase/pull/995#discussion_r364070254
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKAsyncRegistry.java
##
@@ -229,6 +244,44 @@ private void
getMetaRegionLocation(CompletableFuture future,
});
}
+ @Override
+ public CompletableFuture getMetaTableState() {
Review comment:
think you missed the tests for this? we can add to TestZKAsyncRegistry.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] bharathv commented on a change in pull request #995: HBASE-23055 Alter hbase:meta
bharathv commented on a change in pull request #995: HBASE-23055 Alter
hbase:meta
URL: https://github.com/apache/hbase/pull/995#discussion_r364034396
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
##
@@ -948,22 +950,13 @@ protected Void waitOperationResult(long deadlineTs)
throws IOException, TimeoutE
@Override
public boolean isTableEnabled(final TableName tableName) throws IOException {
checkTableExists(tableName);
-return executeCallable(new RpcRetryingCallable() {
- @Override
- protected Boolean rpcCall(int callTimeout) throws Exception {
-TableState tableState =
MetaTableAccessor.getTableState(getConnection(), tableName);
-if (tableState == null) {
- throw new TableNotFoundException(tableName);
-}
-return tableState.inStates(TableState.State.ENABLED);
- }
-});
+return this.connection.getTableState(tableName).isEnabled();
Review comment:
Ack.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] bharathv commented on a change in pull request #995: HBASE-23055 Alter hbase:meta
bharathv commented on a change in pull request #995: HBASE-23055 Alter
hbase:meta
URL: https://github.com/apache/hbase/pull/995#discussion_r364035316
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java
##
@@ -661,42 +662,54 @@ public void run(PRESP resp) {
new DisableTableProcedureBiConsumer(tableName));
}
+ /**
+ * Utility for completing passed {@link CompletableFuture}
future
+ * using passed parameters.
+ */
+ private static CompletableFuture
completeBoolean(CompletableFuture future,
Review comment:
nit: I think checkTableState() or something would be a more appropriate name
since this is not a generic function that works for any boolean?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #995: HBASE-23055 Alter hbase:meta
Apache-HBase commented on issue #995: HBASE-23055 Alter hbase:meta URL: https://github.com/apache/hbase/pull/995#issuecomment-571893043 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 35s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 5 new or modified test files. | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 13s | branch-2 passed | | +1 :green_heart: | compile | 2m 29s | branch-2 passed | | +1 :green_heart: | checkstyle | 3m 6s | branch-2 passed | | +1 :green_heart: | shadedjars | 4m 17s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 51s | branch-2 passed | | +0 :ok: | spotbugs | 0m 35s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 5m 57s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 51s | the patch passed | | +1 :green_heart: | compile | 2m 30s | the patch passed | | +1 :green_heart: | javac | 2m 30s | the patch passed | | -1 :x: | checkstyle | 0m 44s | hbase-client: The patch generated 10 new + 348 unchanged - 5 fixed = 358 total (was 353) | | -1 :x: | checkstyle | 1m 30s | hbase-server: The patch generated 4 new + 400 unchanged - 28 fixed = 404 total (was 428) | | +1 :green_heart: | rubocop | 0m 3s | There were no new rubocop issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 13s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 15m 17s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 1m 51s | the patch passed | | +1 :green_heart: | findbugs | 6m 25s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 3m 6s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 3m 42s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 0m 49s | hbase-zookeeper in the patch passed. | | -1 :x: | unit | 148m 54s | hbase-server in the patch failed. | | +1 :green_heart: | unit | 7m 58s | hbase-shell in the patch passed. | | +1 :green_heart: | asflicense | 2m 40s | The patch does not generate ASF License warnings. | | | | 236m 45s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.client.TestAdmin2 | | | hadoop.hbase.client.TestAsyncTableAdminApi2 | | | hadoop.hbase.client.TestSnapshotTemporaryDirectory | | | hadoop.hbase.client.TestAsyncTableAdminApi3 | | | hadoop.hbase.client.TestSnapshotDFSTemporaryDirectory | | | hadoop.hbase.client.TestSnapshotTemporaryDirectoryWithRegionReplicas | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-995/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/995 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile rubocop | | uname | Linux d56b82ad10c9 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-995/out/precommit/personality/provided.sh | | git revision | branch-2 / 4f7de9e27e | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-995/2/artifact/out/diff-checkstyle-hbase-client.txt | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-995/2/artifact/out/diff-checkstyle-hbase-server.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-995/2/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-995/2/testReport/ | | Max. process+thread count | 4494 (vs. ulimit of 1) | | modules | C: hbase-common hbase-client hbase-zookeeper hbase-server
[GitHub] [hbase] Apache-HBase commented on issue #996: HBASE-23655 Fix flaky TestRSGroupsKillRS: should wait the SCP to finish
Apache-HBase commented on issue #996: HBASE-23655 Fix flaky TestRSGroupsKillRS: should wait the SCP to finish URL: https://github.com/apache/hbase/pull/996#issuecomment-571892871 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 9s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ branch-2.2 Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 36s | branch-2.2 passed | | +1 :green_heart: | compile | 0m 25s | branch-2.2 passed | | +1 :green_heart: | checkstyle | 0m 13s | branch-2.2 passed | | +1 :green_heart: | shadedjars | 4m 25s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 21s | branch-2.2 passed | | +0 :ok: | spotbugs | 0m 47s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 0m 46s | branch-2.2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 9s | the patch passed | | +1 :green_heart: | compile | 0m 24s | the patch passed | | +1 :green_heart: | javac | 0m 24s | the patch passed | | +1 :green_heart: | checkstyle | 0m 11s | the patch passed | | +1 :green_heart: | whitespace | 0m 1s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 23s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 16m 23s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 19s | the patch passed | | +1 :green_heart: | findbugs | 0m 52s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 4m 56s | hbase-rsgroup in the patch passed. | | +1 :green_heart: | asflicense | 0m 12s | The patch does not generate ASF License warnings. | | | | 52m 17s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-996/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/996 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 6d95c8ec68c8 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-996/out/precommit/personality/provided.sh | | git revision | branch-2.2 / 6d22b7c255 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-996/3/testReport/ | | Max. process+thread count | 3867 (vs. ulimit of 1) | | modules | C: hbase-rsgroup U: hbase-rsgroup | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-996/3/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-23055) Alter hbase:meta
[
https://issues.apache.org/jira/browse/HBASE-23055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack updated HBASE-23055:
--
Release Note:
Adds being able to edit hbase:meta table schema. For example,
hbase(main):005:0> disable 'hbase:meta'
Took 0.7362 seconds
hbase(main):006:0> alter 'hbase:meta', {NAME => 'info', DATA_BLOCK_ENCODING =>
'ROW_INDEX_V1'}
Updating all regions with the new schema...
All regions updated.
Done.
Took 1.2138 seconds
hbase(main):007:0> enable 'hbase:meta'
Took 0.7370 seconds
Queries of hbase:meta table state go via a new addition to the HBase Registry.
Default implementation is for hbase:meta table to be ENABLED. Adds an
implementation to ZKAsyncRegistry that queries zk mirrored table state (See
MirroringTableStateManager). If Master crashes while hbase:meta is disabled, on
new Master, hbase:meta will be enabled again.
Known issues are: is if you try to describe tables while hbase:meta is offline,
you'll get scanner timeout as we read all table descriptors even if we only
want one (TODO: refactor how descriptor is done); when hbase:meta is offline,
other requests to meta will fail(no duh!). This feature will add some load to
the zk ensemble as it now fields requests for hbase:meta table state (mitigated
some by a cache of table state added to Connection).
was:
Adds being able to edit hbase:meta table schema. For example,
hbase(main):005:0> disable 'hbase:meta'
Took 0.7362 seconds
hbase(main):006:0> alter 'hbase:meta', {NAME => 'info', DATA_BLOCK_ENCODING =>
'ROW_INDEX_V1'}
Updating all regions with the new schema...
All regions updated.
Done.
Took 1.2138 seconds
hbase(main):007:0> enable 'hbase:meta'
Took 0.7370 seconds
Queries of hbase:meta table state go via a new addition to the HBase Registry.
Default implementation is for hbase:meta table to be ENABLED. Adds an
implementation to ZKAsyncRegistry that queries zk mirrored table state (See
MirroringTableStateManager).
Known issue is if you try to describe tables while hbase:meta is offline,
you'll get scanner timeout as we read all table descriptors even if we only
want one (TODO: refactor how descriptor is done). When hbase:meta is offline,
other requests to meta will fail. If Master crashes while hbase:meta is
disabled, on new Master, hbase:meta will be enabled again.
> Alter hbase:meta
>
>
> Key: HBASE-23055
> URL: https://issues.apache.org/jira/browse/HBASE-23055
> Project: HBase
> Issue Type: Task
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> hbase:meta is currently hardcoded. Its schema cannot be change.
> This issue is about allowing edits to hbase:meta schema. It will allow our
> being able to set encodings such as the block-with-indexes which will help
> quell CPU usage on host carrying hbase:meta. A dynamic hbase:meta is first
> step on road to being able to split meta.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #998: HBASE-23658 Fix flaky TestSnapshotFromMaster
Apache-HBase commented on issue #998: HBASE-23658 Fix flaky TestSnapshotFromMaster URL: https://github.com/apache/hbase/pull/998#issuecomment-571891990 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 15s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 4s | master passed | | +1 :green_heart: | compile | 0m 57s | master passed | | +1 :green_heart: | checkstyle | 1m 19s | master passed | | +1 :green_heart: | shadedjars | 4m 39s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | master passed | | +0 :ok: | spotbugs | 4m 29s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 4m 28s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 2s | the patch passed | | +1 :green_heart: | compile | 0m 54s | the patch passed | | +1 :green_heart: | javac | 0m 54s | the patch passed | | -1 :x: | checkstyle | 1m 19s | hbase-server: The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 37s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 15m 50s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 36s | the patch passed | | +1 :green_heart: | findbugs | 4m 39s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 161m 32s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 0m 34s | The patch does not generate ASF License warnings. | | | | 220m 58s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-998/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/998 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 2f8ded0e5a04 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-998/out/precommit/personality/provided.sh | | git revision | master / 5a7c9443e9 | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-998/1/artifact/out/diff-checkstyle-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-998/1/testReport/ | | Max. process+thread count | 5278 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-998/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23645) Fix remaining Checkstyle violations in tests of hbase-common
[
https://issues.apache.org/jira/browse/HBASE-23645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010359#comment-17010359
]
Hudson commented on HBASE-23645:
Results for branch branch-2
[build #2409 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2409/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2409//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2409//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2409//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Fix remaining Checkstyle violations in tests of hbase-common
>
>
> Key: HBASE-23645
> URL: https://issues.apache.org/jira/browse/HBASE-23645
> Project: HBase
> Issue Type: Improvement
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9
>
>
> In {{hbase-common}} Checkstyle reports a lot of violations. The remaining
> violations in the tests should be fixed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23658) Fix flaky TestSnapshotFromMaster
[
https://issues.apache.org/jira/browse/HBASE-23658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-23658:
---
Attachment: HBASE-23658.branch-2.2.001.patch
> Fix flaky TestSnapshotFromMaster
>
>
> Key: HBASE-23658
> URL: https://issues.apache.org/jira/browse/HBASE-23658
> Project: HBase
> Issue Type: Bug
>Reporter: Guanghao Zhang
>Priority: Major
> Attachments: HBASE-23658.branch-2.2.001.patch
>
>
> testAsyncSnapshotWillNotBlockSnapshotHFileCleaner is flaky. The assert may
> fail.
> {code:java}
> assertTrue(master.getSnapshotManager().isTakingAnySnapshot());
> future.get(); // in branch-2.2, here is Thread.sleep
> assertFalse(master.getSnapshotManager().isTakingAnySnapshot());
> {code}
> See
> [https://builds.apache.org/job/HBase-Flaky-Tests/job/master/5227/testReport/junit/org.apache.hadoop.hbase.master.cleaner/TestSnapshotFromMaster/testAsyncSnapshotWillNotBlockSnapshotHFileCleaner/]
>
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/job/branch-2.2/lastSuccessfulBuild/artifact/dashboard.html]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #954: HBASE-23305: Master based registry implementation
Apache-HBase commented on issue #954: HBASE-23305: Master based registry implementation URL: https://github.com/apache/hbase/pull/954#issuecomment-571884012 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 10 new or modified test files. | ||| _ HBASE-18095/client-locate-meta-no-zookeeper Compile Tests _ | | +0 :ok: | mvndep | 0m 33s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 50s | HBASE-18095/client-locate-meta-no-zookeeper passed | | +1 :green_heart: | compile | 1m 45s | HBASE-18095/client-locate-meta-no-zookeeper passed | | +1 :green_heart: | checkstyle | 2m 33s | HBASE-18095/client-locate-meta-no-zookeeper passed | | +1 :green_heart: | shadedjars | 5m 2s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 17s | HBASE-18095/client-locate-meta-no-zookeeper passed | | +0 :ok: | spotbugs | 4m 55s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 6m 50s | HBASE-18095/client-locate-meta-no-zookeeper passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 26s | the patch passed | | +1 :green_heart: | compile | 1m 46s | the patch passed | | +1 :green_heart: | javac | 1m 46s | the patch passed | | +1 :green_heart: | checkstyle | 0m 26s | hbase-common: The patch generated 0 new + 4 unchanged - 1 fixed = 4 total (was 5) | | +1 :green_heart: | checkstyle | 0m 33s | hbase-client: The patch generated 0 new + 0 unchanged - 12 fixed = 0 total (was 12) | | +1 :green_heart: | checkstyle | 1m 30s | hbase-server: The patch generated 0 new + 223 unchanged - 50 fixed = 223 total (was 273) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 5m 6s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 17m 25s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 1m 16s | the patch passed | | +1 :green_heart: | findbugs | 7m 22s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 3m 8s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 55s | hbase-client in the patch passed. | | -1 :x: | unit | 174m 22s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 1m 9s | The patch does not generate ASF License warnings. | | | | 253m 59s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.client.TestFromClientSide | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-954/11/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/954 | | JIRA Issue | HBASE-23305 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 629e2ad8c337 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-954/out/precommit/personality/provided.sh | | git revision | HBASE-18095/client-locate-meta-no-zookeeper / d016cd9962 | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-954/11/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-954/11/testReport/ | | Max. process+thread count | 4468 (vs. ulimit of 1) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-954/11/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message
[jira] [Commented] (HBASE-23305) Implement master based AsyncRegistry
[
https://issues.apache.org/jira/browse/HBASE-23305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010347#comment-17010347
]
HBase QA commented on HBASE-23305:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
1s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 10 new or modified test
files. {color} |
|| || || || {color:brown} HBASE-18095/client-locate-meta-no-zookeeper Compile
Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
50s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed
{color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
45s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
33s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed
{color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
2s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
17s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed
{color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 4m
55s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
50s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed
{color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} hbase-common: The patch generated 0 new + 4
unchanged - 1 fixed = 4 total (was 5) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
33s{color} | {color:green} hbase-client: The patch generated 0 new + 0
unchanged - 12 fixed = 0 total (was 12) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
30s{color} | {color:green} hbase-server: The patch generated 0 new + 223
unchanged - 50 fixed = 223 total (was 273) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
6s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 25s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
8s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
55s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} |
[jira] [Resolved] (HBASE-23657) AsyncAdmin does not throw table operation exceptions
[
https://issues.apache.org/jira/browse/HBASE-23657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk resolved HBASE-23657.
--
Resolution: Invalid
not a problem. [~zhangduo] and [~bharathv] showed me the error of my ways.
thanks lads.
> AsyncAdmin does not throw table operation exceptions
>
>
> Key: HBASE-23657
> URL: https://issues.apache.org/jira/browse/HBASE-23657
> Project: HBase
> Issue Type: Bug
> Components: asyncclient, Client
>Reporter: Nick Dimiduk
>Priority: Major
> Attachments: 23657-demo-tests.patch
>
>
> {{AsyncAdmin}} does not propagate table operation exceptions back to the
> caller.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] infraio commented on issue #998: HBASE-23658 Fix flaky TestSnapshotFromMaster
infraio commented on issue #998: HBASE-23658 Fix flaky TestSnapshotFromMaster URL: https://github.com/apache/hbase/pull/998#issuecomment-571866814 > What is the test used for? It is named "testAsyncSnapshotWillNotBlockSnapshotHFileCleaner" but where is the HFileCleaner? Introduced by HBASE-21582. SnapshotManager#cleanupSentinels not runed and it will keep one snapshot handler. Then the HFileCleaner will not run too as there are snapshot is taking. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] infraio commented on a change in pull request #996: HBASE-23655 Fix flaky TestRSGroupsKillRS: should wait the SCP to finish
infraio commented on a change in pull request #996: HBASE-23655 Fix flaky
TestRSGroupsKillRS: should wait the SCP to finish
URL: https://github.com/apache/hbase/pull/996#discussion_r364040746
##
File path:
hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroupsKillRS.java
##
@@ -268,7 +269,9 @@ public void testLowerMetaGroupVersion() throws Exception{
assertTrue(VersionInfo.compareVersion(originVersion,
master.getRegionServerVersion(getServerName(servers.iterator().next( > 0);
LOG.debug("wait for META assigned...");
-TEST_UTIL.waitTableAvailable(tableName, 3);
+TEST_UTIL.waitFor(6, () ->
!TEST_UTIL.getHBaseCluster().getMaster().getProcedures().stream()
Review comment:
Yes. This is the root cause. In my test, the waitUntilAllRegionsAssigned and
waitTableAvailable not worked for meta table...
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache9 commented on issue #998: HBASE-23658 Fix flaky TestSnapshotFromMaster
Apache9 commented on issue #998: HBASE-23658 Fix flaky TestSnapshotFromMaster URL: https://github.com/apache/hbase/pull/998#issuecomment-571858856 What is the test used for? It is named "testAsyncSnapshotWillNotBlockSnapshotHFileCleaner" but where is the HFileCleaner? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23656) [MERGETOOL] HBASE Support Merge region by pattern
[ https://issues.apache.org/jira/browse/HBASE-23656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010253#comment-17010253 ] dingbaosheng commented on HBASE-23656: -- :) > [MERGETOOL] HBASE Support Merge region by pattern > - > > Key: HBASE-23656 > URL: https://issues.apache.org/jira/browse/HBASE-23656 > Project: HBase > Issue Type: New Feature > Components: master >Reporter: zhengsicheng >Priority: Minor > Fix For: 3.0.0 > > > Design Objective: > # Merge empty region > # Neat region > # merge expired region > Usage: bin/hbase onlinemerge [--tableName=] [--startRegion=] [--stopRegion=] > [--maxRegionSize=] [--maxRegionCreateTime=] [--numMaxMergePlans=] > [--targetRegionCount=] [--printExecutionPlan=] [--configMergePauseTime=] > Options: > --h or --h print help > --tableName table name must be not null > --startRegion start region > --stopRegion stop region > --maxRegionSize max region size Unit GB > --maxRegionCreateTime max Region Create Time /MM/dd HH:mm:ss > --numMaxMergePlans num MaxMerge Plans > --targetRegionCount target Region Count > --configMergePauseTime config Merge Pause Time In milliseconds > --printExecutionPlan Value default is true print execution plans false is > execution merge > Examples: > bin/hbase onlinemerge --tableName=test:test1 > --startRegion=test:test1,,1576835912332.01d0d6c2b41e204104524d9aec6074fb. > --stopRegion=test:test1,,1573044786980.0c9b5bd93f3b19eb9bd1a1011ddff66f. > --maxRegionSize=0 --maxRegionCreateTime=/MM/dd HH:mm:ss > --numMaxMergePlans=2 --targetRegionCount=4 --printExecutionPlan=false -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache9 commented on a change in pull request #996: HBASE-23655 Fix flaky TestRSGroupsKillRS: should wait the SCP to finish
Apache9 commented on a change in pull request #996: HBASE-23655 Fix flaky
TestRSGroupsKillRS: should wait the SCP to finish
URL: https://github.com/apache/hbase/pull/996#discussion_r364034596
##
File path:
hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroupsKillRS.java
##
@@ -237,12 +238,12 @@ public void testLowerMetaGroupVersion() throws Exception{
addGroup(groupName, groupRSCount);
// move hbase:meta to meta_group
-tableName = TableName.META_TABLE_NAME;
Set toAddTables = new HashSet<>();
-toAddTables.add(tableName);
+toAddTables.add(TableName.META_TABLE_NAME);
rsGroupAdmin.moveTables(toAddTables, groupName);
-
assertTrue(rsGroupAdmin.getRSGroupInfo(groupName).getTables().contains(tableName));
-TEST_UTIL.waitTableAvailable(tableName, 3);
+assertTrue(
+
rsGroupAdmin.getRSGroupInfo(groupName).getTables().contains(TableName.META_TABLE_NAME));
+TEST_UTIL.waitUntilAllRegionsAssigned(TableName.META_TABLE_NAME, 3);
Review comment:
Does this method work for meta table? I'm not sure, just asking. As lots of
the methods in HBTU just read the meta table so can not be used to test meta
table itself...
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache9 commented on a change in pull request #996: HBASE-23655 Fix flaky TestRSGroupsKillRS: should wait the SCP to finish
Apache9 commented on a change in pull request #996: HBASE-23655 Fix flaky
TestRSGroupsKillRS: should wait the SCP to finish
URL: https://github.com/apache/hbase/pull/996#discussion_r364034686
##
File path:
hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroupsKillRS.java
##
@@ -268,7 +269,9 @@ public void testLowerMetaGroupVersion() throws Exception{
assertTrue(VersionInfo.compareVersion(originVersion,
master.getRegionServerVersion(getServerName(servers.iterator().next( > 0);
LOG.debug("wait for META assigned...");
-TEST_UTIL.waitTableAvailable(tableName, 3);
+TEST_UTIL.waitFor(6, () ->
!TEST_UTIL.getHBaseCluster().getMaster().getProcedures().stream()
Review comment:
So this is the root cause.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-23658) Fix flaky TestSnapshotFromMaster
[
https://issues.apache.org/jira/browse/HBASE-23658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010229#comment-17010229
]
Guanghao Zhang commented on HBASE-23658:
Use waitfor to make the ut more stable.
> Fix flaky TestSnapshotFromMaster
>
>
> Key: HBASE-23658
> URL: https://issues.apache.org/jira/browse/HBASE-23658
> Project: HBase
> Issue Type: Bug
>Reporter: Guanghao Zhang
>Priority: Major
>
> testAsyncSnapshotWillNotBlockSnapshotHFileCleaner is flaky. The assert may
> fail.
> {code:java}
> assertTrue(master.getSnapshotManager().isTakingAnySnapshot());
> future.get(); // in branch-2.2, here is Thread.sleep
> assertFalse(master.getSnapshotManager().isTakingAnySnapshot());
> {code}
> See
> [https://builds.apache.org/job/HBase-Flaky-Tests/job/master/5227/testReport/junit/org.apache.hadoop.hbase.master.cleaner/TestSnapshotFromMaster/testAsyncSnapshotWillNotBlockSnapshotHFileCleaner/]
>
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/job/branch-2.2/lastSuccessfulBuild/artifact/dashboard.html]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] infraio opened a new pull request #998: HBASE-23658 Fix flaky TestSnapshotFromMaster
infraio opened a new pull request #998: HBASE-23658 Fix flaky TestSnapshotFromMaster URL: https://github.com/apache/hbase/pull/998 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #996: HBASE-23655 Fix flaky TestRSGroupsKillRS: should wait the SCP to finish
Apache-HBase commented on issue #996: HBASE-23655 Fix flaky TestRSGroupsKillRS: should wait the SCP to finish URL: https://github.com/apache/hbase/pull/996#issuecomment-571844666 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 3m 54s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ branch-2.2 Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 42s | branch-2.2 passed | | +1 :green_heart: | compile | 0m 24s | branch-2.2 passed | | +1 :green_heart: | checkstyle | 0m 13s | branch-2.2 passed | | +1 :green_heart: | shadedjars | 4m 26s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 21s | branch-2.2 passed | | +0 :ok: | spotbugs | 0m 48s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 0m 45s | branch-2.2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 12s | the patch passed | | +1 :green_heart: | compile | 0m 25s | the patch passed | | +1 :green_heart: | javac | 0m 25s | the patch passed | | +1 :green_heart: | checkstyle | 0m 12s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 22s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 16m 27s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 21s | the patch passed | | +1 :green_heart: | findbugs | 0m 52s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 4m 55s | hbase-rsgroup in the patch passed. | | +1 :green_heart: | asflicense | 0m 12s | The patch does not generate ASF License warnings. | | | | 55m 23s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-996/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/996 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 710a1fac4ca7 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-996/out/precommit/personality/provided.sh | | git revision | branch-2.2 / 6d22b7c255 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-996/2/testReport/ | | Max. process+thread count | 3977 (vs. ulimit of 1) | | modules | C: hbase-rsgroup U: hbase-rsgroup | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-996/2/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack commented on issue #995: HBASE-23055 Alter hbase:meta
saintstack commented on issue #995: HBASE-23055 Alter hbase:meta URL: https://github.com/apache/hbase/pull/995#issuecomment-571844515 Address helpful feedback. Fix test failures. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (HBASE-23658) Fix flaky TestSnapshotFromMaster
Guanghao Zhang created HBASE-23658:
--
Summary: Fix flaky TestSnapshotFromMaster
Key: HBASE-23658
URL: https://issues.apache.org/jira/browse/HBASE-23658
Project: HBase
Issue Type: Bug
Reporter: Guanghao Zhang
testAsyncSnapshotWillNotBlockSnapshotHFileCleaner is flaky. The assert may
fail.
{code:java}
assertTrue(master.getSnapshotManager().isTakingAnySnapshot());
future.get(); // in branch-2.2, here is Thread.sleep
assertFalse(master.getSnapshotManager().isTakingAnySnapshot());
{code}
See
[https://builds.apache.org/job/HBase-Flaky-Tests/job/master/5227/testReport/junit/org.apache.hadoop.hbase.master.cleaner/TestSnapshotFromMaster/testAsyncSnapshotWillNotBlockSnapshotHFileCleaner/]
[https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/job/branch-2.2/lastSuccessfulBuild/artifact/dashboard.html]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23655) Fix flaky TestRSGroupsKillRS: should wait the SCP to finish
[ https://issues.apache.org/jira/browse/HBASE-23655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-23655: --- Fix Version/s: 2.2.3 2.3.0 3.0.0 > Fix flaky TestRSGroupsKillRS: should wait the SCP to finish > --- > > Key: HBASE-23655 > URL: https://issues.apache.org/jira/browse/HBASE-23655 > Project: HBase > Issue Type: Bug >Affects Versions: 2.2.2 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 3.0.0, 2.3.0, 2.2.3 > > > In teardownAfterMethod, it will moveTables and delete RSGroups. It will fail > if not wait SCP to finish. > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23655) Fix flaky TestRSGroupsKillRS: should wait the SCP to finish
[ https://issues.apache.org/jira/browse/HBASE-23655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010214#comment-17010214 ] Guanghao Zhang commented on HBASE-23655: Ping [~zhangduo] for reviewing. > Fix flaky TestRSGroupsKillRS: should wait the SCP to finish > --- > > Key: HBASE-23655 > URL: https://issues.apache.org/jira/browse/HBASE-23655 > Project: HBase > Issue Type: Bug >Affects Versions: 2.2.2 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 3.0.0, 2.3.0, 2.2.3 > > > In teardownAfterMethod, it will moveTables and delete RSGroups. It will fail > if not wait SCP to finish. > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23655) Fix flaky TestRSGroupsKillRS: should wait the SCP to finish
[ https://issues.apache.org/jira/browse/HBASE-23655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010213#comment-17010213 ] Guanghao Zhang commented on HBASE-23655: Test it more than 10 times and passed locally. > Fix flaky TestRSGroupsKillRS: should wait the SCP to finish > --- > > Key: HBASE-23655 > URL: https://issues.apache.org/jira/browse/HBASE-23655 > Project: HBase > Issue Type: Bug >Affects Versions: 2.2.2 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > > In teardownAfterMethod, it will moveTables and delete RSGroups. It will fail > if not wait SCP to finish. > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-23655) Fix flaky TestRSGroupsKillRS: should wait the SCP to finish
[ https://issues.apache.org/jira/browse/HBASE-23655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang reassigned HBASE-23655: -- Assignee: Guanghao Zhang > Fix flaky TestRSGroupsKillRS: should wait the SCP to finish > --- > > Key: HBASE-23655 > URL: https://issues.apache.org/jira/browse/HBASE-23655 > Project: HBase > Issue Type: Bug >Affects Versions: 2.2.2 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > > In teardownAfterMethod, it will moveTables and delete RSGroups. It will fail > if not wait SCP to finish. > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] bharathv commented on issue #954: HBASE-23305: Master based registry implementation
bharathv commented on issue #954: HBASE-23305: Master based registry implementation URL: https://github.com/apache/hbase/pull/954#issuecomment-571830020 > [ERROR] org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor.testCheckAndDeleteWithCompareOp[0] Time elapsed: 0.142 s <<< ERROR! @ndimiduk I think I figured out the flaky test problem. I pushed another commit that deflakes the test and all of these TestClient* now pass in my IDE too. To summarize, JUnit parameterization (with test class hiearchy) is a PITA. Without my last commit, *every* unit test of *every parameterized test* run was reseting the cluster state. JUnit does not yet provide of a way to execute a hook at the end of each parameterized test run. So, I hacked my way around it. I'm hoping it should pass now. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] bharathv commented on a change in pull request #954: HBASE-23305: Master based registry implementation
bharathv commented on a change in pull request #954: HBASE-23305: Master based registry implementation URL: https://github.com/apache/hbase/pull/954#discussion_r363885862 ## File path: hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/BlockingRpcClient.java ## @@ -72,6 +74,14 @@ protected BlockingRpcConnection createConnection(ConnectionId remoteId) throws I return new BlockingRpcConnection(this, remoteId); } + @Override + public RpcChannel createHedgedRpcChannel(Set sns, User user, int rpcTimeout) Review comment: Done. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23645) Fix remaining Checkstyle violations in tests of hbase-common
[
https://issues.apache.org/jira/browse/HBASE-23645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010169#comment-17010169
]
Hudson commented on HBASE-23645:
Results for branch branch-2.2
[build #747 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/747/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/747//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/747//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/747//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Fix remaining Checkstyle violations in tests of hbase-common
>
>
> Key: HBASE-23645
> URL: https://issues.apache.org/jira/browse/HBASE-23645
> Project: HBase
> Issue Type: Improvement
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9
>
>
> In {{hbase-common}} Checkstyle reports a lot of violations. The remaining
> violations in the tests should be fixed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-18095) Provide an option for clients to find the server hosting META that does not involve the ZooKeeper client
[
https://issues.apache.org/jira/browse/HBASE-18095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010137#comment-17010137
]
Hudson commented on HBASE-18095:
Results for branch HBASE-18095/client-locate-meta-no-zookeeper
[build #32 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/32/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/32//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/32//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/32//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Provide an option for clients to find the server hosting META that does not
> involve the ZooKeeper client
>
>
> Key: HBASE-18095
> URL: https://issues.apache.org/jira/browse/HBASE-18095
> Project: HBase
> Issue Type: New Feature
> Components: Client
>Reporter: Andrew Kyle Purtell
>Assignee: Bharath Vissapragada
>Priority: Major
> Attachments: HBASE-18095.master-v1.patch, HBASE-18095.master-v2.patch
>
>
> Clients are required to connect to ZooKeeper to find the location of the
> regionserver hosting the meta table region. Site configuration provides the
> client a list of ZK quorum peers and the client uses an embedded ZK client to
> query meta location. Timeouts and retry behavior of this embedded ZK client
> are managed orthogonally to HBase layer settings and in some cases the ZK
> cannot manage what in theory the HBase client can, i.e. fail fast upon outage
> or network partition.
> We should consider new configuration settings that provide a list of
> well-known master and backup master locations, and with this information the
> client can contact any of the master processes directly. Any master in either
> active or passive state will track meta location and respond to requests for
> it with its cached last known location. If this location is stale, the client
> can ask again with a flag set that requests the master refresh its location
> cache and return the up-to-date location. Every client interaction with the
> cluster thus uses only HBase RPC as transport, with appropriate settings
> applied to the connection. The configuration toggle that enables this
> alternative meta location lookup should be false by default.
> This removes the requirement that HBase clients embed the ZK client and
> contact the ZK service directly at the beginning of the connection lifecycle.
> This has several benefits. ZK service need not be exposed to clients, and
> their potential abuse, yet no benefit ZK provides the HBase server cluster is
> compromised. Normalizing HBase client and ZK client timeout settings and
> retry behavior - in some cases, impossible, i.e. for fail-fast - is no longer
> necessary.
> And, from [~ghelmling]: There is an additional complication here for
> token-based authentication. When a delegation token is used for SASL
> authentication, the client uses the cluster ID obtained from Zookeeper to
> select the token identifier to use. So there would also need to be some
> Zookeeper-less, unauthenticated way to obtain the cluster ID as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] saintstack commented on a change in pull request #995: HBASE-23055 Alter hbase:meta
saintstack commented on a change in pull request #995: HBASE-23055 Alter
hbase:meta
URL: https://github.com/apache/hbase/pull/995#discussion_r363954749
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKAsyncRegistry.java
##
@@ -229,6 +242,33 @@ private void
getMetaRegionLocation(CompletableFuture future,
});
}
+ @Override
+ public CompletableFuture getMetaTableState() {
+CompletableFuture result =
+ getAndConvert(this.znodeMirroredMetaTableState,
ZKAsyncRegistry::getTableState)
+ .thenApply(state -> {
+return state == null ||
state.equals(ENABLED_META_TABLE_STATE.getState())?
+ ENABLED_META_TABLE_STATE: new TableState(TableName.META_TABLE_NAME,
state);
+ });
+// If completed exceptionally return default ENABLED state.
+return result.isCompletedExceptionally()? COMPLETED_GET_META_TABLE_STATE:
result;
Review comment:
You are right. Undid this. Letting any exception out instead of trying to
paper it over with a default.
Will add test.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #995: HBASE-23055 Alter hbase:meta
saintstack commented on a change in pull request #995: HBASE-23055 Alter
hbase:meta
URL: https://github.com/apache/hbase/pull/995#discussion_r363953726
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/MasterKeepAliveConnection.java
##
@@ -35,7 +38,8 @@
* final user code. Hence it's package protected.
*/
@InterfaceAudience.Private
-interface MasterKeepAliveConnection extends
MasterProtos.MasterService.BlockingInterface {
+interface MasterKeepAliveConnection extends
+MasterProtos.MasterService.BlockingInterface, Closeable {
// Do this instead of implement Closeable because closeable returning IOE is
PITA.
Review comment:
You are right.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #995: HBASE-23055 Alter hbase:meta
saintstack commented on a change in pull request #995: HBASE-23055 Alter
hbase:meta
URL: https://github.com/apache/hbase/pull/995#discussion_r363953416
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
##
@@ -948,22 +950,13 @@ protected Void waitOperationResult(long deadlineTs)
throws IOException, TimeoutE
@Override
public boolean isTableEnabled(final TableName tableName) throws IOException {
checkTableExists(tableName);
-return executeCallable(new RpcRetryingCallable() {
- @Override
- protected Boolean rpcCall(int callTimeout) throws Exception {
-TableState tableState =
MetaTableAccessor.getTableState(getConnection(), tableName);
-if (tableState == null) {
- throw new TableNotFoundException(tableName);
-}
-return tableState.inStates(TableState.State.ENABLED);
- }
-});
+return this.connection.getTableState(tableName).isEnabled();
Review comment:
Yes. Removed it so symmetric with isDisabled and retrying cached value
seemed odd.
Maybe I should put this back though just to minimize differences.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #995: HBASE-23055 Alter hbase:meta
saintstack commented on a change in pull request #995: HBASE-23055 Alter
hbase:meta
URL: https://github.com/apache/hbase/pull/995#discussion_r363952722
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
##
@@ -1112,7 +1119,7 @@ public static RegionInfo getRegionInfo(final Result r,
byte [] qualifier) {
public static TableState getTableState(Connection conn, TableName tableName)
throws IOException {
if (tableName.equals(TableName.META_TABLE_NAME)) {
- return new TableState(tableName, TableState.State.ENABLED);
+ throw new IllegalAccessError("Not for hbase:meta table state");
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #995: HBASE-23055 Alter hbase:meta
saintstack commented on a change in pull request #995: HBASE-23055 Alter
hbase:meta
URL: https://github.com/apache/hbase/pull/995#discussion_r363947735
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
##
@@ -154,6 +156,16 @@
public static final String RETRIES_BY_SERVER_KEY =
"hbase.client.retries.by.server";
private static final Logger LOG =
LoggerFactory.getLogger(ConnectionImplementation.class);
+ /**
+ * TableState cache.
+ */
+ private final LoadingCache tableStateCache;
Review comment:
Added more comment on the cache data member explaining why reproduced here:
* Table States change super rarely. In synchronous client, state can be
queried a lot
* particularly when Regions are moving. Its ok if we are not super
responsive noticing
* Table State change. So, cache the last look up for a period. Use
* {@link #TABLESTATE_CACHE_DURATION_MS} to change default of one second.
On your nit, yeah, I think it fine reading table state from cache even if it
stale. It will update after 1000ms. Region location lookups are more likely
because of server crash or region balance... If table is being offlined,
there'll be a little lag because of the cache but in scheme of things --
especially if a table disable -- I think it ok.
There is no cache for async client. It goes against async API. TODO.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #995: HBASE-23055 Alter hbase:meta
saintstack commented on a change in pull request #995: HBASE-23055 Alter
hbase:meta
URL: https://github.com/apache/hbase/pull/995#discussion_r363944743
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
##
@@ -330,6 +342,26 @@ public void newDead(ServerName sn) {
close();
throw e;
}
+// Create tablestate cache. Add a loader that know how to find table state.
+int duration = this.conf.getInt(TABLESTATE_CACHE_DURATION_MS, 1000);
+this.tableStateCache = CacheBuilder.newBuilder().
+ expireAfterWrite(duration, TimeUnit.MILLISECONDS).
+ recordStats().
Review comment:
Let me fix.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #995: HBASE-23055 Alter hbase:meta
saintstack commented on a change in pull request #995: HBASE-23055 Alter
hbase:meta
URL: https://github.com/apache/hbase/pull/995#discussion_r363944460
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRegistry.java
##
@@ -29,12 +30,26 @@
*/
@InterfaceAudience.Private
interface AsyncRegistry extends Closeable {
+ /**
+ * A completed CompletableFuture to host default hbase:meta table state
(ENABLED).
+ */
+ TableState ENABLED_META_TABLE_STATE =
+new TableState(TableName.META_TABLE_NAME, TableState.State.ENABLED);
+ CompletableFuture COMPLETED_GET_META_TABLE_STATE =
+CompletableFuture.completedFuture(ENABLED_META_TABLE_STATE);
/**
* Get the location of meta region.
*/
CompletableFuture getMetaRegionLocation();
+ /**
+ * The hbase:meta table state.
+ */
+ default CompletableFuture getMetaTableState() {
Review comment:
No. The opposite I'd say. Up to this, meta table state was hard-wired
ENABLED -- i.e. the default here. There are two Registry implementations
currently. The ZK-based one and the one you are working on. The ZKRegistry got
an impl as part of this patch. Will work on one for when your Registry lands.
Meantime, default impl.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #995: HBASE-23055 Alter hbase:meta
saintstack commented on a change in pull request #995: HBASE-23055 Alter
hbase:meta
URL: https://github.com/apache/hbase/pull/995#discussion_r363943136
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/TestHBaseMetaEdit.java
##
@@ -0,0 +1,111 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase;
+
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.ColumnFamilyDescriptor;
+import org.apache.hadoop.hbase.client.ColumnFamilyDescriptorBuilder;
+import org.apache.hadoop.hbase.client.RegionInfoBuilder;
+import org.apache.hadoop.hbase.client.TableDescriptor;
+import org.apache.hadoop.hbase.io.encoding.DataBlockEncoding;
+import org.apache.hadoop.hbase.regionserver.Region;
+import org.apache.hadoop.hbase.testclassification.LargeTests;
+import org.apache.hadoop.hbase.testclassification.MiscTests;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.ClassRule;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.rules.TestName;
+
+import java.io.IOException;
+import java.util.concurrent.ExecutionException;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertTrue;
+
+/**
+ * Test being able to edit hbase:meta.
+ */
+@Category({MiscTests.class, LargeTests.class})
+public class TestHBaseMetaEdit {
+ @ClassRule
+ public static final HBaseClassTestRule CLASS_RULE =
+ HBaseClassTestRule.forClass(TestHBaseMetaEdit.class);
+ @Rule
+ public TestName name = new TestName();
+ private final static HBaseTestingUtility UTIL = new HBaseTestingUtility();
+
+ @Before
+ public void before() throws Exception {
+UTIL.startMiniCluster();
+ }
+
+ @After
+ public void after() throws Exception {
+UTIL.shutdownMiniCluster();
+ }
+
+ /**
+ * Set versions, set HBASE-16213 indexed block encoding, and add a column
family.
+ * Verify they are all in place by looking at TableDescriptor AND by checking
+ * what the RegionServer sees after opening Region.
+ */
+ @Test
+ public void testEditMeta() throws IOException, ExecutionException,
InterruptedException {
Review comment:
Thanks.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Updated] (HBASE-23657) AsyncAdmin does not throw table operation exceptions
[
https://issues.apache.org/jira/browse/HBASE-23657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-23657:
-
Attachment: 23657-demo-tests.patch
> AsyncAdmin does not throw table operation exceptions
>
>
> Key: HBASE-23657
> URL: https://issues.apache.org/jira/browse/HBASE-23657
> Project: HBase
> Issue Type: Bug
> Components: asyncclient, Client
>Reporter: Nick Dimiduk
>Priority: Major
> Attachments: 23657-demo-tests.patch
>
>
> {{AsyncAdmin}} does not propagate table operation exceptions back to the
> caller.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23657) AsyncAdmin does not throw table operation exceptions
[
https://issues.apache.org/jira/browse/HBASE-23657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010036#comment-17010036
]
Nick Dimiduk commented on HBASE-23657:
--
I think they are swallowed somewhere in {{ProcedureBiConsumer}}, but I'm still
trying to understand the interfaces in use, reading the {{CompletableFuture}}
API docs and examples, so I don't have a more complete diagnosis. Attaching a
patch that provides a test demonstrating the issue, making use of
{{disableTable}}.
> AsyncAdmin does not throw table operation exceptions
>
>
> Key: HBASE-23657
> URL: https://issues.apache.org/jira/browse/HBASE-23657
> Project: HBase
> Issue Type: Bug
> Components: asyncclient, Client
>Reporter: Nick Dimiduk
>Priority: Major
> Attachments: 23657-demo-tests.patch
>
>
> {{AsyncAdmin}} does not propagate table operation exceptions back to the
> caller.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #978: HBASE-22285 A normalizer which merges small size regions with adjacen…
Apache-HBase commented on issue #978: HBASE-22285 A normalizer which merges small size regions with adjacen… URL: https://github.com/apache/hbase/pull/978#issuecomment-571745043 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 9s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 34s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 52s | master passed | | +1 :green_heart: | compile | 1m 28s | master passed | | +1 :green_heart: | checkstyle | 2m 1s | master passed | | +0 :ok: | refguide | 6m 26s | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | shadedjars | 5m 6s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 4s | master passed | | +0 :ok: | spotbugs | 5m 39s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 6m 27s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 43s | the patch passed | | +1 :green_heart: | compile | 1m 23s | the patch passed | | +1 :green_heart: | javac | 1m 23s | the patch passed | | +1 :green_heart: | checkstyle | 1m 55s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. | | +0 :ok: | refguide | 6m 18s | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | shadedjars | 5m 40s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 21m 35s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 1m 13s | the patch passed | | +1 :green_heart: | findbugs | 6m 44s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 3m 29s | hbase-common in the patch passed. | | -1 :x: | unit | 169m 22s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 50s | The patch does not generate ASF License warnings. | | | | 264m 3s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.master.TestMasterShutdown | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-978/7/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/978 | | Optional Tests | dupname asflicense javac javadoc unit refguide xml spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 7e8d6fb1b3d6 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-978/out/precommit/personality/provided.sh | | git revision | master / 5a7c9443e9 | | Default Java | 1.8.0_181 | | refguide | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-978/7/artifact/out/branch-site/book.html | | refguide | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-978/7/artifact/out/patch-site/book.html | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-978/7/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-978/7/testReport/ | | Max. process+thread count | 5467 (vs. ulimit of 1) | | modules | C: hbase-common hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-978/7/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the
[jira] [Created] (HBASE-23657) AsyncAdmin does not throw table operation exceptions
Nick Dimiduk created HBASE-23657:
Summary: AsyncAdmin does not throw table operation exceptions
Key: HBASE-23657
URL: https://issues.apache.org/jira/browse/HBASE-23657
Project: HBase
Issue Type: Bug
Components: asyncclient, Client
Reporter: Nick Dimiduk
{{AsyncAdmin}} does not propagate table operation exceptions back to the caller.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HBASE-23639) Change InterfaceAudience for ClusterManager interface and HBaseClusterManager class public
[ https://issues.apache.org/jira/browse/HBASE-23639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009962#comment-17009962 ] Andrew Kyle Purtell edited comment on HBASE-23639 at 1/7/20 6:35 PM: - Test code ... anything under test/ and compiled into test-jar artifacts ... is test code without any guarantees about anything and audience or stability annotations would be meaningless. What I would suggest is refactoring what you want into a new maven module, put the classes you want to be a stable/public API and implementation under src/ (instead of test/), and decorate the interfaces you desire to be public and stable with the appropriate annotation. Then we have a way to move forward. was (Author: apurtell): Test code ... anything under test/ and compiled into test-jar artifacts ... is test code without any guarantees about anything and audience or stability annotations would be meaningless. What I would suggest is refactoring what you want into a new maven module, put it under src/ instead of test/, and decorate the interfaces you desire to be public and stable with the appropriate annotation. Then we have a way to move forward. > Change InterfaceAudience for ClusterManager interface and HBaseClusterManager > class public > --- > > Key: HBASE-23639 > URL: https://issues.apache.org/jira/browse/HBASE-23639 > Project: HBase > Issue Type: Improvement >Reporter: Mihir Monani >Assignee: Mihir Monani >Priority: Minor > > In hbase-it package, Class like RESTApiClusterManager and HBaseClusterManager > which has some part of implementation code for Chaos Action. > > If any user wants to create private implementation which resembles > HBaseClusterManager, then it has to be merged with hbase-it package as > ClusterManager Interface and it's extended implementations are private. > > We should make them public so anyone can have their own implementation and > need to be merged with hbase-it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HBASE-23639) Change InterfaceAudience for ClusterManager interface and HBaseClusterManager class public
[ https://issues.apache.org/jira/browse/HBASE-23639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009962#comment-17009962 ] Andrew Kyle Purtell edited comment on HBASE-23639 at 1/7/20 6:35 PM: - Test code ... anything under test/ and compiled into test-jar artifacts ... is test code without any guarantees about anything and audience or stability annotations would be meaningless. What I would suggest is refactoring what you want into a new maven module, put it under src/ instead of test/, and decorate the interfaces you desire to be public and stable with the appropriate annotation. Then we have a way to move forward. was (Author: apurtell): Test code ... anything under test/ and compiled into test-jar artifacts ... is test code without any guarantees about anything and stability annotations would be meaningless. What I would suggest is refactoring what you want into a new maven module, put it under src/ instead of test/, and decorate the interfaces you desire to be public and stable with the appropriate annotation. Then we have a way to move forward. > Change InterfaceAudience for ClusterManager interface and HBaseClusterManager > class public > --- > > Key: HBASE-23639 > URL: https://issues.apache.org/jira/browse/HBASE-23639 > Project: HBase > Issue Type: Improvement >Reporter: Mihir Monani >Assignee: Mihir Monani >Priority: Minor > > In hbase-it package, Class like RESTApiClusterManager and HBaseClusterManager > which has some part of implementation code for Chaos Action. > > If any user wants to create private implementation which resembles > HBaseClusterManager, then it has to be merged with hbase-it package as > ClusterManager Interface and it's extended implementations are private. > > We should make them public so anyone can have their own implementation and > need to be merged with hbase-it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23639) Change InterfaceAudience for ClusterManager interface and HBaseClusterManager class public
[ https://issues.apache.org/jira/browse/HBASE-23639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009962#comment-17009962 ] Andrew Kyle Purtell commented on HBASE-23639: - Test code ... anything under test/ and compiled into test-jar artifacts ... is test code without any guarantees about anything and stability annotations would be meaningless. What I would suggest is refactoring what you want into a new maven module, put it under src/ instead of test/, and decorate the interfaces you desire to be public and stable with the appropriate annotation. Then we have a way to move forward. > Change InterfaceAudience for ClusterManager interface and HBaseClusterManager > class public > --- > > Key: HBASE-23639 > URL: https://issues.apache.org/jira/browse/HBASE-23639 > Project: HBase > Issue Type: Improvement >Reporter: Mihir Monani >Assignee: Mihir Monani >Priority: Minor > > In hbase-it package, Class like RESTApiClusterManager and HBaseClusterManager > which has some part of implementation code for Chaos Action. > > If any user wants to create private implementation which resembles > HBaseClusterManager, then it has to be merged with hbase-it package as > ClusterManager Interface and it's extended implementations are private. > > We should make them public so anyone can have their own implementation and > need to be merged with hbase-it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23640) Move HBaseClusterManager.CommandProvider and it's different implementations as seperate class/interface
[ https://issues.apache.org/jira/browse/HBASE-23640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009959#comment-17009959 ] Andrew Kyle Purtell commented on HBASE-23640: - In past discussions I have been a part of, we don't consider anything in test/ a public API. The class in question then was the RemoteTable *test helper* class that someone wanted to use as a public/supported client class for the REST API. Test code is test code, stability annotations are not meaningful. What I would suggest is refactoring what you want into a new maven module, put it under src/ instead of test/, and decorate the interfaces you desire to be public and stable with the appropriate annotation. Like what Nick said above. Then we have a way to move forward. For the record I am -1 any thought of making anything under test/ "public" > Move HBaseClusterManager.CommandProvider and it's different implementations > as seperate class/interface > --- > > Key: HBASE-23640 > URL: https://issues.apache.org/jira/browse/HBASE-23640 > Project: HBase > Issue Type: Improvement >Reporter: Mihir Monani >Assignee: Mihir Monani >Priority: Major > > HBaseClusterManager.CommandProvider and it's implementations are private to > HBaseClusterManager and hbase-it package. > If any one wants to do custom implementations of ClusterManager Interface, > they cant's use this code as it is not visible outside the hbase-it package. > This Jira is also dependents on > [HBASE-23639|https://issues.apache.org/jira/browse/HBASE-23639] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #997: HBASE-23656 [MERGETOOL]JDHBASE Support Merge region by pattern
Apache-HBase commented on issue #997: HBASE-23656 [MERGETOOL]JDHBASE Support Merge region by pattern URL: https://github.com/apache/hbase/pull/997#issuecomment-571711750 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 35s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 25s | master passed | | +1 :green_heart: | compile | 3m 5s | master passed | | +1 :green_heart: | checkstyle | 2m 34s | master passed | | +1 :green_heart: | shadedjars | 4m 46s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 3m 31s | master passed | | +0 :ok: | spotbugs | 4m 5s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 18m 55s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 7s | the patch passed | | +1 :green_heart: | compile | 3m 13s | the patch passed | | +1 :green_heart: | javac | 3m 13s | the patch passed | | -1 :x: | checkstyle | 2m 30s | root: The patch generated 15 new + 0 unchanged - 0 fixed = 15 total (was 0) | | +1 :green_heart: | shellcheck | 0m 4s | There were no new shellcheck issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 42s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 17m 29s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 3m 42s | the patch passed | | -1 :x: | findbugs | 4m 52s | hbase-server generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | -1 :x: | findbugs | 14m 36s | root generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | -1 :x: | unit | 169m 0s | root in the patch failed. | | +1 :green_heart: | asflicense | 1m 12s | The patch does not generate ASF License warnings. | | | | 273m 35s | | | Reason | Tests | |---:|:--| | FindBugs | module:hbase-server | | | Dead store to hris in org.apache.hadoop.hbase.util.OnlineMergeTool.executionPlan() At OnlineMergeTool.java:org.apache.hadoop.hbase.util.OnlineMergeTool.executionPlan() At OnlineMergeTool.java:[line 277] | | FindBugs | module:root | | | Dead store to hris in org.apache.hadoop.hbase.util.OnlineMergeTool.executionPlan() At OnlineMergeTool.java:org.apache.hadoop.hbase.util.OnlineMergeTool.executionPlan() At OnlineMergeTool.java:[line 277] | | Failed junit tests | hadoop.hbase.util.TestOnlineMergeTool | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-997/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/997 | | Optional Tests | dupname asflicense shellcheck shelldocs javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux bc820955f156 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-997/out/precommit/personality/provided.sh | | git revision | master / 1eafba5603 | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-997/1/artifact/out/diff-checkstyle-root.txt | | findbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-997/1/artifact/out/new-findbugs-hbase-server.html | | findbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-997/1/artifact/out/new-findbugs-root.html | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-997/1/artifact/out/patch-unit-root.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-997/1/testReport/ | | Max. process+thread count | 4927 (vs. ulimit of 1) |
[GitHub] [hbase] ndimiduk commented on a change in pull request #954: HBASE-23305: Master based registry implementation
ndimiduk commented on a change in pull request #954: HBASE-23305: Master based
registry implementation
URL: https://github.com/apache/hbase/pull/954#discussion_r363839020
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/AbstractRpcClient.java
##
@@ -527,6 +528,12 @@ public RpcChannel createRpcChannel(ServerName sn, User
user, int rpcTimeout)
return new RpcChannelImplementation(this, createAddr(sn), user,
rpcTimeout);
}
+ @Override
+ public RpcChannel createHedgedRpcChannel(Set sns, User user, int
rpcTimeout)
+ throws UnknownHostException {
+throw new NotImplementedException("Should be called on an
AbstractRpcClient implementation.");
Review comment:
I think you mean NettyRpcClient. Simply "method not supported for this
implementation." is fine.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] ndimiduk commented on a change in pull request #954: HBASE-23305: Master based registry implementation
ndimiduk commented on a change in pull request #954: HBASE-23305: Master based registry implementation URL: https://github.com/apache/hbase/pull/954#discussion_r363839731 ## File path: hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/BlockingRpcClient.java ## @@ -72,6 +74,14 @@ protected BlockingRpcConnection createConnection(ConnectionId remoteId) throws I return new BlockingRpcConnection(this, remoteId); } + @Override + public RpcChannel createHedgedRpcChannel(Set sns, User user, int rpcTimeout) Review comment: There's no need for an implementation of this method on this class -- it doesn't support this part of the interface and the parent abstract class's implementation is sufficient. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] ndimiduk commented on issue #954: HBASE-23305: Master based registry implementation
ndimiduk commented on issue #954: HBASE-23305: Master based registry implementation URL: https://github.com/apache/hbase/pull/954#issuecomment-571664717 Surprising. ``` [ERROR] org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor.testCheckAndDeleteWithCompareOp[0] Time elapsed: 0.142 s <<< ERROR! java.lang.OutOfMemoryError: Java heap space at org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor.(TestFromClientSideWithCoprocessor.java:50) ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] busbey merged pull request #993: HBASE-22908 Link To HBase 1.x Documentation on HBase Site
busbey merged pull request #993: HBASE-22908 Link To HBase 1.x Documentation on HBase Site URL: https://github.com/apache/hbase/pull/993 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23645) Fix remaining Checkstyle violations in tests of hbase-common
[
https://issues.apache.org/jira/browse/HBASE-23645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009783#comment-17009783
]
Hudson commented on HBASE-23645:
Results for branch master
[build #1591 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1591/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1591//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1591//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1591//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Fix remaining Checkstyle violations in tests of hbase-common
>
>
> Key: HBASE-23645
> URL: https://issues.apache.org/jira/browse/HBASE-23645
> Project: HBase
> Issue Type: Improvement
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9
>
>
> In {{hbase-common}} Checkstyle reports a lot of violations. The remaining
> violations in the tests should be fixed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] wchevreuil commented on issue #997: HBASE-23656 [MERGETOOL]JDHBASE Support Merge region by pattern
wchevreuil commented on issue #997: HBASE-23656 [MERGETOOL]JDHBASE Support Merge region by pattern URL: https://github.com/apache/hbase/pull/997#issuecomment-571614569 Thanks for the contribution, @mygood ! As mentioned on review comment, I think we could add these as part of the merge tool already being implemented on HBASE-23562. Feel free to share your thoughts on HBASE-23562 PR. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Assigned] (HBASE-23656) [MERGETOOL] HBASE Support Merge region by pattern
[ https://issues.apache.org/jira/browse/HBASE-23656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xuqinya reassigned HBASE-23656: --- Assignee: (was: xuqinya) > [MERGETOOL] HBASE Support Merge region by pattern > - > > Key: HBASE-23656 > URL: https://issues.apache.org/jira/browse/HBASE-23656 > Project: HBase > Issue Type: New Feature > Components: master >Reporter: zhengsicheng >Priority: Minor > Fix For: 3.0.0 > > > Design Objective: > # Merge empty region > # Neat region > # merge expired region > Usage: bin/hbase onlinemerge [--tableName=] [--startRegion=] [--stopRegion=] > [--maxRegionSize=] [--maxRegionCreateTime=] [--numMaxMergePlans=] > [--targetRegionCount=] [--printExecutionPlan=] [--configMergePauseTime=] > Options: > --h or --h print help > --tableName table name must be not null > --startRegion start region > --stopRegion stop region > --maxRegionSize max region size Unit GB > --maxRegionCreateTime max Region Create Time /MM/dd HH:mm:ss > --numMaxMergePlans num MaxMerge Plans > --targetRegionCount target Region Count > --configMergePauseTime config Merge Pause Time In milliseconds > --printExecutionPlan Value default is true print execution plans false is > execution merge > Examples: > bin/hbase onlinemerge --tableName=test:test1 > --startRegion=test:test1,,1576835912332.01d0d6c2b41e204104524d9aec6074fb. > --stopRegion=test:test1,,1573044786980.0c9b5bd93f3b19eb9bd1a1011ddff66f. > --maxRegionSize=0 --maxRegionCreateTime=/MM/dd HH:mm:ss > --numMaxMergePlans=2 --targetRegionCount=4 --printExecutionPlan=false -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-23656) [MERGETOOL] HBASE Support Merge region by pattern
[ https://issues.apache.org/jira/browse/HBASE-23656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xuqinya reassigned HBASE-23656: --- Assignee: xuqinya > [MERGETOOL] HBASE Support Merge region by pattern > - > > Key: HBASE-23656 > URL: https://issues.apache.org/jira/browse/HBASE-23656 > Project: HBase > Issue Type: New Feature > Components: master >Reporter: zhengsicheng >Assignee: xuqinya >Priority: Minor > Fix For: 3.0.0 > > > Design Objective: > # Merge empty region > # Neat region > # merge expired region > Usage: bin/hbase onlinemerge [--tableName=] [--startRegion=] [--stopRegion=] > [--maxRegionSize=] [--maxRegionCreateTime=] [--numMaxMergePlans=] > [--targetRegionCount=] [--printExecutionPlan=] [--configMergePauseTime=] > Options: > --h or --h print help > --tableName table name must be not null > --startRegion start region > --stopRegion stop region > --maxRegionSize max region size Unit GB > --maxRegionCreateTime max Region Create Time /MM/dd HH:mm:ss > --numMaxMergePlans num MaxMerge Plans > --targetRegionCount target Region Count > --configMergePauseTime config Merge Pause Time In milliseconds > --printExecutionPlan Value default is true print execution plans false is > execution merge > Examples: > bin/hbase onlinemerge --tableName=test:test1 > --startRegion=test:test1,,1576835912332.01d0d6c2b41e204104524d9aec6074fb. > --stopRegion=test:test1,,1573044786980.0c9b5bd93f3b19eb9bd1a1011ddff66f. > --maxRegionSize=0 --maxRegionCreateTime=/MM/dd HH:mm:ss > --numMaxMergePlans=2 --targetRegionCount=4 --printExecutionPlan=false -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23656) [MERGETOOL] HBASE Support Merge region by pattern
[ https://issues.apache.org/jira/browse/HBASE-23656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengsicheng updated HBASE-23656: - Summary: [MERGETOOL] HBASE Support Merge region by pattern (was: [MERGETOOL]JDHBASE Support Merge region by pattern) > [MERGETOOL] HBASE Support Merge region by pattern > - > > Key: HBASE-23656 > URL: https://issues.apache.org/jira/browse/HBASE-23656 > Project: HBase > Issue Type: New Feature > Components: master >Reporter: zhengsicheng >Priority: Minor > Fix For: 3.0.0 > > > Design Objective: > # Merge empty region > # Neat region > # merge expired region > Usage: bin/hbase onlinemerge [--tableName=] [--startRegion=] [--stopRegion=] > [--maxRegionSize=] [--maxRegionCreateTime=] [--numMaxMergePlans=] > [--targetRegionCount=] [--printExecutionPlan=] [--configMergePauseTime=] > Options: > --h or --h print help > --tableName table name must be not null > --startRegion start region > --stopRegion stop region > --maxRegionSize max region size Unit GB > --maxRegionCreateTime max Region Create Time /MM/dd HH:mm:ss > --numMaxMergePlans num MaxMerge Plans > --targetRegionCount target Region Count > --configMergePauseTime config Merge Pause Time In milliseconds > --printExecutionPlan Value default is true print execution plans false is > execution merge > Examples: > bin/hbase onlinemerge --tableName=test:test1 > --startRegion=test:test1,,1576835912332.01d0d6c2b41e204104524d9aec6074fb. > --stopRegion=test:test1,,1573044786980.0c9b5bd93f3b19eb9bd1a1011ddff66f. > --maxRegionSize=0 --maxRegionCreateTime=/MM/dd HH:mm:ss > --numMaxMergePlans=2 --targetRegionCount=4 --printExecutionPlan=false -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23656) [MERGETOOL]JDHBASE Support Merge region by pattern
[ https://issues.apache.org/jira/browse/HBASE-23656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengsicheng updated HBASE-23656: - Description: Design Objective: # Merge empty region # Neat region # merge expired region Usage: bin/hbase onlinemerge [--tableName=] [--startRegion=] [--stopRegion=] [--maxRegionSize=] [--maxRegionCreateTime=] [--numMaxMergePlans=] [--targetRegionCount=] [--printExecutionPlan=] [--configMergePauseTime=] Options: --h or --h print help --tableName table name must be not null --startRegion start region --stopRegion stop region --maxRegionSize max region size Unit GB --maxRegionCreateTime max Region Create Time /MM/dd HH:mm:ss --numMaxMergePlans num MaxMerge Plans --targetRegionCount target Region Count --configMergePauseTime config Merge Pause Time In milliseconds --printExecutionPlan Value default is true print execution plans false is execution merge Examples: bin/hbase onlinemerge --tableName=test:test1 --startRegion=test:test1,,1576835912332.01d0d6c2b41e204104524d9aec6074fb. --stopRegion=test:test1,,1573044786980.0c9b5bd93f3b19eb9bd1a1011ddff66f. --maxRegionSize=0 --maxRegionCreateTime=/MM/dd HH:mm:ss --numMaxMergePlans=2 --targetRegionCount=4 --printExecutionPlan=false was: Usage: bin/hbase onlinemerge [--tableName=] [--startRegion=] [--stopRegion=] [--maxRegionSize=] [--maxRegionCreateTime=] [--numMaxMergePlans=] [--targetRegionCount=] [--printExecutionPlan=] [--configMergePauseTime=] Options: --h or --h print help --tableName table name must be not null --startRegion start region --stopRegion stop region --maxRegionSize max region size Unit GB --maxRegionCreateTime max Region Create Time /MM/dd HH:mm:ss --numMaxMergePlans num MaxMerge Plans --targetRegionCount target Region Count --configMergePauseTime config Merge Pause Time In milliseconds --printExecutionPlan Value default is true print execution plans false is execution merge Examples: bin/hbase onlinemerge --tableName=test:test1 --startRegion=test:test1,,1576835912332.01d0d6c2b41e204104524d9aec6074fb. --stopRegion=test:test1,,1573044786980.0c9b5bd93f3b19eb9bd1a1011ddff66f. --maxRegionSize=0 --maxRegionCreateTime=/MM/dd HH:mm:ss --numMaxMergePlans=2 --targetRegionCount=4 --printExecutionPlan=false > [MERGETOOL]JDHBASE Support Merge region by pattern > -- > > Key: HBASE-23656 > URL: https://issues.apache.org/jira/browse/HBASE-23656 > Project: HBase > Issue Type: New Feature > Components: master >Reporter: zhengsicheng >Priority: Minor > Fix For: 3.0.0 > > > Design Objective: > # Merge empty region > # Neat region > # merge expired region > Usage: bin/hbase onlinemerge [--tableName=] [--startRegion=] [--stopRegion=] > [--maxRegionSize=] [--maxRegionCreateTime=] [--numMaxMergePlans=] > [--targetRegionCount=] [--printExecutionPlan=] [--configMergePauseTime=] > Options: > --h or --h print help > --tableName table name must be not null > --startRegion start region > --stopRegion stop region > --maxRegionSize max region size Unit GB > --maxRegionCreateTime max Region Create Time /MM/dd HH:mm:ss > --numMaxMergePlans num MaxMerge Plans > --targetRegionCount target Region Count > --configMergePauseTime config Merge Pause Time In milliseconds > --printExecutionPlan Value default is true print execution plans false is > execution merge > Examples: > bin/hbase onlinemerge --tableName=test:test1 > --startRegion=test:test1,,1576835912332.01d0d6c2b41e204104524d9aec6074fb. > --stopRegion=test:test1,,1573044786980.0c9b5bd93f3b19eb9bd1a1011ddff66f. > --maxRegionSize=0 --maxRegionCreateTime=/MM/dd HH:mm:ss > --numMaxMergePlans=2 --targetRegionCount=4 --printExecutionPlan=false -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] mygood opened a new pull request #997: HBASE-23656 [MERGETOOL]JDHBASE Support Merge region by pattern
mygood opened a new pull request #997: HBASE-23656 [MERGETOOL]JDHBASE Support Merge region by pattern URL: https://github.com/apache/hbase/pull/997 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (HBASE-23656) [MERGETOOL]JDHBASE Support Merge region by pattern
zhengsicheng created HBASE-23656: Summary: [MERGETOOL]JDHBASE Support Merge region by pattern Key: HBASE-23656 URL: https://issues.apache.org/jira/browse/HBASE-23656 Project: HBase Issue Type: New Feature Components: master Reporter: zhengsicheng Fix For: 3.0.0 Usage: bin/hbase onlinemerge [--tableName=] [--startRegion=] [--stopRegion=] [--maxRegionSize=] [--maxRegionCreateTime=] [--numMaxMergePlans=] [--targetRegionCount=] [--printExecutionPlan=] [--configMergePauseTime=] Options: --h or --h print help --tableName table name must be not null --startRegion start region --stopRegion stop region --maxRegionSize max region size Unit GB --maxRegionCreateTime max Region Create Time /MM/dd HH:mm:ss --numMaxMergePlans num MaxMerge Plans --targetRegionCount target Region Count --configMergePauseTime config Merge Pause Time In milliseconds --printExecutionPlan Value default is true print execution plans false is execution merge Examples: bin/hbase onlinemerge --tableName=test:test1 --startRegion=test:test1,,1576835912332.01d0d6c2b41e204104524d9aec6074fb. --stopRegion=test:test1,,1573044786980.0c9b5bd93f3b19eb9bd1a1011ddff66f. --maxRegionSize=0 --maxRegionCreateTime=/MM/dd HH:mm:ss --numMaxMergePlans=2 --targetRegionCount=4 --printExecutionPlan=false -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #996: HBASE-23655 Fix flaky TestRSGroupsKillRS: should wait the SCP to finish
Apache-HBase commented on issue #996: HBASE-23655 Fix flaky TestRSGroupsKillRS: should wait the SCP to finish URL: https://github.com/apache/hbase/pull/996#issuecomment-571581773 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 10s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ branch-2.2 Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 39s | branch-2.2 passed | | +1 :green_heart: | compile | 0m 24s | branch-2.2 passed | | +1 :green_heart: | checkstyle | 0m 13s | branch-2.2 passed | | +1 :green_heart: | shadedjars | 4m 27s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 21s | branch-2.2 passed | | +0 :ok: | spotbugs | 0m 48s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 0m 47s | branch-2.2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 14s | the patch passed | | +1 :green_heart: | compile | 0m 25s | the patch passed | | +1 :green_heart: | javac | 0m 25s | the patch passed | | -1 :x: | checkstyle | 0m 11s | hbase-rsgroup: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 25s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 16m 32s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 19s | the patch passed | | +1 :green_heart: | findbugs | 0m 54s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 4m 50s | hbase-rsgroup in the patch passed. | | +1 :green_heart: | asflicense | 0m 12s | The patch does not generate ASF License warnings. | | | | 52m 46s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-996/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/996 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 2cf6d35a0d99 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-996/out/precommit/personality/provided.sh | | git revision | branch-2.2 / 6d22b7c255 | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-996/1/artifact/out/diff-checkstyle-hbase-rsgroup.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-996/1/testReport/ | | Max. process+thread count | 3961 (vs. ulimit of 1) | | modules | C: hbase-rsgroup U: hbase-rsgroup | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-996/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] infraio opened a new pull request #996: HBASE-23655 Fix flaky TestRSGroupsKillRS: should wait the SCP to finish
infraio opened a new pull request #996: HBASE-23655 Fix flaky TestRSGroupsKillRS: should wait the SCP to finish URL: https://github.com/apache/hbase/pull/996 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-23655) Fix flaky TestRSGroupsKillRS: should wait the SCP to finish
[ https://issues.apache.org/jira/browse/HBASE-23655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-23655: --- Description: In teardownAfterMethod, it will moveTables and delete RSGroups. It will fail if not wait SCP to finish. > Fix flaky TestRSGroupsKillRS: should wait the SCP to finish > --- > > Key: HBASE-23655 > URL: https://issues.apache.org/jira/browse/HBASE-23655 > Project: HBase > Issue Type: Bug >Affects Versions: 2.2.2 >Reporter: Guanghao Zhang >Priority: Major > > In teardownAfterMethod, it will moveTables and delete RSGroups. It will fail > if not wait SCP to finish. > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-23655) Fix flaky TestRSGroupsKillRS: should wait the SCP to finish
Guanghao Zhang created HBASE-23655: -- Summary: Fix flaky TestRSGroupsKillRS: should wait the SCP to finish Key: HBASE-23655 URL: https://issues.apache.org/jira/browse/HBASE-23655 Project: HBase Issue Type: Bug Affects Versions: 2.2.2 Reporter: Guanghao Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase-operator-tools] wchevreuil commented on issue #46: HBASE-23562 [operator tools] Add a RegionsMerge tool that allows for …
wchevreuil commented on issue #46: HBASE-23562 [operator tools] Add a RegionsMerge tool that allows for … URL: https://github.com/apache/hbase-operator-tools/pull/46#issuecomment-571532798 - Tests are passing now; - Updated readme to mention that MOB data is not considered when calculating region size; - The checkstyle issue is weird, as it's pointing to _RegionsMerger_ under _hbck2_ module, but it has been moved from there on subsequent PR commits. Ping @busbey: on the open questions, would you think the CLI option a must for this first version? Also, any thoughts on avoiding using `@Private` interfaces and which actual visibility level this should have? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #990: HBASE-23646 Resolved remaining Checkstyle violations in tests of hbase-rest
Apache-HBase commented on issue #990: HBASE-23646 Resolved remaining Checkstyle violations in tests of hbase-rest URL: https://github.com/apache/hbase/pull/990#issuecomment-571520226 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 36s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 5 new or modified test files. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 36s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 42s | master passed | | +1 :green_heart: | compile | 0m 43s | master passed | | +1 :green_heart: | checkstyle | 2m 44s | master passed | | +1 :green_heart: | shadedjars | 4m 52s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | master passed | | +0 :ok: | spotbugs | 0m 55s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +0 :ok: | findbugs | 0m 17s | branch/hbase-checkstyle no findbugs output file (findbugsXml.xml) | | -0 :warning: | patch | 1m 9s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 6s | the patch passed | | +1 :green_heart: | compile | 0m 46s | the patch passed | | +1 :green_heart: | javac | 0m 46s | the patch passed | | +1 :green_heart: | checkstyle | 2m 47s | root: The patch generated 0 new + 0 unchanged - 168 fixed = 0 total (was 168) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedjars | 4m 45s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 15m 44s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 36s | the patch passed | | +0 :ok: | findbugs | 0m 14s | hbase-checkstyle has no data from findbugs | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 14s | hbase-checkstyle in the patch passed. | | +1 :green_heart: | unit | 4m 16s | hbase-rest in the patch passed. | | +1 :green_heart: | asflicense | 0m 26s | The patch does not generate ASF License warnings. | | | | 59m 46s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-990/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/990 | | Optional Tests | dupname asflicense checkstyle javac javadoc unit xml spotbugs findbugs shadedjars hadoopcheck hbaseanti compile | | uname | Linux 5f13f4516290 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-990/out/precommit/personality/provided.sh | | git revision | master / 1eafba5603 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-990/2/testReport/ | | Max. process+thread count | 1976 (vs. ulimit of 1) | | modules | C: hbase-checkstyle hbase-rest U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-990/2/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #991: HBASE-23651 Region balance throttling can be disabled
Apache-HBase commented on issue #991: HBASE-23651 Region balance throttling can be disabled URL: https://github.com/apache/hbase/pull/991#issuecomment-571516661 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 3s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 33s | master passed | | +1 :green_heart: | compile | 0m 56s | master passed | | +1 :green_heart: | checkstyle | 1m 24s | master passed | | +1 :green_heart: | shadedjars | 4m 38s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | master passed | | +0 :ok: | spotbugs | 4m 30s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 4m 28s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 6s | the patch passed | | +1 :green_heart: | compile | 0m 57s | the patch passed | | +1 :green_heart: | javac | 0m 57s | the patch passed | | +1 :green_heart: | checkstyle | 1m 21s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 39s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 15m 53s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 36s | the patch passed | | +1 :green_heart: | findbugs | 4m 48s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 155m 29s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 0m 34s | The patch does not generate ASF License warnings. | | | | 215m 24s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-991/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/991 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux f474e9787b27 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-991/out/precommit/personality/provided.sh | | git revision | master / 33f45d4413 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-991/2/testReport/ | | Max. process+thread count | 5043 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-991/2/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22514) Move rsgroup feature into core of HBase
[
https://issues.apache.org/jira/browse/HBASE-22514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009563#comment-17009563
]
Hudson commented on HBASE-22514:
Results for branch HBASE-22514
[build #234 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/234/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/234//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/234//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/234//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Move rsgroup feature into core of HBase
> ---
>
> Key: HBASE-22514
> URL: https://issues.apache.org/jira/browse/HBASE-22514
> Project: HBase
> Issue Type: Umbrella
> Components: Admin, Client, rsgroup
>Reporter: Yechao Chen
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HBASE-22514.master.001.patch,
> image-2019-05-31-18-25-38-217.png
>
>
> The class RSGroupAdminClient is not public
> we need to use java api RSGroupAdminClient to manager RSG
> so RSGroupAdminClient should be public
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] mnpoonia commented on a change in pull request #978: HBASE-22285 A normalizer which merges small size regions with adjacen…
mnpoonia commented on a change in pull request #978: HBASE-22285 A normalizer
which merges small size regions with adjacen…
URL: https://github.com/apache/hbase/pull/978#discussion_r363658766
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/AbstractRegionNormalizer.java
##
@@ -0,0 +1,213 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.master.normalizer;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.hadoop.hbase.RegionMetrics;
+import org.apache.hadoop.hbase.ServerName;
+import org.apache.hadoop.hbase.Size;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.MasterSwitchType;
+import org.apache.hadoop.hbase.client.RegionInfo;
+import org.apache.hadoop.hbase.client.TableDescriptor;
+import org.apache.hadoop.hbase.master.MasterRpcServices;
+import org.apache.hadoop.hbase.master.MasterServices;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hbase.thirdparty.com.google.protobuf.ServiceException;
+
+import org.apache.hadoop.hbase.shaded.protobuf.RequestConverter;
+
+@InterfaceAudience.Private
+public abstract class AbstractRegionNormalizer implements RegionNormalizer {
+ private static final Logger LOG =
LoggerFactory.getLogger(AbstractRegionNormalizer.class);
+ protected MasterServices masterServices;
+ protected MasterRpcServices masterRpcServices;
Review comment:
I think it is not possible unless we move the initialization of these two in
constructor.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] mnpoonia commented on a change in pull request #978: HBASE-22285 A normalizer which merges small size regions with adjacen…
mnpoonia commented on a change in pull request #978: HBASE-22285 A normalizer which merges small size regions with adjacen… URL: https://github.com/apache/hbase/pull/978#discussion_r363660079 ## File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SimpleRegionNormalizer.java ## @@ -18,73 +18,45 @@ */ package org.apache.hadoop.hbase.master.normalizer; -import java.io.IOException; import java.util.ArrayList; import java.util.Collections; import java.util.Comparator; import java.util.List; import org.apache.hadoop.hbase.HBaseConfiguration; import org.apache.hadoop.hbase.HBaseIOException; -import org.apache.hadoop.hbase.RegionMetrics; -import org.apache.hadoop.hbase.ServerName; -import org.apache.hadoop.hbase.Size; import org.apache.hadoop.hbase.TableName; -import org.apache.hadoop.hbase.client.MasterSwitchType; import org.apache.hadoop.hbase.client.RegionInfo; -import org.apache.hadoop.hbase.client.TableDescriptor; -import org.apache.hadoop.hbase.master.MasterRpcServices; -import org.apache.hadoop.hbase.master.MasterServices; import org.apache.hadoop.hbase.master.normalizer.NormalizationPlan.PlanType; import org.apache.yetus.audience.InterfaceAudience; import org.slf4j.Logger; import org.slf4j.LoggerFactory; -import org.apache.hadoop.hbase.shaded.protobuf.RequestConverter; - /** - * Simple implementation of region normalizer. - * - * Logic in use: - * - * - * Get all regions of a given table - * Get avg size S of each region (by total size of store files reported in RegionMetrics) - * Seek every single region one by one. If a region R0 is bigger than S * 2, it is - * kindly requested to split. Thereon evaluate the next region R1 - * Otherwise, if R0 + R1 is smaller than S, R0 and R1 are kindly requested to merge. - * Thereon evaluate the next region R2 - * Otherwise, R1 is evaluated + * Simple implementation of region normalizer. Logic in use: + * + * Get all regions of a given table + * Get avg size S of each region (by total size of store files reported in RegionMetrics) + * Seek every single region one by one. If a region R0 is bigger than S * 2, it is kindly + * requested to split. Thereon evaluate the next region R1 + * Otherwise, if R0 + R1 is smaller than S, R0 and R1 are kindly requested to merge. Thereon + * evaluate the next region R2 + * Otherwise, R1 is evaluated * * - * Region sizes are coarse and approximate on the order of megabytes. Additionally, - * "empty" regions (less than 1MB, with the previous note) are not merged away. This - * is by design to prevent normalization from undoing the pre-splitting of a table. + * Region sizes are coarse and approximate on the order of megabytes. Additionally, "empty" regions + * (less than 1MB, with the previous note) are not merged away. This is by design to prevent + * normalization from undoing the pre-splitting of a table. Review comment: I think i explained individual normalizer but didn't highlight how they are different. Let me add that. Thanks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] mnpoonia commented on a change in pull request #978: HBASE-22285 A normalizer which merges small size regions with adjacen…
mnpoonia commented on a change in pull request #978: HBASE-22285 A normalizer
which merges small size regions with adjacen…
URL: https://github.com/apache/hbase/pull/978#discussion_r363656264
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/AbstractRegionNormalizer.java
##
@@ -0,0 +1,213 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.master.normalizer;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.hadoop.hbase.RegionMetrics;
+import org.apache.hadoop.hbase.ServerName;
+import org.apache.hadoop.hbase.Size;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.MasterSwitchType;
+import org.apache.hadoop.hbase.client.RegionInfo;
+import org.apache.hadoop.hbase.client.TableDescriptor;
+import org.apache.hadoop.hbase.master.MasterRpcServices;
+import org.apache.hadoop.hbase.master.MasterServices;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hbase.thirdparty.com.google.protobuf.ServiceException;
+
+import org.apache.hadoop.hbase.shaded.protobuf.RequestConverter;
+
+@InterfaceAudience.Private
+public abstract class AbstractRegionNormalizer implements RegionNormalizer {
+ private static final Logger LOG =
LoggerFactory.getLogger(AbstractRegionNormalizer.class);
+ protected MasterServices masterServices;
+ protected MasterRpcServices masterRpcServices;
+
+ /**
+ * Set the master service.
+ * @param masterServices inject instance of MasterServices
+ */
+ @Override
+ public void setMasterServices(MasterServices masterServices) {
+this.masterServices = masterServices;
+ }
+
+ @Override
+ public void setMasterRpcServices(MasterRpcServices masterRpcServices) {
+this.masterRpcServices = masterRpcServices;
Review comment:
So we can surely move them to constructor. The reason i didn't do so is
because they were present in interface already. I am not sure why they were
kept as different methods in initial design.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Updated] (HBASE-23645) Fix remaining Checkstyle violations in tests of hbase-common
[
https://issues.apache.org/jira/browse/HBASE-23645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jan Hentschel updated HBASE-23645:
--
Fix Version/s: 2.1.9
2.2.3
2.3.0
3.0.0
Hadoop Flags: Reviewed
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Fix remaining Checkstyle violations in tests of hbase-common
>
>
> Key: HBASE-23645
> URL: https://issues.apache.org/jira/browse/HBASE-23645
> Project: HBase
> Issue Type: Improvement
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9
>
>
> In {{hbase-common}} Checkstyle reports a lot of violations. The remaining
> violations in the tests should be fixed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] bharathv commented on a change in pull request #995: HBASE-23055 Alter hbase:meta
bharathv commented on a change in pull request #995: HBASE-23055 Alter
hbase:meta
URL: https://github.com/apache/hbase/pull/995#discussion_r363621918
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKAsyncRegistry.java
##
@@ -229,6 +242,33 @@ private void
getMetaRegionLocation(CompletableFuture future,
});
}
+ @Override
+ public CompletableFuture getMetaTableState() {
+CompletableFuture result =
+ getAndConvert(this.znodeMirroredMetaTableState,
ZKAsyncRegistry::getTableState)
+ .thenApply(state -> {
+return state == null ||
state.equals(ENABLED_META_TABLE_STATE.getState())?
+ ENABLED_META_TABLE_STATE: new TableState(TableName.META_TABLE_NAME,
state);
+ });
+// If completed exceptionally return default ENABLED state.
+return result.isCompletedExceptionally()? COMPLETED_GET_META_TABLE_STATE:
result;
Review comment:
Putting it out here means its blocking and not async right? (or did I read
it incorrectly?)
I think this method could use some test coverage for a bunch of table states.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] bharathv commented on a change in pull request #995: HBASE-23055 Alter hbase:meta
bharathv commented on a change in pull request #995: HBASE-23055 Alter
hbase:meta
URL: https://github.com/apache/hbase/pull/995#discussion_r363618587
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
##
@@ -948,22 +950,13 @@ protected Void waitOperationResult(long deadlineTs)
throws IOException, TimeoutE
@Override
public boolean isTableEnabled(final TableName tableName) throws IOException {
checkTableExists(tableName);
-return executeCallable(new RpcRetryingCallable() {
- @Override
- protected Boolean rpcCall(int callTimeout) throws Exception {
-TableState tableState =
MetaTableAccessor.getTableState(getConnection(), tableName);
-if (tableState == null) {
- throw new TableNotFoundException(tableName);
-}
-return tableState.inStates(TableState.State.ENABLED);
- }
-});
+return this.connection.getTableState(tableName).isEnabled();
Review comment:
nit: is this change in behavior intentional? Retrying -> non-retrying
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] bharathv commented on a change in pull request #995: HBASE-23055 Alter hbase:meta
bharathv commented on a change in pull request #995: HBASE-23055 Alter
hbase:meta
URL: https://github.com/apache/hbase/pull/995#discussion_r363617699
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
##
@@ -1112,7 +1119,7 @@ public static RegionInfo getRegionInfo(final Result r,
byte [] qualifier) {
public static TableState getTableState(Connection conn, TableName tableName)
throws IOException {
if (tableName.equals(TableName.META_TABLE_NAME)) {
- return new TableState(tableName, TableState.State.ENABLED);
+ throw new IllegalAccessError("Not for hbase:meta table state");
Review comment:
nit: can be simplified to Preconditions.checkArgument()?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] bharathv commented on a change in pull request #995: HBASE-23055 Alter hbase:meta
bharathv commented on a change in pull request #995: HBASE-23055 Alter
hbase:meta
URL: https://github.com/apache/hbase/pull/995#discussion_r363614389
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
##
@@ -330,6 +342,26 @@ public void newDead(ServerName sn) {
close();
throw e;
}
+// Create tablestate cache. Add a loader that know how to find table state.
+int duration = this.conf.getInt(TABLESTATE_CACHE_DURATION_MS, 1000);
+this.tableStateCache = CacheBuilder.newBuilder().
+ expireAfterWrite(duration, TimeUnit.MILLISECONDS).
+ recordStats().
Review comment:
nit: The patch doesn't use the recorded states anywhere. Unnecessary
overhead, can be removed?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] bharathv commented on a change in pull request #995: HBASE-23055 Alter hbase:meta
bharathv commented on a change in pull request #995: HBASE-23055 Alter
hbase:meta
URL: https://github.com/apache/hbase/pull/995#discussion_r363610243
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/TestHBaseMetaEdit.java
##
@@ -0,0 +1,111 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase;
+
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.ColumnFamilyDescriptor;
+import org.apache.hadoop.hbase.client.ColumnFamilyDescriptorBuilder;
+import org.apache.hadoop.hbase.client.RegionInfoBuilder;
+import org.apache.hadoop.hbase.client.TableDescriptor;
+import org.apache.hadoop.hbase.io.encoding.DataBlockEncoding;
+import org.apache.hadoop.hbase.regionserver.Region;
+import org.apache.hadoop.hbase.testclassification.LargeTests;
+import org.apache.hadoop.hbase.testclassification.MiscTests;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.ClassRule;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.rules.TestName;
+
+import java.io.IOException;
+import java.util.concurrent.ExecutionException;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertTrue;
+
+/**
+ * Test being able to edit hbase:meta.
+ */
+@Category({MiscTests.class, LargeTests.class})
+public class TestHBaseMetaEdit {
+ @ClassRule
+ public static final HBaseClassTestRule CLASS_RULE =
+ HBaseClassTestRule.forClass(TestHBaseMetaEdit.class);
+ @Rule
+ public TestName name = new TestName();
+ private final static HBaseTestingUtility UTIL = new HBaseTestingUtility();
+
+ @Before
+ public void before() throws Exception {
+UTIL.startMiniCluster();
+ }
+
+ @After
+ public void after() throws Exception {
+UTIL.shutdownMiniCluster();
+ }
+
+ /**
+ * Set versions, set HBASE-16213 indexed block encoding, and add a column
family.
+ * Verify they are all in place by looking at TableDescriptor AND by checking
+ * what the RegionServer sees after opening Region.
+ */
+ @Test
+ public void testEditMeta() throws IOException, ExecutionException,
InterruptedException {
Review comment:
nit: ExecutionException, InterruptedException are not thrown anywhere?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] bharathv commented on a change in pull request #995: HBASE-23055 Alter hbase:meta
bharathv commented on a change in pull request #995: HBASE-23055 Alter
hbase:meta
URL: https://github.com/apache/hbase/pull/995#discussion_r363618881
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/MasterKeepAliveConnection.java
##
@@ -35,7 +38,8 @@
* final user code. Hence it's package protected.
*/
@InterfaceAudience.Private
-interface MasterKeepAliveConnection extends
MasterProtos.MasterService.BlockingInterface {
+interface MasterKeepAliveConnection extends
+MasterProtos.MasterService.BlockingInterface, Closeable {
// Do this instead of implement Closeable because closeable returning IOE is
PITA.
Review comment:
nit: update the comment? Thats contradicting.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] bharathv commented on a change in pull request #995: HBASE-23055 Alter hbase:meta
bharathv commented on a change in pull request #995: HBASE-23055 Alter
hbase:meta
URL: https://github.com/apache/hbase/pull/995#discussion_r363616695
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
##
@@ -154,6 +156,16 @@
public static final String RETRIES_BY_SERVER_KEY =
"hbase.client.retries.by.server";
private static final Logger LOG =
LoggerFactory.getLogger(ConnectionImplementation.class);
+ /**
+ * TableState cache.
+ */
+ private final LoadingCache tableStateCache;
Review comment:
Q: whats the motivation to use a cache here? To reduce the number of rpcs
for table state look ups?
nit: One behavior I noticed is that the general patten of client is to force
reload if something doesn't look as expected (retries..). For example
relocateRegion() doesn't care if any meta hrls are cached, it force reloads
them. Does that pattern also apply for table state cache? For example, in the
following snippet, isTableDisabled can now be served from cache. Not big of a
deal probably because the cache ttl is very low and table states probably
don't flip between disabled/enabled that often but wanted to point it out.
```
@Override
public RegionLocations relocateRegion(final TableName tableName,
final byte [] row, int replicaId) throws IOException{
if (isTableDisabled(tableName)) {
throw new TableNotEnabledException(tableName.getNameAsString() + " is
disabled.");
}
return locateRegion(tableName, row, false, true, replicaId);
}
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] bharathv commented on a change in pull request #995: HBASE-23055 Alter hbase:meta
bharathv commented on a change in pull request #995: HBASE-23055 Alter
hbase:meta
URL: https://github.com/apache/hbase/pull/995#discussion_r363613879
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRegistry.java
##
@@ -29,12 +30,26 @@
*/
@InterfaceAudience.Private
interface AsyncRegistry extends Closeable {
+ /**
+ * A completed CompletableFuture to host default hbase:meta table state
(ENABLED).
+ */
+ TableState ENABLED_META_TABLE_STATE =
+new TableState(TableName.META_TABLE_NAME, TableState.State.ENABLED);
+ CompletableFuture COMPLETED_GET_META_TABLE_STATE =
+CompletableFuture.completedFuture(ENABLED_META_TABLE_STATE);
/**
* Get the location of meta region.
*/
CompletableFuture getMetaRegionLocation();
+ /**
+ * The hbase:meta table state.
+ */
+ default CompletableFuture getMetaTableState() {
Review comment:
More of a question: whats the need for a default implementation? This is
almost always overriden by an implementation?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
