[GitHub] [hbase] Apache-HBase commented on pull request #2385: HBASE-24950 Another solution for Splittable Meta
Apache-HBase commented on pull request #2385: URL: https://github.com/apache/hbase/pull/2385#issuecomment-707529345 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2521: HBASE-25117 ReplicationSourceShipper thread can not be finished
Apache-HBase commented on pull request #2521: URL: https://github.com/apache/hbase/pull/2521#issuecomment-707522059 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 4m 16s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 5s | master passed | | +1 :green_heart: | compile | 0m 59s | master passed | | +1 :green_heart: | shadedjars | 7m 1s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 31s | the patch passed | | +1 :green_heart: | compile | 1m 2s | the patch passed | | +1 :green_heart: | javac | 1m 2s | the patch passed | | +1 :green_heart: | shadedjars | 6m 56s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 144m 16s | hbase-server in the patch passed. | | | | 175m 15s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2521/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2521 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 07f4fb20707c 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / bc20203965 | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2521/2/testReport/ | | Max. process+thread count | 3858 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2521/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2521: HBASE-25117 ReplicationSourceShipper thread can not be finished
Apache-HBase commented on pull request #2521: URL: https://github.com/apache/hbase/pull/2521#issuecomment-707521413 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 17s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 59s | master passed | | +1 :green_heart: | compile | 1m 6s | master passed | | +1 :green_heart: | shadedjars | 7m 28s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 45s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 56s | the patch passed | | +1 :green_heart: | compile | 1m 19s | the patch passed | | +1 :green_heart: | javac | 1m 19s | the patch passed | | +1 :green_heart: | shadedjars | 8m 10s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 47s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 141m 47s | hbase-server in the patch passed. | | | | 173m 44s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2521/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2521 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 065f3cd080ff 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / bc20203965 | | Default Java | 2020-01-14 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2521/2/testReport/ | | Max. process+thread count | 3851 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2521/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2385: HBASE-24950 Another solution for Splittable Meta
Apache-HBase commented on pull request #2385: URL: https://github.com/apache/hbase/pull/2385#issuecomment-707485163 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 24s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 2s | No case conflicting files found. | | +0 :ok: | prototool | 0m 2s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 26s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 50s | master passed | | +1 :green_heart: | checkstyle | 3m 5s | master passed | | +1 :green_heart: | spotbugs | 8m 23s | master passed | | -0 :warning: | patch | 2m 54s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 44s | the patch passed | | +1 :green_heart: | checkstyle | 0m 9s | The patch passed checkstyle in hbase-protocol-shaded | | +1 :green_heart: | checkstyle | 0m 24s | The patch passed checkstyle in hbase-common | | +1 :green_heart: | checkstyle | 0m 31s | hbase-client: The patch generated 0 new + 207 unchanged - 1 fixed = 207 total (was 208) | | +1 :green_heart: | checkstyle | 0m 12s | The patch passed checkstyle in hbase-zookeeper | | +1 :green_heart: | checkstyle | 0m 11s | The patch passed checkstyle in hbase-balancer | | -0 :warning: | checkstyle | 1m 23s | hbase-server: The patch generated 1 new + 673 unchanged - 8 fixed = 674 total (was 681) | | +1 :green_heart: | checkstyle | 0m 15s | The patch passed checkstyle in hbase-it | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 18m 49s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | hbaseprotoc | 3m 21s | the patch passed | | +1 :green_heart: | spotbugs | 9m 31s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 1m 10s | The patch does not generate ASF License warnings. | | | | 67m 38s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2385/12/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2385 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle cc hbaseprotoc prototool | | uname | Linux 61618c4c50d4 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / bc20203965 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2385/12/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 84 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-zookeeper hbase-balancer hbase-server hbase-it U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2385/12/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2521: HBASE-25117 ReplicationSourceShipper thread can not be finished
Apache-HBase commented on pull request #2521: URL: https://github.com/apache/hbase/pull/2521#issuecomment-707475000 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 59s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 46s | master passed | | +1 :green_heart: | checkstyle | 1m 5s | master passed | | +1 :green_heart: | spotbugs | 2m 0s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 28s | the patch passed | | +1 :green_heart: | checkstyle | 1m 3s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 17m 4s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 9s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 39m 5s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2521/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2521 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 62ce341936d3 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / bc20203965 | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2521/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on pull request #2445: HBASE-11288 Splittable Meta (WIP) Draft
saintstack commented on pull request #2445: URL: https://github.com/apache/hbase/pull/2445#issuecomment-707473657 First pass notes above. Point me at the overview on what this patch delivers. Looks like it doesn't include splitting meta support yet? It inserts root. The blocking in AM waiting on all of hbase:meta to be up before progress is coarse; the whole table must be up before progress can be made. To be addressed in a follow-on. Seems like it has gotten a bit stale. Needs rebase (conflicts above and some stuff has changed since). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #2445: HBASE-11288 Splittable Meta (WIP) Draft
saintstack commented on a change in pull request #2445:
URL: https://github.com/apache/hbase/pull/2445#discussion_r503642848
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionInfoBuilder.java
##
@@ -33,6 +33,25 @@
public static final RegionInfo UNDEFINED =
RegionInfoBuilder.newBuilder(TableName.valueOf("__UNDEFINED__")).build();
+ /**
+ * RegionInfo for first root region
+ * You cannot use this builder to make an instance of the {@link
#ROOT_REGIONINFO}.
+ * Just refer to this instance. Also, while the instance is actually a
MutableRI, its type is
+ * just RI so the mutable methods are not available (unless you go casting);
it appears
+ * as immutable (I tried adding Immutable type but it just makes a mess).
Review comment:
Good
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionInfoBuilder.java
##
@@ -33,6 +33,25 @@
public static final RegionInfo UNDEFINED =
RegionInfoBuilder.newBuilder(TableName.valueOf("__UNDEFINED__")).build();
+ /**
+ * RegionInfo for first root region
Review comment:
first? Only?
##
File path:
hbase-common/src/main/java/org/apache/hadoop/hbase/CellComparatorImpl.java
##
@@ -300,8 +300,12 @@ public static CellComparator getCellComparator(TableName
tableName) {
* @return CellComparator to use going off the {@code tableName} passed.
*/
public static CellComparator getCellComparator(byte [] tableName) {
-// FYI, TableName.toBytes does not create an array; just returns existing
array pointer.
-return Bytes.equals(tableName, TableName.META_TABLE_NAME.toBytes())?
- MetaCellComparator.META_COMPARATOR: CellComparatorImpl.COMPARATOR;
+if (Bytes.equals(tableName, TableName.ROOT_TABLE_NAME.toBytes())) {
+ return RootCellComparator.ROOT_COMPARATOR;
+}
+if (Bytes.equals(tableName, TableName.META_TABLE_NAME.toBytes())) {
+ return MetaCellComparator.META_COMPARATOR;
+}
+return CellComparatorImpl.COMPARATOR;
Review comment:
Can't this call the Interface version?
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java
##
@@ -2531,6 +2531,7 @@ public GetTableStateResponse
setTableStateInMeta(RpcController controller,
*
* @return previous states of the regions
*/
+ //TODO francis support root here
Review comment:
Whats this?
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
##
@@ -152,9 +159,19 @@
private static final int DEFAULT_RIT_STUCK_WARNING_THRESHOLD = 60 * 1000;
public static final String UNEXPECTED_STATE_REGION = "Unexpected state for ";
- private final ProcedureEvent metaAssignEvent = new ProcedureEvent<>("meta
assign");
+ private final ProcedureEvent rootAssignEvent = new ProcedureEvent<>("root
assign");
+ private final ProcedureEvent rootLoadEvent = new ProcedureEvent<>("root
load");
+
private final ProcedureEvent metaLoadEvent = new ProcedureEvent<>("meta
load");
+ private final ConcurrentSkipListMap>
metaAssignEventMap =
Review comment:
Whats this about?
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/TableDescriptorBuilder.java
##
@@ -250,6 +260,7 @@
DEFAULT_VALUES.put(PRIORITY, String.valueOf(DEFAULT_PRIORITY));
DEFAULT_VALUES.keySet().stream()
.map(s -> new
Bytes(Bytes.toBytes(s))).forEach(RESERVED_KEYWORDS::add);
+RESERVED_KEYWORDS.add(IS_ROOT_KEY);
RESERVED_KEYWORDS.add(IS_META_KEY);
Review comment:
ouch. who did this.
Ok. I suppose ROOT needs to follow form.
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionInfoBuilder.java
##
@@ -33,6 +33,25 @@
public static final RegionInfo UNDEFINED =
RegionInfoBuilder.newBuilder(TableName.valueOf("__UNDEFINED__")).build();
+ /**
+ * RegionInfo for first root region
+ * You cannot use this builder to make an instance of the {@link
#ROOT_REGIONINFO}.
+ * Just refer to this instance. Also, while the instance is actually a
MutableRI, its type is
+ * just RI so the mutable methods are not available (unless you go casting);
it appears
+ * as immutable (I tried adding Immutable type but it just makes a mess).
+ *
+ * We are using the non-legacy encoding format to reduce the boilerplace code
Review comment:
Whats this?
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/TableDescriptorBuilder.java
##
@@ -124,6 +124,16 @@
@InterfaceAudience.Private
public static final String FLUSH_POLICY = "FLUSH_POLICY";
private static final Bytes FLUSH_POLICY_KEY = new
Bytes(Bytes.toBytes(FLUSH_POLICY));
+
+ /**
+ * Used by rest interface to access this metadata attribute
+ * which denotes if it is a catalog table, either hbase:meta .
+ */
+
[jira] [Resolved] (HBASE-25099) Change meta replica count by altering meta table descriptor
[ https://issues.apache.org/jira/browse/HBASE-25099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-25099. --- Resolution: Fixed > Change meta replica count by altering meta table descriptor > --- > > Key: HBASE-25099 > URL: https://issues.apache.org/jira/browse/HBASE-25099 > Project: HBase > Issue Type: Umbrella > Components: meta, read replicas >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > As now we support altering meta table, it will be better to also deal with > changing meta replica number using altering meta, i.e, we could unify the > logic in MasterMetaBootstrap to ModifyTableProcedure, and another benefit is > that we do not need to restart master when changing the replica number for > meta. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25099) Change meta replica count by altering meta table descriptor
[ https://issues.apache.org/jira/browse/HBASE-25099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-25099: -- Fix Version/s: 2.4.0 3.0.0-alpha-1 Release Note: Now you can change the region replication config for meta table by altering meta table. The old "hbase.meta.replica.count" is deprecated and will be removed in 4.0.0. But if it is set, we will still honor it, which means, when master restart, if we find out that the value of 'hbase.meta.replica.count' is different with the region replication config of meta table, we will schedule an alter table operation to change the region replication config to the value you configured for 'hbase.meta.replica.count'. > Change meta replica count by altering meta table descriptor > --- > > Key: HBASE-25099 > URL: https://issues.apache.org/jira/browse/HBASE-25099 > Project: HBase > Issue Type: Umbrella > Components: meta, read replicas >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > As now we support altering meta table, it will be better to also deal with > changing meta replica number using altering meta, i.e, we could unify the > logic in MasterMetaBootstrap to ModifyTableProcedure, and another benefit is > that we do not need to restart master when changing the replica number for > meta. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] infraio commented on a change in pull request #2521: HBASE-25117 ReplicationSourceShipper thread can not be finished
infraio commented on a change in pull request #2521:
URL: https://github.com/apache/hbase/pull/2521#discussion_r503649422
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java
##
@@ -691,6 +691,9 @@ public void terminate(String reason, Exception cause,
boolean clearMetrics,
}
}
+if (this.replicationEndpoint != null) {
Review comment:
ReplicationSourceShipper and entryReader are thread. replicationEndpoint
is not a thread and is used by ReplicationSourceShipper thread. I am +1 for
this: stop firstly, then interrupt thread.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #2445: HBASE-11288 Splittable Meta (WIP) Draft
saintstack commented on a change in pull request #2445:
URL: https://github.com/apache/hbase/pull/2445#discussion_r503638276
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncNonRootRegionLocator.java
##
@@ -174,10 +184,14 @@ private boolean tryComplete(LocateRequest req,
CompletableFuture= req.row. Here we split it to
endKey == req.row ||
// (endKey > req.row && startKey < req.row). The two conditions are
equal since startKey <
// endKey.
+KeyValue.KVComparator comparator =
getComparator(loc.getRegion().getTable());
byte[] endKey = loc.getRegion().getEndKey();
-int c = Bytes.compareTo(endKey, req.row);
+int c = comparator.compareRows(endKey, 0, endKey.length,
+ req.row,0, req.row.length);
Review comment:
This works even if the Cell is ByteBuffer backed?
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncNonRootRegionLocator.java
##
@@ -68,9 +70,9 @@
* The asynchronous locator for regions other than meta.
*/
@InterfaceAudience.Private
-class AsyncNonMetaRegionLocator {
+class AsyncNonRootRegionLocator {
Review comment:
This locates user-space and hbase:meta Regions?
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java
##
@@ -2391,42 +2406,91 @@ public void run(Timeout timeout) throws Exception {
return failedFuture(new IllegalArgumentException("Passed region name
can't be null"));
}
try {
+ TableName parentTable;
CompletableFuture> future;
if (RegionInfo.isEncodedRegionName(regionNameOrEncodedRegionName)) {
String encodedName = Bytes.toString(regionNameOrEncodedRegionName);
-if (encodedName.length() < RegionInfo.MD5_HEX_LENGTH) {
- // old format encodedName, should be meta region
+
+//TODO francis do we really need to support encoded name for root?
Review comment:
Dunno. I think the region id used to be 0. Could it be the encoded name
too?
The encoded name is usually md5 hash but old style Regions like hbase:meta
used the original simple hash encoded number.
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/MutableRegionInfo.java
##
@@ -206,16 +209,20 @@ public TableName getTable() {
*/
@Override
public boolean containsRange(byte[] rangeStartKey, byte[] rangeEndKey) {
-if (Bytes.compareTo(rangeStartKey, rangeEndKey) > 0) {
+CellComparator comparator = CellComparator.getComparator(tableName);
Review comment:
Yeah, this getComparator is a dupe method? Can we have just one?
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java
##
@@ -2391,42 +2406,91 @@ public void run(Timeout timeout) throws Exception {
return failedFuture(new IllegalArgumentException("Passed region name
can't be null"));
}
try {
+ TableName parentTable;
CompletableFuture> future;
if (RegionInfo.isEncodedRegionName(regionNameOrEncodedRegionName)) {
String encodedName = Bytes.toString(regionNameOrEncodedRegionName);
-if (encodedName.length() < RegionInfo.MD5_HEX_LENGTH) {
- // old format encodedName, should be meta region
+
+//TODO francis do we really need to support encoded name for root?
+boolean isRoot = false;
Review comment:
isRoot is name of the method that checks a boolean, not the name of
variable.
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncNonRootRegionLocator.java
##
@@ -667,4 +689,14 @@ int getNumberOfCachedRegionLocations(TableName tableName) {
}
return
tableCache.cache.values().stream().mapToInt(RegionLocations::numNonNullElements).sum();
}
+
+ private static KeyValue.KVComparator getComparator(TableName tableName) {
Review comment:
Is this the right place for this method?
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/MutableRegionInfo.java
##
@@ -126,7 +127,9 @@ private static int checkReplicaId(int regionId) {
this.replicaId = checkReplicaId(replicaId);
this.offLine = offLine;
this.regionName = RegionInfo.createRegionName(this.tableName,
this.startKey, this.regionId,
- this.replicaId, !this.tableName.equals(TableName.META_TABLE_NAME));
+ this.replicaId,
+ //1 is region id of FIRST_META_REGION_INFO
+ !(this.tableName.equals(TableName.META_TABLE_NAME) && regionId == 1));
Review comment:
What is happening here? '1' is the id of the hardcoded single region
hbase:meta table but is it going to be the region id when we split meta?
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java
##
@@ -1130,22 +1143,24 @@ public void run(PRESP resp) {
* List all region locations for the spec
[GitHub] [hbase] ddupg commented on a change in pull request #2521: HBASE-25117 ReplicationSourceShipper thread can not be finished
ddupg commented on a change in pull request #2521:
URL: https://github.com/apache/hbase/pull/2521#discussion_r503638903
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HBaseInterClusterReplicationEndpoint.java
##
@@ -475,7 +476,7 @@ public boolean replicate(ReplicateContext replicateContext)
{
}
List> batches = createBatches(replicateContext.getEntries());
-while (this.isRunning() && !exec.isShutdown()) {
+while (this.isRunning() && !exec.isShutdown() &&
!Thread.currentThread().isInterrupted()) {
Review comment:
How about using `ReplicationSource.isSourceActive()` instead of
`!Thread.currentThread().isInterrupted()` ? HBaseReplicationEndpoint manages
lifecycle by extent `AbstractService`, adding a flag in
HBaseInterClusterReplicationEndpoint is a bit redundant?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1531: HBASE-24200 Upgrade to Yetus 0.12.0
Apache-HBase commented on pull request #1531: URL: https://github.com/apache/hbase/pull/1531#issuecomment-707452190 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 6m 58s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 47s | master passed | | +1 :green_heart: | compile | 2m 21s | master passed | | +1 :green_heart: | shadedjars | 6m 42s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 16s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 26s | the patch passed | | +1 :green_heart: | compile | 2m 22s | the patch passed | | +1 :green_heart: | javac | 2m 22s | the patch passed | | +1 :green_heart: | shadedjars | 6m 28s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 8s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 338m 10s | root in the patch failed. | | | | 377m 13s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-1531/6/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1531 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux e6064b44b857 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c28220522d | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-1531/6/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-root.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-1531/6/testReport/ | | Max. process+thread count | 5149 (vs. ulimit of 3) | | modules | C: . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-1531/6/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2490: HBASE-24419 Normalizer merge plans should consider more than 2 regions when possible
Apache-HBase commented on pull request #2490: URL: https://github.com/apache/hbase/pull/2490#issuecomment-707446320 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 41s | master passed | | +1 :green_heart: | compile | 1m 23s | master passed | | +1 :green_heart: | shadedjars | 6m 43s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 59s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 33s | the patch passed | | +1 :green_heart: | compile | 1m 24s | the patch passed | | +1 :green_heart: | javac | 1m 24s | the patch passed | | +1 :green_heart: | shadedjars | 6m 32s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 59s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 22s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 140m 11s | hbase-server in the patch passed. | | | | 170m 24s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2490 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux f401c3625c38 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c28220522d | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/4/testReport/ | | Max. process+thread count | 3998 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/4/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25093) the RSGroupBasedLoadBalancer#retainAssignment throws NPE
[ https://issues.apache.org/jira/browse/HBASE-25093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212791#comment-17212791 ] niuyulin commented on HBASE-25093: -- [~zhangduo] of course, I have submitted another PR for branch-2 > the RSGroupBasedLoadBalancer#retainAssignment throws NPE > > > Key: HBASE-25093 > URL: https://issues.apache.org/jira/browse/HBASE-25093 > Project: HBase > Issue Type: Bug > Components: rsgroup >Affects Versions: 3.0.0-alpha-1, 2.3.1, 2.2.3 >Reporter: Bo Cui >Assignee: niuyulin >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.3, 2.4.0, 2.2.7 > > > when BaseLoadBalancer# > https://github.com/apache/hbase/blob/8bfa2cb2eedcf050b26a28961e1b77dbf3cd8c95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java#L1433 > If the result of the BaseLoadBalancer#retainAssignment is null, the > RSGroupBasedLoadBalancer#retainAssignment will throw NPE. > https://github.com/apache/hbase/blob/8bfa2cb2eedcf050b26a28961e1b77dbf3cd8c95/hbase-server/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupBasedLoadBalancer.java#L206 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25178) Remove the hadoop 3.3.0 personality hadoopcheck for branch-2.2/branch-2.3
[ https://issues.apache.org/jira/browse/HBASE-25178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-25178. Resolution: Duplicate Already fixed by HBASE-25144. > Remove the hadoop 3.3.0 personality hadoopcheck for branch-2.2/branch-2.3 > - > > Key: HBASE-25178 > URL: https://issues.apache.org/jira/browse/HBASE-25178 > Project: HBase > Issue Type: Bug >Affects Versions: 2.2.6 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > > For branch-2.2, see > [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/88/execution/node/163/log/] > It will fail when run "mvn clean install -DskipTests -DHBasePatchProcess > -Dhadoop-three.version=3.3.0 -Dhadoop.profile=3.0". > > For branch-2.3, see HBASE-23834. HBase failed to start on hadoop 3.3.0 > because the jetty problem. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2490: HBASE-24419 Normalizer merge plans should consider more than 2 regions when possible
Apache-HBase commented on pull request #2490: URL: https://github.com/apache/hbase/pull/2490#issuecomment-707444790 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 31s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 27s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 2s | master passed | | +1 :green_heart: | compile | 1m 31s | master passed | | +1 :green_heart: | shadedjars | 6m 30s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 3s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 58s | the patch passed | | +1 :green_heart: | compile | 1m 35s | the patch passed | | +1 :green_heart: | javac | 1m 35s | the patch passed | | +1 :green_heart: | shadedjars | 6m 38s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 4s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 32s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 132m 8s | hbase-server in the patch passed. | | | | 164m 44s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2490 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 7ee3b777a7ee 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c28220522d | | Default Java | 2020-01-14 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/4/testReport/ | | Max. process+thread count | 4603 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/4/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25156) TestMasterFailover.testSimpleMasterFailover is flaky
[
https://issues.apache.org/jira/browse/HBASE-25156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212788#comment-17212788
]
Duo Zhang commented on HBASE-25156:
---
It does not help, the test is still failing...
Error Message
Waiting timed out after [300,000] msec
Stacktrace
java.lang.AssertionError: Waiting timed out after [300,000] msec
at
org.apache.hadoop.hbase.master.TestMasterFailover.testSimpleMasterFailover(TestMasterFailover.java:131)
> TestMasterFailover.testSimpleMasterFailover is flaky
> -
>
> Key: HBASE-25156
> URL: https://issues.apache.org/jira/browse/HBASE-25156
> Project: HBase
> Issue Type: Test
> Components: test
>Affects Versions: 3.0.0-alpha-1
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0, 2.2.7
>
>
> {noformat}
> [ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 32.996 s <<< FAILURE! - in org.apache.hadoop.hbase.master.TestMasterFailover
> [ERROR]
> org.apache.hadoop.hbase.master.TestMasterFailover.testSimpleMasterFailover
> Time elapsed: 12.317 s <<< FAILURE!
> java.lang.AssertionError: expected:<1> but was:<2>
> at
> org.apache.hadoop.hbase.master.TestMasterFailover.testSimpleMasterFailover(TestMasterFailover.java:133)
> {noformat}
> Looks like this test depends on metrics to be updated as a source of
> side-effect used to verify the test. Seems like it should retry the check a
> few times, or maybe we need a last-updated monotonic value that the test can
> check before and after it expects a change to be visible.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] infraio closed pull request #2533: HBASE-25178 Remove the hadoop 3.3.0 personality hadoopcheck for branc…
infraio closed pull request #2533: URL: https://github.com/apache/hbase/pull/2533 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] infraio merged pull request #2536: "HBASE-25144 Add Hadoop-3.3.0 to personality hadoopcheck" try again
infraio merged pull request #2536: URL: https://github.com/apache/hbase/pull/2536 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-25169) Update documentation about meta region replica
[ https://issues.apache.org/jira/browse/HBASE-25169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-25169. --- Hadoop Flags: Reviewed Resolution: Fixed Pushed to master and branch-2. Thanks [~stack] for reviewing. > Update documentation about meta region replica > -- > > Key: HBASE-25169 > URL: https://issues.apache.org/jira/browse/HBASE-25169 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25169) Update documentation about meta region replica
[ https://issues.apache.org/jira/browse/HBASE-25169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-25169: -- Fix Version/s: 2.4.0 3.0.0-alpha-1 > Update documentation about meta region replica > -- > > Key: HBASE-25169 > URL: https://issues.apache.org/jira/browse/HBASE-25169 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25164) Make ModifyTableProcedure support changing meta replica count
[ https://issues.apache.org/jira/browse/HBASE-25164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-25164. --- Hadoop Flags: Reviewed Resolution: Fixed Pushed to master and branch-2. Thanks [~stack] for reviewing. > Make ModifyTableProcedure support changing meta replica count > - > > Key: HBASE-25164 > URL: https://issues.apache.org/jira/browse/HBASE-25164 > Project: HBase > Issue Type: Sub-task > Components: meta, read replicas >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > And also make it compatible with the old way where we use a Configuration to > set the replica number. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25164) Make ModifyTableProcedure support changing meta replica count
[ https://issues.apache.org/jira/browse/HBASE-25164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-25164: -- Fix Version/s: 2.4.0 3.0.0-alpha-1 > Make ModifyTableProcedure support changing meta replica count > - > > Key: HBASE-25164 > URL: https://issues.apache.org/jira/browse/HBASE-25164 > Project: HBase > Issue Type: Sub-task > Components: meta, read replicas >Reporter: Duo Zhang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > And also make it compatible with the old way where we use a Configuration to > set the replica number. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-25164) Make ModifyTableProcedure support changing meta replica count
[ https://issues.apache.org/jira/browse/HBASE-25164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reassigned HBASE-25164: - Assignee: Duo Zhang > Make ModifyTableProcedure support changing meta replica count > - > > Key: HBASE-25164 > URL: https://issues.apache.org/jira/browse/HBASE-25164 > Project: HBase > Issue Type: Sub-task > Components: meta, read replicas >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > And also make it compatible with the old way where we use a Configuration to > set the replica number. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache9 commented on pull request #2536: "HBASE-25144 Add Hadoop-3.3.0 to personality hadoopcheck" try again
Apache9 commented on pull request #2536: URL: https://github.com/apache/hbase/pull/2536#issuecomment-707433866 And we should also add 3.1.4 and 3.2.2 to hadoop check, but since the title is add 3.3.0, can do this in another issue. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 merged pull request #2528: HBASE-25169 Update documentation about meta region replica
Apache9 merged pull request #2528: URL: https://github.com/apache/hbase/pull/2528 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 merged pull request #2513: HBASE-25164 Make ModifyTableProcedure support changing meta replica c…
Apache9 merged pull request #2513: URL: https://github.com/apache/hbase/pull/2513 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25168) Unify WAL name timestamp parsers
[
https://issues.apache.org/jira/browse/HBASE-25168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212760#comment-17212760
]
Hudson commented on HBASE-25168:
Results for branch branch-2.2
[build #92 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/92/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/92//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/92//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/92//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Unify WAL name timestamp parsers
>
>
> Key: HBASE-25168
> URL: https://issues.apache.org/jira/browse/HBASE-25168
> Project: HBase
> Issue Type: Bug
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 2.4.0, 2.2.7
>
>
> Turns out there are two methods for extracting timestamp from WAL filename.
> Fix.
> Spotted by [~zhangduo] in review of HBASE-22976...
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #1531: HBASE-24200 Upgrade to Yetus 0.12.0
Apache-HBase commented on pull request #1531: URL: https://github.com/apache/hbase/pull/1531#issuecomment-707425156 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 12s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 49s | master passed | | +1 :green_heart: | compile | 3m 11s | master passed | | +1 :green_heart: | shadedjars | 7m 19s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 3m 7s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 36s | the patch passed | | +1 :green_heart: | compile | 3m 5s | the patch passed | | +1 :green_heart: | javac | 3m 5s | the patch passed | | +1 :green_heart: | shadedjars | 7m 19s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 3m 3s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 237m 0s | root in the patch passed. | | | | 276m 50s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-1531/6/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1531 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux fda7229eed06 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c28220522d | | Default Java | AdoptOpenJDK-11.0.6+10 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-1531/6/testReport/ | | Max. process+thread count | 4802 (vs. ulimit of 3) | | modules | C: . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-1531/6/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] brfrn169 commented on pull request #2498: HBASE-24996 Support CheckAndMutate in Region.batchMutate()
brfrn169 commented on pull request #2498: URL: https://github.com/apache/hbase/pull/2498#issuecomment-707424113 @saintstack @Apache9 @joshelser Could you please review this when you have time? I wrote down the summary of this change in the following comment: https://github.com/apache/hbase/pull/2498#issue-497366487 Thank you in advance. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] brfrn169 commented on a change in pull request #2498: HBASE-24996 Support CheckAndMutate in Region.batchMutate()
brfrn169 commented on a change in pull request #2498: URL: https://github.com/apache/hbase/pull/2498#discussion_r503612295 ## File path: hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java ## @@ -3069,27 +3073,33 @@ private RegionScannerImpl getScanner(Scan scan, List additional checkFamily(family); } } - return instantiateRegionScanner(scan, additionalScanners, nonceGroup, nonce); + return instantiateRegionScanner(scan, additionalScanners, additionalScannersForStores, Review comment: Let me explain this. There was one requirement where the additional cells of a mutation need to be visible for followed mutations. For example, when we have a Put operation and a CheckAndMutate operation in a single mutation batch, the additional cells of the Put operation should be visible for the followed CheckAndMutate operation. The following code keeps additional cells of mutations: https://github.com/brfrn169/hbase/blob/HBASE-24996/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java#L3833-L3835 To achieve this, I first tried to use the existing additionalScanners of RegionScannerImpl: https://github.com/brfrn169/hbase/blob/HBASE-24996/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java#L3061 However, I found it's much easier to add a new additionalScanners to StoreScanner and use it than to use the existing additionalScanners of RegionScannerImpl. So, I decided to add additionalScanners to StoreScanner and was able to make the additional cell visible for the followed mutations. https://github.com/brfrn169/hbase/blob/HBASE-24996/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java#L251-L253 By the way, the test code of this is as as follows: https://github.com/brfrn169/hbase/blob/HBASE-24996/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java#L7101-L7174 Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] brfrn169 commented on a change in pull request #2498: HBASE-24996 Support CheckAndMutate in Region.batchMutate()
brfrn169 commented on a change in pull request #2498:
URL: https://github.com/apache/hbase/pull/2498#discussion_r503611555
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -4074,29 +4183,128 @@ private static long getLongValue(final Cell cell)
throws DoNotRetryIOException {
return PrivateCellUtil.getValueAsLong(cell);
}
-/**
- * Do a specific Get on passed columnFamily and column
qualifiers.
- * @param mutation Mutation we are doing this Get for.
- * @param store Which column family on row (TODO: Go all Gets in one go)
- * @param coordinates Cells from mutation used as coordinates
applied to Get.
- * @return Return list of Cells found.
- */
-private List get(Mutation mutation, HStore store, List
coordinates,
- TimeRange tr) throws IOException {
- // Sort the cells so that they match the order that they appear in the
Get results.
- // Otherwise, we won't be able to find the existing values if the cells
are not specified
- // in order by the client since cells are in an array list.
- // TODO: I don't get why we are sorting. St.Ack 20150107
Review comment:
Let me explain this.
There was one requirement where the additional cells of a mutation need to
be visible for followed mutations. For example, when we have a Put operation
and a CheckAndMutate operation in a single mutation batch, the additional cells
of the Put operation should be visible for the followed CheckAndMutate
operation.
The following code keeps additional cells of mutations:
https://github.com/brfrn169/hbase/blob/HBASE-24996/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java#L3833-L3835
To achieve this, I first tried to use the existing `additionalScanners` of
`RegionScannerImpl`:
https://github.com/brfrn169/hbase/blob/HBASE-24996/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java#L3061
However, I found it's much easier to add a new `additionalScanners` to
`StoreScanner` and use it than to use the existing `additionalScanners` of
`RegionScannerImpl`.
So, I decided to add `additionalScanners` to `StoreScanner` and was able to
make the additional cell visible for the followed mutations.
https://github.com/brfrn169/hbase/blob/HBASE-24996/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java#L251-L253
By the way, the test code of this is as as follows:
https://github.com/brfrn169/hbase/blob/HBASE-24996/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java#L7101-L7174
Thanks.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2490: HBASE-24419 Normalizer merge plans should consider more than 2 regions when possible
Apache-HBase commented on pull request #2490: URL: https://github.com/apache/hbase/pull/2490#issuecomment-707413725 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 28s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 30s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 21s | master passed | | +1 :green_heart: | checkstyle | 1m 39s | master passed | | +1 :green_heart: | spotbugs | 2m 54s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 7s | the patch passed | | +1 :green_heart: | checkstyle | 1m 40s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 19m 13s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 3m 21s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 22s | The patch does not generate ASF License warnings. | | | | 47m 59s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/4/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2490 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 86d65c00cba9 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c28220522d | | Max. process+thread count | 84 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/4/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25144) Add Hadoop-3.3.0 to personality hadoopcheck
[ https://issues.apache.org/jira/browse/HBASE-25144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212734#comment-17212734 ] Michael Stack commented on HBASE-25144: --- HBASE-23834 made it so hadoop 3.3.0 works with hbase-2.4.0 and master/hbase-3 via hbase-thirdparty additions. > Add Hadoop-3.3.0 to personality hadoopcheck > --- > > Key: HBASE-25144 > URL: https://issues.apache.org/jira/browse/HBASE-25144 > Project: HBase > Issue Type: Task > Components: build, community >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 3.0.0-alpha-1 > > > Now that Hadoop 3.3.0 is released, let's figure out where it goes in our > testing matrix. Start by adding it to precommit checks. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2514: HBASE-25166 MobFileCompactionChore is closing the master's shared clu…
Apache-HBase commented on pull request #2514: URL: https://github.com/apache/hbase/pull/2514#issuecomment-707397758 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 26s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 45s | master passed | | +1 :green_heart: | compile | 0m 56s | master passed | | +1 :green_heart: | shadedjars | 6m 28s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 24s | the patch passed | | +1 :green_heart: | compile | 0m 54s | the patch passed | | +1 :green_heart: | javac | 0m 54s | the patch passed | | +1 :green_heart: | shadedjars | 6m 36s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 35s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 140m 4s | hbase-server in the patch passed. | | | | 165m 44s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2514/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2514 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 10ab5e6d503c 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c28220522d | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2514/3/testReport/ | | Max. process+thread count | 4826 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2514/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2514: HBASE-25166 MobFileCompactionChore is closing the master's shared clu…
Apache-HBase commented on pull request #2514: URL: https://github.com/apache/hbase/pull/2514#issuecomment-707395821 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 16s | master passed | | +1 :green_heart: | compile | 1m 4s | master passed | | +1 :green_heart: | shadedjars | 6m 37s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 5s | the patch passed | | +1 :green_heart: | compile | 1m 5s | the patch passed | | +1 :green_heart: | javac | 1m 5s | the patch passed | | +1 :green_heart: | shadedjars | 6m 37s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 131m 17s | hbase-server in the patch passed. | | | | 158m 47s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2514/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2514 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 57746a5dd30f 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c28220522d | | Default Java | 2020-01-14 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2514/3/testReport/ | | Max. process+thread count | 4023 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2514/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-24464) Normalizer should have configuration for minimum region size
[
https://issues.apache.org/jira/browse/HBASE-24464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk resolved HBASE-24464.
--
Resolution: Won't Fix
I don't think there's a strong case for this functionality. Reopen if you feel
otherwise.
> Normalizer should have configuration for minimum region size
>
>
> Key: HBASE-24464
> URL: https://issues.apache.org/jira/browse/HBASE-24464
> Project: HBase
> Issue Type: Improvement
> Components: master, Normalizer
>Reporter: Nick Dimiduk
>Priority: Major
>
> Another issue raised in PR review on HBASE-24418. With the current
> implementation, the lower-bound on how small the normalizer will make a
> region is its lowest granularity of size measurement: 1mb. That means it will
> happily balance a table out to be 1000 * 1mb regions (probably not likely,
> but seems plausible). The proposal was to add configuration knob that
> specifies the lower bound on region size as a guard against excessive
> splitting, something of an analogue to {{hbase.hregion.max.filesize}}.
> I'm not convinced this is really needed; I think the table would need to have
> been crafted intentionally to trigger this behavior. However, it seems like a
> reasonable guard rail to install. Discuss.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] ndimiduk commented on pull request #2523: HBASE-25167 Normalizer support for hot config reloading
ndimiduk commented on pull request #2523: URL: https://github.com/apache/hbase/pull/2523#issuecomment-707394637 I think the failure in `TestAsyncTable#testCheckAndIncrement` is unrelated. Anyway it would be nicer if that test would print the exception with the underlying failure case instead of just `assertTrue(result.isSuccess()`. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk commented on pull request #2490: HBASE-24419 Normalizer merge plans should consider more than 2 regions when possible
ndimiduk commented on pull request #2490: URL: https://github.com/apache/hbase/pull/2490#issuecomment-707393565 Rebase and address checkstyle nits. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Work stopped] (HBASE-24673) TransitionRegionStateProcedure of non-meta regions should yield when meta is unavailable
[ https://issues.apache.org/jira/browse/HBASE-24673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-24673 stopped by Nick Dimiduk. > TransitionRegionStateProcedure of non-meta regions should yield when meta is > unavailable > > > Key: HBASE-24673 > URL: https://issues.apache.org/jira/browse/HBASE-24673 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > > One observation from HBASE-24526 is that while meta is unavailable, other > region movement procedures are getting stuck on meta RPCs. Let's make it so > that non-meta transitions check the state of meta before attempting any RPCs. > If meta is known unavailable, release the thread back to the scheduler. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2514: HBASE-25166 MobFileCompactionChore is closing the master's shared clu…
Apache-HBase commented on pull request #2514: URL: https://github.com/apache/hbase/pull/2514#issuecomment-707356713 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 12s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 46s | master passed | | +1 :green_heart: | checkstyle | 1m 3s | master passed | | +1 :green_heart: | spotbugs | 2m 19s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 31s | the patch passed | | -0 :warning: | checkstyle | 1m 2s | hbase-server: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 18m 50s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 34s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 12s | The patch does not generate ASF License warnings. | | | | 42m 33s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2514/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2514 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 52147aac57f3 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c28220522d | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2514/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2514/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1531: HBASE-24200 Upgrade to Yetus 0.12.0
Apache-HBase commented on pull request #1531: URL: https://github.com/apache/hbase/pull/1531#issuecomment-707348986 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 15s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | ||| _ Patch Compile Tests _ | | +1 :green_heart: | hadolint | 0m 2s | There were no new hadolint issues. | | +1 :green_heart: | shellcheck | 0m 1s | There were no new shellcheck issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +1 :green_heart: | hadoopcheck | 24m 2s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 19s | The patch does not generate ASF License warnings. | | | | 36m 18s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-1531/6/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1531 | | Optional Tests | dupname asflicense shellcheck shelldocs hadolint hadoopcheck xml | | uname | Linux 0f140acb9978 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c28220522d | | Max. process+thread count | 63 (vs. ulimit of 3) | | modules | C: . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-1531/6/console | | versions | git=2.17.1 maven=3.6.3 shellcheck=0.4.6 hadolint=1.17.5-0-g443423c | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk commented on pull request #1531: HBASE-24200 Upgrade to Yetus 0.12.0
ndimiduk commented on pull request #1531: URL: https://github.com/apache/hbase/pull/1531#issuecomment-707334131 Rebased. Grepped the source tree for `0\.11` and found a couple more occurrences. We should delete `jenkins_precommit_jira_yetus.sh` since we don't do builds from Jira tickets anymore. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk commented on a change in pull request #1531: HBASE-24200 Upgrade to Yetus 0.12.0
ndimiduk commented on a change in pull request #1531: URL: https://github.com/apache/hbase/pull/1531#discussion_r503518341 ## File path: pom.xml ## @@ -3982,4 +3982,5 @@ file:///tmp + Review comment: Yes, this line would not be merged. It's here to provoke a test run; without it, Yetus won't run over any maven modules. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25065) WAL archival to be done by a separate thread
[ https://issues.apache.org/jira/browse/HBASE-25065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212658#comment-17212658 ] Nick Dimiduk commented on HBASE-25065: -- [~ram_krish] Reviewed the PRs. I'm leaning towards "no" for a patch release, but I'd like to hear why you think otherwise. > WAL archival to be done by a separate thread > > > Key: HBASE-25065 > URL: https://issues.apache.org/jira/browse/HBASE-25065 > Project: HBase > Issue Type: Improvement > Components: wal >Affects Versions: 3.0.0-alpha-1, 2.4.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > Currently we do clean up of logs once we ensure that the region data has been > flushed. We track the sequence number and if we ensure that the seq number > has been flushed for any given region and the WAL that was rolled has that > seq number then those WAL can be archived. > When we have around ~50 files to archive (per RS) - we do the archiving one > after the other. Since archiving is nothing but a rename operation it adds to > the meta operation load of Cloud based FS. > Not only that - the entire archival is done inside the rollWriterLock. Though > we have closed the writer and created a new writer and the writes are ongoing > - we never release the lock until we are done with the archiving. > What happens is that during that period our logs grow in size compared to the > default size configured (when we have consistent writes happening). > So the proposal is to move the log archival to a seperate thread and ensure > we can do some kind of throttling or batching so that we don't do archival at > one shot. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] ndimiduk commented on a change in pull request #2501: HBASE-25065 WAL archival to be done by a separate thread
ndimiduk commented on a change in pull request #2501:
URL: https://github.com/apache/hbase/pull/2501#discussion_r503517527
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestFailedAppendAndSync.java
##
@@ -256,6 +273,27 @@ public void testLockupAroundBadAssignSync() throws
IOException {
Threads.sleep(1);
}
}
+
+ try {
+dodgyWAL.throwAppendException = false;
+dodgyWAL.throwSyncException = false;
+dodgyWAL.throwArchiveException = true;
+Pair pair = new Pair();
+pair.setFirst(new Path("/a/b/"));
+pair.setSecond(100L);
+dodgyWAL.archive(pair);
+ } catch (Throwable ioe) {
+ }
+ while (true) {
+try {
+ // one more abort needs to be called
+ Mockito.verify(services,
Mockito.atLeast(2)).abort(Mockito.anyString(),
+(Throwable) Mockito.anyObject());
+ break;
+} catch (WantedButNotInvoked t) {
+ Threads.sleep(1);
Review comment:
This will be a flaky test. Please use the `waitFor` pattern already
provided on the testing utility class.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] ndimiduk commented on a change in pull request #2501: HBASE-25065 WAL archival to be done by a separate thread
ndimiduk commented on a change in pull request #2501:
URL: https://github.com/apache/hbase/pull/2501#discussion_r503513496
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AbstractFSWAL.java
##
@@ -84,8 +87,12 @@
import org.apache.yetus.audience.InterfaceAudience;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-
import
org.apache.hbase.thirdparty.com.google.common.annotations.VisibleForTesting;
+import
org.apache.hbase.thirdparty.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+
Review comment:
nit: why all the extra whitespace?
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AbstractFSWAL.java
##
@@ -482,6 +505,8 @@ protected SyncFuture initialValue() {
this.walTooOldNs = TimeUnit.SECONDS.toNanos(conf.getInt(
SURVIVED_TOO_LONG_SEC_KEY, SURVIVED_TOO_LONG_SEC_DEFAULT));
this.useHsync = conf.getBoolean(HRegion.WAL_HSYNC_CONF_KEY,
HRegion.DEFAULT_WAL_HSYNC);
+archiveRetries =
this.conf.getInt("hbase.regionserver.walroll.archive.retries", 0);
Review comment:
Introducing a new configuration is a hint that this is too "big/complex"
to add on a patch release.
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AbstractFSWAL.java
##
@@ -715,11 +740,39 @@ private void cleanOldLogs() throws IOException {
regionsBlockingThisWal.clear();
}
}
+
if (logsToArchive != null) {
- for (Pair logAndSize : logsToArchive) {
-this.totalLogSize.addAndGet(-logAndSize.getSecond());
-archiveLogFile(logAndSize.getFirst());
-this.walFile2Props.remove(logAndSize.getFirst());
+ final List> localLogsToArchive = logsToArchive;
+ // make it async
+ for (Pair log : localLogsToArchive) {
+logArchiveExecutor.execute(() -> {
+ archive(log);
+});
+this.walFile2Props.remove(log.getFirst());
+ }
+}
+ }
+
+ protected void archive(final Pair log) {
+int retry = 1;
+while (true) {
Review comment:
No backoff of any kind in the retry mechanism?
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/AbstractTestLogRolling.java
##
@@ -175,10 +175,15 @@ public void testLogRollOnNothingWritten() throws
Exception {
}
}
- private void assertLogFileSize(WAL log) {
+ private void assertLogFileSize(WAL log) throws InterruptedException {
if (AbstractFSWALProvider.getNumRolledLogFiles(log) > 0) {
assertTrue(AbstractFSWALProvider.getLogFileSize(log) > 0);
} else {
+ for (int i = 0; i < 10; i++) {
Review comment:
This will be a flaky test. Please use the `waitFor` pattern already
provided on the testing utility class.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Comment Edited] (HBASE-22938) Fold all the system tables to hbase:meta
[
https://issues.apache.org/jira/browse/HBASE-22938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212603#comment-17212603
]
Michael Stack edited comment on HBASE-22938 at 10/12/20, 7:15 PM:
--
Just to repeat note added on HBASE-15867, some of the obstacles noted above
have been undone; i.e. region replicas are working to make it so they do not
need to keep state (see '4.1 Skip maintaining zookeeper replication queue
(offsets/WALs)' in
https://docs.google.com/document/d/1jJWVc-idHhhgL4KDRpjMsQJKCl_NRaCLGiH3Wqwd3O8/edit#heading=h.5hn1d8pikvrr).
Folding all system tables into hbase:meta should run a bit smoother (with
asserts as suggested above that we do not replicate system/catalog tables).
was (Author: stack):
{quote}This will make the read replicas feature can not 100% work for meta and
system tables, as we can not use in-cluster replication for them to spread the
edits to the secondary replicas any more.
{quote}
IIUC, this was once the case but no longer given read replicas no longer keep
ongoing state (see '4.1 Skip maintaining zookeeper replication queue
(offsets/WALs)' in
https://docs.google.com/document/d/1jJWVc-idHhhgL4KDRpjMsQJKCl_NRaCLGiH3Wqwd3O8/edit#heading=h.5hn1d8pikvrr)
> Fold all the system tables to hbase:meta
>
>
> Key: HBASE-22938
> URL: https://issues.apache.org/jira/browse/HBASE-22938
> Project: HBase
> Issue Type: Brainstorming
>Reporter: Duo Zhang
>Priority: Major
>
> Quote my post on HBASE-15867 here, on how to deal with the dead lock when we
> want to store replication queues to hbase:replication table.
> {quote}
> We could add a special prefix in the row key for different system tables, and
> make a special family for it. For example, for all the records in hbase:acl,
> we could introduce a prefix like ':::acl:::', since we do not allow ':' in
> either namespace or table name, so it will not conflict with the existing
> table related records. And the family could be namd as 'acl'.
> And we could make a special split policy that only splits at these special
> prefixs, so it will not break any assumptions so far, as all the records for
> the 'system table' are in the same region.
> {quote}
> And I think there are also other advantages, for example the start up logic
> can be greatly simplified.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2535: HBASE-25116 RegionMonitor support RegionTask count normalize
Apache-HBase commented on pull request #2535: URL: https://github.com/apache/hbase/pull/2535#issuecomment-707297199 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 18s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 0s | master passed | | +1 :green_heart: | compile | 1m 42s | master passed | | +1 :green_heart: | shadedjars | 7m 34s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 9s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 43s | the patch passed | | +1 :green_heart: | compile | 1m 50s | the patch passed | | +1 :green_heart: | javac | 1m 50s | the patch passed | | +1 :green_heart: | shadedjars | 7m 59s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 21s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 11s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 208m 2s | hbase-server in the patch passed. | | | | 246m 44s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2535/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2535 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 64f159ad2320 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c28220522d | | Default Java | 2020-01-14 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2535/2/testReport/ | | Max. process+thread count | 3417 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2535/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-22938) Fold all the system tables to hbase:meta
[
https://issues.apache.org/jira/browse/HBASE-22938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212603#comment-17212603
]
Michael Stack commented on HBASE-22938:
---
{quote}This will make the read replicas feature can not 100% work for meta and
system tables, as we can not use in-cluster replication for them to spread the
edits to the secondary replicas any more.
{quote}
IIUC, this was once the case but no longer given read replicas no longer keep
ongoing state (see '4.1 Skip maintaining zookeeper replication queue
(offsets/WALs)' in
https://docs.google.com/document/d/1jJWVc-idHhhgL4KDRpjMsQJKCl_NRaCLGiH3Wqwd3O8/edit#heading=h.5hn1d8pikvrr)
> Fold all the system tables to hbase:meta
>
>
> Key: HBASE-22938
> URL: https://issues.apache.org/jira/browse/HBASE-22938
> Project: HBase
> Issue Type: Brainstorming
>Reporter: Duo Zhang
>Priority: Major
>
> Quote my post on HBASE-15867 here, on how to deal with the dead lock when we
> want to store replication queues to hbase:replication table.
> {quote}
> We could add a special prefix in the row key for different system tables, and
> make a special family for it. For example, for all the records in hbase:acl,
> we could introduce a prefix like ':::acl:::', since we do not allow ':' in
> either namespace or table name, so it will not conflict with the existing
> table related records. And the family could be namd as 'acl'.
> And we could make a special split policy that only splits at these special
> prefixs, so it will not break any assumptions so far, as all the records for
> the 'system table' are in the same region.
> {quote}
> And I think there are also other advantages, for example the start up logic
> can be greatly simplified.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2536: "HBASE-25144 Add Hadoop-3.3.0 to personality hadoopcheck" try again
Apache-HBase commented on pull request #2536: URL: https://github.com/apache/hbase/pull/2536#issuecomment-707279625 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 31s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 26s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 8s | Maven dependency ordering for patch | ||| _ Other Tests _ | | | | 3m 56s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2536/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2536 | | Optional Tests | | | uname | Linux c1eb6ae53830 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c28220522d | | Max. process+thread count | 45 (vs. ulimit of 3) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2536/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25144) Add Hadoop-3.3.0 to personality hadoopcheck
[ https://issues.apache.org/jira/browse/HBASE-25144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212581#comment-17212581 ] Nick Dimiduk commented on HBASE-25144: -- New PR for this ticket is posted, https://github.com/apache/hbase/pull/2536. Also filed HBASE-25180. > Add Hadoop-3.3.0 to personality hadoopcheck > --- > > Key: HBASE-25144 > URL: https://issues.apache.org/jira/browse/HBASE-25144 > Project: HBase > Issue Type: Task > Components: build, community >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 3.0.0-alpha-1 > > > Now that Hadoop 3.3.0 is released, let's figure out where it goes in our > testing matrix. Start by adding it to precommit checks. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2536: "HBASE-25144 Add Hadoop-3.3.0 to personality hadoopcheck" try again
Apache-HBase commented on pull request #2536: URL: https://github.com/apache/hbase/pull/2536#issuecomment-707278922 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 3s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 8s | Maven dependency ordering for patch | | +1 :green_heart: | shellcheck | 0m 2s | There were no new shellcheck issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | ||| _ Other Tests _ | | +0 :ok: | asflicense | 0m 0s | ASF License check generated no output? | | | | 2m 27s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2536/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2536 | | Optional Tests | dupname asflicense shellcheck shelldocs | | uname | Linux 3066ab8cd473 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c28220522d | | Max. process+thread count | 49 (vs. ulimit of 3) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2536/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) shellcheck=0.4.6 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2536: "HBASE-25144 Add Hadoop-3.3.0 to personality hadoopcheck" try again
Apache-HBase commented on pull request #2536: URL: https://github.com/apache/hbase/pull/2536#issuecomment-707278285 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2536/1/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2536: "HBASE-25144 Add Hadoop-3.3.0 to personality hadoopcheck" try again
Apache-HBase commented on pull request #2536: URL: https://github.com/apache/hbase/pull/2536#issuecomment-707278559 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 7s | Maven dependency ordering for patch | ||| _ Other Tests _ | | | | 1m 39s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2536/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2536 | | Optional Tests | | | uname | Linux 72f24ba64209 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c28220522d | | Max. process+thread count | 58 (vs. ulimit of 3) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2536/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-25180) Update Hadoop compatibility guidelines in book re: Hadoop 3.3.0
Nick Dimiduk created HBASE-25180: Summary: Update Hadoop compatibility guidelines in book re: Hadoop 3.3.0 Key: HBASE-25180 URL: https://issues.apache.org/jira/browse/HBASE-25180 Project: HBase Issue Type: Task Components: community, documentation Affects Versions: 3.0.0-alpha-1 Reporter: Nick Dimiduk Discussion raised on HBASE-25144 indicated that the book need an explicit mention of Hadoop 3.3.0. Work out what that update is and apply it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2536: "HBASE-25144 Add Hadoop-3.3.0 to personality hadoopcheck" try again
Apache-HBase commented on pull request #2536: URL: https://github.com/apache/hbase/pull/2536#issuecomment-707278128 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk opened a new pull request #2536: "HBASE-25144 Add Hadoop-3.3.0 to personality hadoopcheck" try again
ndimiduk opened a new pull request #2536: URL: https://github.com/apache/hbase/pull/2536 Revert the previous commit, apply a new set of rules based on discussion on JIRA. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25144) Add Hadoop-3.3.0 to personality hadoopcheck
[ https://issues.apache.org/jira/browse/HBASE-25144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212570#comment-17212570 ] Sean Busbey commented on HBASE-25144: - Why? Jetty shouldn't be in our client exposed api? Or is this an operational compatibility concern for coprocessors that for some reason rely on jetty? > Add Hadoop-3.3.0 to personality hadoopcheck > --- > > Key: HBASE-25144 > URL: https://issues.apache.org/jira/browse/HBASE-25144 > Project: HBase > Issue Type: Task > Components: build, community >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 3.0.0-alpha-1 > > > Now that Hadoop 3.3.0 is released, let's figure out where it goes in our > testing matrix. Start by adding it to precommit checks. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25144) Add Hadoop-3.3.0 to personality hadoopcheck
[ https://issues.apache.org/jira/browse/HBASE-25144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212559#comment-17212559 ] Nick Dimiduk commented on HBASE-25144: -- So in order to run on any Hadoop that's upgraded Jetty, we need to either upgrade our own jetty dependency or move to shaded jetty, a la HBASE-19256 ? According to our own compatibility guidelines, the latter is the only viable approach that keeps 2.2 and 2.3 alive. > Add Hadoop-3.3.0 to personality hadoopcheck > --- > > Key: HBASE-25144 > URL: https://issues.apache.org/jira/browse/HBASE-25144 > Project: HBase > Issue Type: Task > Components: build, community >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 3.0.0-alpha-1 > > > Now that Hadoop 3.3.0 is released, let's figure out where it goes in our > testing matrix. Start by adding it to precommit checks. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2535: HBASE-25116 RegionMonitor support RegionTask count normalize
Apache-HBase commented on pull request #2535: URL: https://github.com/apache/hbase/pull/2535#issuecomment-707262663 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 26s | master passed | | +1 :green_heart: | compile | 1m 19s | master passed | | +1 :green_heart: | shadedjars | 6m 27s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 59s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 30s | the patch passed | | +1 :green_heart: | compile | 1m 20s | the patch passed | | +1 :green_heart: | javac | 1m 20s | the patch passed | | +1 :green_heart: | shadedjars | 6m 34s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 59s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 29s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 139m 31s | hbase-server in the patch passed. | | | | 168m 54s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2535/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2535 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux d9a2e8f516ee 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c28220522d | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2535/2/testReport/ | | Max. process+thread count | 4587 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2535/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] mnpoonia commented on a change in pull request #2523: HBASE-25167 Normalizer support for hot config reloading
mnpoonia commented on a change in pull request #2523:
URL: https://github.com/apache/hbase/pull/2523#discussion_r503440906
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SimpleRegionNormalizer.java
##
@@ -71,38 +76,81 @@
static final String MERGE_MIN_REGION_SIZE_MB_KEY =
"hbase.normalizer.merge.min_region_size.mb";
static final int DEFAULT_MERGE_MIN_REGION_SIZE_MB = 1;
- private Configuration conf;
private MasterServices masterServices;
- private boolean splitEnabled;
- private boolean mergeEnabled;
- private int minRegionCount;
- private Period mergeMinRegionAge;
- private int mergeMinRegionSizeMb;
+
+ /** Ensure configuration changes are applied atomically. */
+ private final ReadWriteLock configUpdateLock = new ReentrantReadWriteLock();
+ @GuardedBy("configUpdateLock") private Configuration conf;
Review comment:
This is cool. I believe this will help us avoid partial config update!
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-25156) TestMasterFailover.testSimpleMasterFailover is flaky
[
https://issues.apache.org/jira/browse/HBASE-25156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212530#comment-17212530
]
Nick Dimiduk commented on HBASE-25156:
--
Looks like this test finally landed in the flakey list. It wasn't run as part
of build#93.
> TestMasterFailover.testSimpleMasterFailover is flaky
> -
>
> Key: HBASE-25156
> URL: https://issues.apache.org/jira/browse/HBASE-25156
> Project: HBase
> Issue Type: Test
> Components: test
>Affects Versions: 3.0.0-alpha-1
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0, 2.2.7
>
>
> {noformat}
> [ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 32.996 s <<< FAILURE! - in org.apache.hadoop.hbase.master.TestMasterFailover
> [ERROR]
> org.apache.hadoop.hbase.master.TestMasterFailover.testSimpleMasterFailover
> Time elapsed: 12.317 s <<< FAILURE!
> java.lang.AssertionError: expected:<1> but was:<2>
> at
> org.apache.hadoop.hbase.master.TestMasterFailover.testSimpleMasterFailover(TestMasterFailover.java:133)
> {noformat}
> Looks like this test depends on metrics to be updated as a source of
> side-effect used to verify the test. Seems like it should retry the check a
> few times, or maybe we need a last-updated monotonic value that the test can
> check before and after it expects a change to be visible.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2531: HBASE-25065 - WAL archival to be done by a separate thread
Apache-HBase commented on pull request #2531: URL: https://github.com/apache/hbase/pull/2531#issuecomment-707223957 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 29s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 12s | branch-2 passed | | +1 :green_heart: | compile | 1m 13s | branch-2 passed | | +1 :green_heart: | shadedjars | 7m 32s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 43s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 0s | the patch passed | | +1 :green_heart: | compile | 1m 36s | the patch passed | | +1 :green_heart: | javac | 1m 36s | the patch passed | | +1 :green_heart: | shadedjars | 8m 2s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 44s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 203m 2s | hbase-server in the patch passed. | | | | 236m 21s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2531/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2531 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux f1028b375403 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 3beae80c60 | | Default Java | 2020-01-14 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2531/4/testReport/ | | Max. process+thread count | 2768 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2531/4/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-25065) WAL archival to be done by a separate thread
[ https://issues.apache.org/jira/browse/HBASE-25065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan resolved HBASE-25065. Fix Version/s: 2.4.0 3.0.0-alpha-1 Resolution: Fixed > WAL archival to be done by a separate thread > > > Key: HBASE-25065 > URL: https://issues.apache.org/jira/browse/HBASE-25065 > Project: HBase > Issue Type: Improvement > Components: wal >Affects Versions: 3.0.0-alpha-1, 2.4.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > Currently we do clean up of logs once we ensure that the region data has been > flushed. We track the sequence number and if we ensure that the seq number > has been flushed for any given region and the WAL that was rolled has that > seq number then those WAL can be archived. > When we have around ~50 files to archive (per RS) - we do the archiving one > after the other. Since archiving is nothing but a rename operation it adds to > the meta operation load of Cloud based FS. > Not only that - the entire archival is done inside the rollWriterLock. Though > we have closed the writer and created a new writer and the writes are ongoing > - we never release the lock until we are done with the archiving. > What happens is that during that period our logs grow in size compared to the > default size configured (when we have consistent writes happening). > So the proposal is to move the log archival to a seperate thread and ensure > we can do some kind of throttling or batching so that we don't do archival at > one shot. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] ramkrish86 commented on pull request #2531: HBASE-25065 - WAL archival to be done by a separate thread
ramkrish86 commented on pull request #2531: URL: https://github.com/apache/hbase/pull/2531#issuecomment-707221300 Pushed to branch-2. Removed the unused imports in test case and pushed it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ramkrish86 merged pull request #2531: HBASE-25065 - WAL archival to be done by a separate thread
ramkrish86 merged pull request #2531: URL: https://github.com/apache/hbase/pull/2531 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2399: HBASE-24877 Add option to avoid aborting RS process upon uncaught exceptions happen on replication source
Apache-HBase commented on pull request #2399: URL: https://github.com/apache/hbase/pull/2399#issuecomment-707217570 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 23s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 37s | branch-2 passed | | +1 :green_heart: | compile | 0m 57s | branch-2 passed | | +1 :green_heart: | shadedjars | 5m 57s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | branch-2 passed | | -0 :warning: | patch | 6m 49s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 15s | the patch passed | | +1 :green_heart: | compile | 0m 55s | the patch passed | | +1 :green_heart: | javac | 0m 55s | the patch passed | | +1 :green_heart: | shadedjars | 5m 57s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 137m 9s | hbase-server in the patch passed. | | | | 162m 30s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2399/7/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2399 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 7e736c0e2198 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 3beae80c60 | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2399/7/testReport/ | | Max. process+thread count | 3927 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2399/7/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2399: HBASE-24877 Add option to avoid aborting RS process upon uncaught exceptions happen on replication source
Apache-HBase commented on pull request #2399: URL: https://github.com/apache/hbase/pull/2399#issuecomment-707214438 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 36s | Docker mode activated. | | -0 :warning: | yetus | 0m 8s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 10s | branch-2 passed | | +1 :green_heart: | compile | 1m 6s | branch-2 passed | | +1 :green_heart: | shadedjars | 6m 41s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | branch-2 passed | | -0 :warning: | patch | 7m 34s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 1s | the patch passed | | +1 :green_heart: | compile | 1m 5s | the patch passed | | +1 :green_heart: | javac | 1m 5s | the patch passed | | +1 :green_heart: | shadedjars | 6m 38s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 129m 4s | hbase-server in the patch passed. | | | | 156m 48s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2399/7/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2399 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 56acbac3dd6b 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 3beae80c60 | | Default Java | 2020-01-14 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2399/7/testReport/ | | Max. process+thread count | 3788 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2399/7/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-25179) Assert format is incorrect in HFilePerformanceEvaluation class.
Rushabh Shah created HBASE-25179:
Summary: Assert format is incorrect in HFilePerformanceEvaluation
class.
Key: HBASE-25179
URL: https://issues.apache.org/jira/browse/HBASE-25179
Project: HBase
Issue Type: Improvement
Components: Performance
Reporter: Rushabh Shah
Assignee: Rushabh Shah
[HFilePerformanceEvaluation
|https://github.com/apache/hbase/blob/master/hbase-server/src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java#L518]
The format of expected and actual is interchanged.
{code:java}
PerformanceEvaluationCommons.assertValueSize(c.getValueLength(), ROW_LENGTH);
{code}
The first argument should be expected and second should be actual.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] huaxiangsun commented on pull request #2522: HBASE-25215 Create a ReplicationEndPoint for meta/root replica replic…
huaxiangsun commented on pull request #2522: URL: https://github.com/apache/hbase/pull/2522#issuecomment-707209866 Will update a new addressing review comments today. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on pull request #2515: HBASE-14067 bundle ruby files for hbase shell into a jar.
saintstack commented on pull request #2515: URL: https://github.com/apache/hbase/pull/2515#issuecomment-707204870 @busbey doesn't seem related [ERROR] TestBucketCache.testRetrieveFromPMem:296->testRetrievalUtils:333 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #2528: HBASE-25169 Update documentation about meta region replica
saintstack commented on a change in pull request #2528: URL: https://github.com/apache/hbase/pull/2528#discussion_r503387562 ## File path: src/main/asciidoc/_chapters/architecture.adoc ## @@ -2753,8 +2753,6 @@ See the above HDFS Architecture link for more information. [[arch.timelineconsistent.reads]] == Timeline-consistent High Available Reads -NOTE: The current <> does not work well with region replica, so this feature maybe broken. Use it with caution. - Review comment: Good This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2535: HBASE-25116 RegionMonitor support RegionTask count normalize
Apache-HBase commented on pull request #2535: URL: https://github.com/apache/hbase/pull/2535#issuecomment-707200650 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 3s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 20s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 0s | master passed | | +1 :green_heart: | checkstyle | 1m 35s | master passed | | +1 :green_heart: | spotbugs | 2m 50s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 47s | the patch passed | | +1 :green_heart: | checkstyle | 1m 34s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 18m 47s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 3m 9s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 22s | The patch does not generate ASF License warnings. | | | | 45m 42s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2535/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2535 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 1bd8fef449eb 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c28220522d | | Max. process+thread count | 84 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2535/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2531: HBASE-25065 - WAL archival to be done by a separate thread
Apache-HBase commented on pull request #2531: URL: https://github.com/apache/hbase/pull/2531#issuecomment-707192733 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 37s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 42s | branch-2 passed | | +1 :green_heart: | compile | 1m 11s | branch-2 passed | | +1 :green_heart: | shadedjars | 8m 1s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 47s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 23s | the patch passed | | +1 :green_heart: | compile | 1m 10s | the patch passed | | +1 :green_heart: | javac | 1m 10s | the patch passed | | +1 :green_heart: | shadedjars | 7m 36s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 42s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 146m 24s | hbase-server in the patch passed. | | | | 177m 49s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2531/4/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2531 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 597528d19ec8 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 3beae80c60 | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2531/4/testReport/ | | Max. process+thread count | 3493 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2531/4/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-25168) Unify WAL name timestamp parsers
[ https://issues.apache.org/jira/browse/HBASE-25168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack resolved HBASE-25168. --- Fix Version/s: 2.2.7 2.4.0 2.3.3 3.0.0-alpha-1 Hadoop Flags: Reviewed Assignee: Michael Stack Resolution: Fixed Merged to branch-2.2+ Thanks for reviews [~zhangduo] and [~psomogyi] > Unify WAL name timestamp parsers > > > Key: HBASE-25168 > URL: https://issues.apache.org/jira/browse/HBASE-25168 > Project: HBase > Issue Type: Bug >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.3, 2.4.0, 2.2.7 > > > Turns out there are two methods for extracting timestamp from WAL filename. > Fix. > Spotted by [~zhangduo] in review of HBASE-22976... > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] saintstack commented on pull request #2524: HBASE-25168 Unify WAL name timestamp parsers
saintstack commented on pull request #2524: URL: https://github.com/apache/hbase/pull/2524#issuecomment-707175000 Merged manually after fixing checkstyle. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack closed pull request #2524: HBASE-25168 Unify WAL name timestamp parsers
saintstack closed pull request #2524: URL: https://github.com/apache/hbase/pull/2524 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25144) Add Hadoop-3.3.0 to personality hadoopcheck
[ https://issues.apache.org/jira/browse/HBASE-25144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212427#comment-17212427 ] Duo Zhang commented on HBASE-25144: --- For for 2.3.x and 2.2.x, we only support 3.1.1-3.1.3, 3.2.0-3.2.1. For 3.3.0,3.2.2 and 3.1.4, the current support matrix will all be X, until we release 2.4.0. > Add Hadoop-3.3.0 to personality hadoopcheck > --- > > Key: HBASE-25144 > URL: https://issues.apache.org/jira/browse/HBASE-25144 > Project: HBase > Issue Type: Task > Components: build, community >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 3.0.0-alpha-1 > > > Now that Hadoop 3.3.0 is released, let's figure out where it goes in our > testing matrix. Start by adding it to precommit checks. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2498: HBASE-24996 Support CheckAndMutate in Region.batchMutate()
Apache-HBase commented on pull request #2498: URL: https://github.com/apache/hbase/pull/2498#issuecomment-707151396 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 30s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 27s | master passed | | +1 :green_heart: | compile | 1m 43s | master passed | | +1 :green_heart: | shadedjars | 6m 30s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 19s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 31s | the patch passed | | +1 :green_heart: | compile | 1m 44s | the patch passed | | +1 :green_heart: | javac | 1m 44s | the patch passed | | +1 :green_heart: | shadedjars | 6m 29s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 20s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 31s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 2s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 147m 52s | hbase-server in the patch passed. | | | | 179m 42s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2498/6/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2498 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux a2e91b155083 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 8eea052359 | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2498/6/testReport/ | | Max. process+thread count | 5036 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2498/6/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2498: HBASE-24996 Support CheckAndMutate in Region.batchMutate()
Apache-HBase commented on pull request #2498: URL: https://github.com/apache/hbase/pull/2498#issuecomment-707150939 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 24s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 28s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 50s | master passed | | +1 :green_heart: | compile | 2m 15s | master passed | | +1 :green_heart: | shadedjars | 7m 54s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 37s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 11s | the patch passed | | +1 :green_heart: | compile | 2m 3s | the patch passed | | +1 :green_heart: | javac | 2m 3s | the patch passed | | +1 :green_heart: | shadedjars | 6m 58s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 31s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 32s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 4s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 138m 32s | hbase-server in the patch passed. | | | | 178m 29s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2498/6/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2498 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 17a3c5b8bc8f 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 8eea052359 | | Default Java | 2020-01-14 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2498/6/testReport/ | | Max. process+thread count | 4343 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2498/6/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2399: HBASE-24877 Add option to avoid aborting RS process upon uncaught exceptions happen on replication source
Apache-HBase commented on pull request #2399: URL: https://github.com/apache/hbase/pull/2399#issuecomment-707151036 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 6m 31s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 0s | branch-2 passed | | +1 :green_heart: | checkstyle | 1m 20s | branch-2 passed | | +1 :green_heart: | spotbugs | 2m 18s | branch-2 passed | | -0 :warning: | patch | 2m 27s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 50s | the patch passed | | +1 :green_heart: | checkstyle | 1m 18s | hbase-server: The patch generated 0 new + 1 unchanged - 1 fixed = 1 total (was 2) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 14m 2s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 19s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 12s | The patch does not generate ASF License warnings. | | | | 43m 35s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2399/7/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2399 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 5ddbf1240e45 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 3beae80c60 | | Max. process+thread count | 84 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2399/7/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25144) Add Hadoop-3.3.0 to personality hadoopcheck
[ https://issues.apache.org/jira/browse/HBASE-25144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212404#comment-17212404 ] Sean Busbey commented on HBASE-25144: - What should the ref guide Hadoop recommendations table look like then? > Add Hadoop-3.3.0 to personality hadoopcheck > --- > > Key: HBASE-25144 > URL: https://issues.apache.org/jira/browse/HBASE-25144 > Project: HBase > Issue Type: Task > Components: build, community >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 3.0.0-alpha-1 > > > Now that Hadoop 3.3.0 is released, let's figure out where it goes in our > testing matrix. Start by adding it to precommit checks. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2531: HBASE-25065 - WAL archival to be done by a separate thread
Apache-HBase commented on pull request #2531: URL: https://github.com/apache/hbase/pull/2531#issuecomment-707111472 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 8s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 35s | branch-2 passed | | +1 :green_heart: | checkstyle | 1m 15s | branch-2 passed | | +1 :green_heart: | spotbugs | 2m 4s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 34s | the patch passed | | -0 :warning: | checkstyle | 1m 15s | hbase-server: The patch generated 2 new + 75 unchanged - 1 fixed = 77 total (was 76) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 12m 33s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 14s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 11s | The patch does not generate ASF License warnings. | | | | 35m 19s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2531/4/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2531 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 7eefccc2fbe8 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 3beae80c60 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2531/4/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 84 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2531/4/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2535: HBASE-25116 RegionMonitor support RegionTask count normalize
Apache-HBase commented on pull request #2535: URL: https://github.com/apache/hbase/pull/2535#issuecomment-707107253 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 8s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 20s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 43s | master passed | | +1 :green_heart: | compile | 1m 36s | master passed | | +1 :green_heart: | shadedjars | 7m 23s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 4s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 37s | the patch passed | | +1 :green_heart: | compile | 1m 36s | the patch passed | | +1 :green_heart: | javac | 1m 36s | the patch passed | | +1 :green_heart: | shadedjars | 7m 22s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 4s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 58s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 196m 34s | hbase-server in the patch passed. | | | | 231m 47s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2535/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2535 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux bbc3914cee91 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 8eea052359 | | Default Java | 2020-01-14 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2535/1/testReport/ | | Max. process+thread count | 3296 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2535/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ramkrish86 commented on pull request #2531: HBASE-25065 - WAL archival to be done by a separate thread
ramkrish86 commented on pull request #2531: URL: https://github.com/apache/hbase/pull/2531#issuecomment-707093235 TestMasterRegionWALCleaner failure is related to the change here. Not a logical issue but the MasterWAL cleaner tries to move the data from the walArchive dir to globalWALArchive dir. If by the time the afterRoll() method does not see the oldWAL directory with the WAL file it will just return without moving. This is causing the failure to happen. So i just changed the test to wait for > 8 secs (as it was doing previously) to verify at the end the log cleaner has deleted the globalWALArchive dir. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] brfrn169 commented on a change in pull request #2498: HBASE-24996 Support CheckAndMutate in Region.batchMutate()
brfrn169 commented on a change in pull request #2498:
URL: https://github.com/apache/hbase/pull/2498#discussion_r503230843
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HMobStore.java
##
@@ -139,7 +139,8 @@ public Configuration getConfiguration() {
*/
@Override
protected KeyValueScanner createScanner(Scan scan, ScanInfo scanInfo,
- NavigableSet targetCols, long readPt) throws IOException {
+NavigableSet targetCols, long readPt,List
additionalScanners)
Review comment:
It Has to be a KVScanner. I will explain the reason in the following
comment:
https://github.com/apache/hbase/pull/2498#discussion_r502654326
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] brfrn169 commented on a change in pull request #2498: HBASE-24996 Support CheckAndMutate in Region.batchMutate()
brfrn169 commented on a change in pull request #2498:
URL: https://github.com/apache/hbase/pull/2498#discussion_r503229958
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -4074,29 +4183,128 @@ private static long getLongValue(final Cell cell)
throws DoNotRetryIOException {
return PrivateCellUtil.getValueAsLong(cell);
}
-/**
- * Do a specific Get on passed columnFamily and column
qualifiers.
- * @param mutation Mutation we are doing this Get for.
- * @param store Which column family on row (TODO: Go all Gets in one go)
- * @param coordinates Cells from mutation used as coordinates
applied to Get.
- * @return Return list of Cells found.
- */
-private List get(Mutation mutation, HStore store, List
coordinates,
- TimeRange tr) throws IOException {
- // Sort the cells so that they match the order that they appear in the
Get results.
- // Otherwise, we won't be able to find the existing values if the cells
are not specified
- // in order by the client since cells are in an array list.
- // TODO: I don't get why we are sorting. St.Ack 20150107
Review comment:
We need the sorting actually, otherwise it will return a wrong result to
the client. I keep this sorting here:
https://github.com/brfrn169/hbase/blob/HBASE-24996/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java#L4072-L4075
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-25156) TestMasterFailover.testSimpleMasterFailover is flaky
[
https://issues.apache.org/jira/browse/HBASE-25156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212324#comment-17212324
]
Hudson commented on HBASE-25156:
Results for branch master
[build #93 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/93/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/93/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/93/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/93/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> TestMasterFailover.testSimpleMasterFailover is flaky
> -
>
> Key: HBASE-25156
> URL: https://issues.apache.org/jira/browse/HBASE-25156
> Project: HBase
> Issue Type: Test
> Components: test
>Affects Versions: 3.0.0-alpha-1
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0, 2.2.7
>
>
> {noformat}
> [ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 32.996 s <<< FAILURE! - in org.apache.hadoop.hbase.master.TestMasterFailover
> [ERROR]
> org.apache.hadoop.hbase.master.TestMasterFailover.testSimpleMasterFailover
> Time elapsed: 12.317 s <<< FAILURE!
> java.lang.AssertionError: expected:<1> but was:<2>
> at
> org.apache.hadoop.hbase.master.TestMasterFailover.testSimpleMasterFailover(TestMasterFailover.java:133)
> {noformat}
> Looks like this test depends on metrics to be updated as a source of
> side-effect used to verify the test. Seems like it should retry the check a
> few times, or maybe we need a last-updated monotonic value that the test can
> check before and after it expects a change to be visible.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25093) the RSGroupBasedLoadBalancer#retainAssignment throws NPE
[
https://issues.apache.org/jira/browse/HBASE-25093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212326#comment-17212326
]
Hudson commented on HBASE-25093:
Results for branch master
[build #93 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/93/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/93/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/93/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/93/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> the RSGroupBasedLoadBalancer#retainAssignment throws NPE
>
>
> Key: HBASE-25093
> URL: https://issues.apache.org/jira/browse/HBASE-25093
> Project: HBase
> Issue Type: Bug
> Components: rsgroup
>Affects Versions: 3.0.0-alpha-1, 2.3.1, 2.2.3
>Reporter: Bo Cui
>Assignee: niuyulin
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 2.4.0, 2.2.7
>
>
> when BaseLoadBalancer#
> https://github.com/apache/hbase/blob/8bfa2cb2eedcf050b26a28961e1b77dbf3cd8c95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java#L1433
> If the result of the BaseLoadBalancer#retainAssignment is null, the
> RSGroupBasedLoadBalancer#retainAssignment will throw NPE.
> https://github.com/apache/hbase/blob/8bfa2cb2eedcf050b26a28961e1b77dbf3cd8c95/hbase-server/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupBasedLoadBalancer.java#L206
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25175) Remove the constructors of HBaseConfiguration
[
https://issues.apache.org/jira/browse/HBASE-25175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212325#comment-17212325
]
Hudson commented on HBASE-25175:
Results for branch master
[build #93 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/93/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/93/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/93/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/93/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Remove the constructors of HBaseConfiguration
> -
>
> Key: HBASE-25175
> URL: https://issues.apache.org/jira/browse/HBASE-25175
> Project: HBase
> Issue Type: Sub-task
> Components: API
>Affects Versions: 3.0.0-alpha-1
>Reporter: Duo Zhang
>Assignee: niuyulin
>Priority: Major
> Fix For: 3.0.0-alpha-1
>
>
> As they have been marked as deprecated long ago.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24840) Avoid shows closing region task when create table
[
https://issues.apache.org/jira/browse/HBASE-24840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212323#comment-17212323
]
Hudson commented on HBASE-24840:
Results for branch master
[build #93 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/93/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/93/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/93/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/93/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Avoid shows closing region task when create table
> -
>
> Key: HBASE-24840
> URL: https://issues.apache.org/jira/browse/HBASE-24840
> Project: HBase
> Issue Type: Improvement
> Components: monitoring
>Reporter: Zheng Wang
>Assignee: Zheng Wang
>Priority: Major
> Attachments: HBASE-24840-afterpatch.png, HBASE-24840-beforepatch.png
>
>
> Currently when we create region dir, we new a HRegion and then close it, this
> step is before the real initialize, so there will exist a closing region task
> at first, it is strange, and will confuse users.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2498: HBASE-24996 Support CheckAndMutate in Region.batchMutate()
Apache-HBase commented on pull request #2498: URL: https://github.com/apache/hbase/pull/2498#issuecomment-707082936 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 27s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 19s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 57s | master passed | | +1 :green_heart: | checkstyle | 2m 8s | master passed | | +1 :green_heart: | spotbugs | 3m 50s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 45s | the patch passed | | +1 :green_heart: | checkstyle | 0m 23s | The patch passed checkstyle in hbase-common | | +1 :green_heart: | checkstyle | 0m 28s | The patch passed checkstyle in hbase-client | | +1 :green_heart: | checkstyle | 1m 17s | hbase-server: The patch generated 0 new + 401 unchanged - 1 fixed = 401 total (was 402) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 18m 52s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 4m 17s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 32s | The patch does not generate ASF License warnings. | | | | 50m 54s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2498/6/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2498 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux cc46e5a00bac 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 8eea052359 | | Max. process+thread count | 84 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2498/6/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2535: HBASE-25116 RegionMonitor support RegionTask count normalize
Apache-HBase commented on pull request #2535: URL: https://github.com/apache/hbase/pull/2535#issuecomment-707082516 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 42s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 24s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 44s | master passed | | +1 :green_heart: | compile | 1m 38s | master passed | | +1 :green_heart: | shadedjars | 9m 5s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 12s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 4s | the patch passed | | +1 :green_heart: | compile | 1m 37s | the patch passed | | +1 :green_heart: | javac | 1m 37s | the patch passed | | +1 :green_heart: | shadedjars | 8m 43s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 8s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 43s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 142m 30s | hbase-server in the patch passed. | | | | 180m 16s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2535/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2535 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux c668c1f63953 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 8eea052359 | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2535/1/testReport/ | | Max. process+thread count | 4230 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2535/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2531: HBASE-25065 - WAL archival to be done by a separate thread
Apache-HBase commented on pull request #2531: URL: https://github.com/apache/hbase/pull/2531#issuecomment-707068659 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 29s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 45s | branch-2 passed | | +1 :green_heart: | compile | 1m 11s | branch-2 passed | | +1 :green_heart: | shadedjars | 7m 29s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 43s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 30s | the patch passed | | +1 :green_heart: | compile | 1m 11s | the patch passed | | +1 :green_heart: | javac | 1m 11s | the patch passed | | +1 :green_heart: | shadedjars | 7m 24s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 41s | hbase-server generated 1 new + 68 unchanged - 0 fixed = 69 total (was 68) | ||| _ Other Tests _ | | +1 :green_heart: | unit | 191m 56s | hbase-server in the patch passed. | | | | 223m 1s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2531/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2531 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux d2a8ddc719b7 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 3beae80c60 | | Default Java | 2020-01-14 | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2531/3/artifact/yetus-jdk11-hadoop3-check/output/diff-javadoc-javadoc-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2531/3/testReport/ | | Max. process+thread count | 3004 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2531/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (HBASE-25176) MasterStoppedException should be moved to hbase-client module
[ https://issues.apache.org/jira/browse/HBASE-25176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] niuyulin reassigned HBASE-25176: Assignee: niuyulin > MasterStoppedException should be moved to hbase-client module > - > > Key: HBASE-25176 > URL: https://issues.apache.org/jira/browse/HBASE-25176 > Project: HBase > Issue Type: Bug > Components: Client >Reporter: Duo Zhang >Assignee: niuyulin >Priority: Major > > Since it will be thrown from the HMaster.checkInitialized which will be > called in a rpc call method, which means it will be propagated to client, we > should move this class to hbase-client. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2531: HBASE-25065 - WAL archival to be done by a separate thread
Apache-HBase commented on pull request #2531: URL: https://github.com/apache/hbase/pull/2531#issuecomment-707042205 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 16s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 40s | branch-2 passed | | +1 :green_heart: | compile | 1m 2s | branch-2 passed | | +1 :green_heart: | shadedjars | 6m 17s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 28s | the patch passed | | +1 :green_heart: | compile | 1m 1s | the patch passed | | +1 :green_heart: | javac | 1m 1s | the patch passed | | +1 :green_heart: | shadedjars | 6m 11s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 37s | hbase-server generated 1 new + 13 unchanged - 0 fixed = 14 total (was 13) | ||| _ Other Tests _ | | -1 :x: | unit | 138m 52s | hbase-server in the patch failed. | | | | 165m 10s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2531/3/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2531 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux d5baa7310456 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 3beae80c60 | | Default Java | 1.8.0_232 | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2531/3/artifact/yetus-jdk8-hadoop2-check/output/diff-javadoc-javadoc-hbase-server.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2531/3/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2531/3/testReport/ | | Max. process+thread count | 4179 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2531/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2534: Backport HBASE-25093 the RSGroupBasedLoadBalancer#retainAssignment throws NPE
Apache-HBase commented on pull request #2534: URL: https://github.com/apache/hbase/pull/2534#issuecomment-707029068 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 6m 49s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 33s | branch-2 passed | | +1 :green_heart: | compile | 1m 20s | branch-2 passed | | +1 :green_heart: | shadedjars | 5m 57s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 10s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 26s | the patch passed | | +1 :green_heart: | compile | 1m 25s | the patch passed | | +1 :green_heart: | javac | 1m 25s | the patch passed | | +1 :green_heart: | shadedjars | 6m 3s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 58s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 147m 57s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 4m 15s | hbase-rsgroup in the patch passed. | | | | 185m 55s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2534/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2534 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 85347644f8d8 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 3beae80c60 | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2534/1/testReport/ | | Max. process+thread count | 3787 (vs. ulimit of 12500) | | modules | C: hbase-server hbase-rsgroup U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2534/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2534: Backport HBASE-25093 the RSGroupBasedLoadBalancer#retainAssignment throws NPE
Apache-HBase commented on pull request #2534: URL: https://github.com/apache/hbase/pull/2534#issuecomment-707019907 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 4m 1s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 22s | branch-2 passed | | +1 :green_heart: | compile | 1m 30s | branch-2 passed | | +1 :green_heart: | shadedjars | 6m 40s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 5s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 59s | the patch passed | | +1 :green_heart: | compile | 1m 32s | the patch passed | | +1 :green_heart: | javac | 1m 32s | the patch passed | | +1 :green_heart: | shadedjars | 6m 38s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 4s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 132m 0s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 4m 3s | hbase-rsgroup in the patch passed. | | | | 169m 56s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2534/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2534 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux a954286f5471 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 3beae80c60 | | Default Java | 2020-01-14 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2534/1/testReport/ | | Max. process+thread count | 4317 (vs. ulimit of 12500) | | modules | C: hbase-server hbase-rsgroup U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2534/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
