[jira] [Commented] (HBASE-25472) HBASE-24260 breaks JDK7 compilation in branch-1
[
https://issues.apache.org/jira/browse/HBASE-25472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17260297#comment-17260297
]
Viraj Jasani commented on HBASE-25472:
--
[~weichiu] I think this is taken care of by
[https://github.com/apache/hbase/commit/73d079eb67abdded404e0438eb75d2c4b1264644]
JDK7 nightly report seems fine, mvninstall is green:
https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/73/JDK7_20Nightly_20Build_20Report/
> HBASE-24260 breaks JDK7 compilation in branch-1
> ---
>
> Key: HBASE-25472
> URL: https://issues.apache.org/jira/browse/HBASE-25472
> Project: HBase
> Issue Type: Bug
>Reporter: Wei-Chiu Chuang
>Priority: Blocker
>
> Found in PR-2799.
> {code}
> public static String getZKQuorum(Configuration conf) {
> String port =
> Integer.toString(conf.getInt(HConstants.ZOOKEEPER_CLIENT_PORT, 2181));
> String[] serverHosts = conf.getStrings(HConstants.ZOOKEEPER_QUORUM,
> "localhost");
> for (int i = 0; i < serverHosts.length; i++) {
> serverHosts[i] = serverHosts[i] + ":" + port;
> }
> return String.join(",", serverHosts);
> }
> {code}
> Fails to compile
> {quote}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile
> (default-compile) on project hbase-it: Compilation failure
> [ERROR]
> /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2799/src/hbase-it/src/main/java/org/apache/hadoop/hbase/chaos/ChaosUtils.java:[46,18]
> cannot find symbol
> [ERROR] symbol: method join(java.lang.String,java.lang.String[])
> [ERROR] location: class java.lang.String
> {quote}
> because String.join() is only available since JDK8.
> CC: [~lokiore] [~ndimiduk]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HBASE-21876) Kill the region server if we can not send request to it for a long time in RSProcedureDispatcher
[ https://issues.apache.org/jira/browse/HBASE-21876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laxman Goswami reassigned HBASE-21876: -- Assignee: Laxman Goswami > Kill the region server if we can not send request to it for a long time in > RSProcedureDispatcher > - > > Key: HBASE-21876 > URL: https://issues.apache.org/jira/browse/HBASE-21876 > Project: HBase > Issue Type: Improvement > Components: proc-v2 >Reporter: Duo Zhang >Assignee: Laxman Goswami >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] virajjasani commented on pull request #2844: HBASE-25445: Use WAL FS instead of master FS in SplitWALManager
virajjasani commented on pull request #2844: URL: https://github.com/apache/hbase/pull/2844#issuecomment-755947259 I think there seems some problem with Jenkins, the recent build is aborted and I am not able to schedule a new build. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on a change in pull request #2779: HBASE-23340 hmaster /hbase/replication/rs session expired (hbase repl…
Apache9 commented on a change in pull request #2779:
URL: https://github.com/apache/hbase/pull/2779#discussion_r553155282
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java
##
@@ -92,12 +96,20 @@ public boolean apply(FileStatus file) {
}
@Override
- public void setConf(Configuration config) {
-// Make my own Configuration. Then I'll have my own connection to zk that
-// I can close myself when comes time.
-Configuration conf = new Configuration(config);
+ public void init(Map params) {
+super.init(params);
try {
- setConf(conf, new ZKWatcher(conf, "replicationLogCleaner", null));
+ if (MapUtils.isNotEmpty(params)) {
+Object master = params.get(HMaster.MASTER);
+if (master != null && master instanceof HMaster) {
+ zkw = ((HMaster) master).getZooKeeper();
+ shareZK = true;
Review comment:
Better set this flag to true by default, and in the below `if (zkw ==
null)`, set it to false. Without any other assumptions, it is possible that
HMaster.getZooKeeper returns null and we will create a new ZKWatcher but
shareZK is false then we will not close it...
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] pankaj72981 commented on pull request #2779: HBASE-23340 hmaster /hbase/replication/rs session expired (hbase repl…
pankaj72981 commented on pull request #2779: URL: https://github.com/apache/hbase/pull/2779#issuecomment-755934024 > Will take a look soon. Thank you.. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] pankaj72981 edited a comment on pull request #2779: HBASE-23340 hmaster /hbase/replication/rs session expired (hbase repl…
pankaj72981 edited a comment on pull request #2779: URL: https://github.com/apache/hbase/pull/2779#issuecomment-755926452 @Apache9 Should we merge this PR? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #2779: HBASE-23340 hmaster /hbase/replication/rs session expired (hbase repl…
Apache9 commented on pull request #2779: URL: https://github.com/apache/hbase/pull/2779#issuecomment-755933293 Will take a look soon. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] virajjasani commented on pull request #2844: HBASE-25445: Use WAL FS instead of master FS in SplitWALManager
virajjasani commented on pull request #2844: URL: https://github.com/apache/hbase/pull/2844#issuecomment-755930193 Once QA results are in, will merge this PR. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] pankaj72981 commented on pull request #2779: HBASE-23340 hmaster /hbase/replication/rs session expired (hbase repl…
pankaj72981 commented on pull request #2779: URL: https://github.com/apache/hbase/pull/2779#issuecomment-755926452 @Apache9 Should we merge this MR? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-25473) [create-release] checkcompatibility.py failing with "KeyError: 'binary'"
Michael Stack created HBASE-25473:
-
Summary: [create-release] checkcompatibility.py failing with
"KeyError: 'binary'"
Key: HBASE-25473
URL: https://issues.apache.org/jira/browse/HBASE-25473
Project: HBase
Issue Type: Bug
Reporter: Michael Stack
Below is excerpt from log of RC0 2.3.4 build this afternoon when we try to do
the compatibility check (inside docker create-release build). It had failed for
me previously on another run of different versions. The failure is
unfortunately cryptic.
{code}
2021-01-07T01:08:34Z generate_api_report start
INFO:root:Source revision: rel/2.3.3
INFO:root:Destination revision: 2.3.4RC0
INFO:root:Filtering classes using 1 annotation(s):
INFO:root: org.apache.yetus.audience.InterfaceAudience.Public
INFO:root:Downloading Java ACC...
INFO:root:Java ACC version: 2.4
INFO:root:Removing scratch dir
/home/vagrant/hbase-rm/output/hbase/target/compat-check
INFO:root:Creating empty scratch dir
/home/vagrant/hbase-rm/output/hbase/target/compat-check
INFO:root:Checking out 5f7fe66efbdfc1c0b3262c0677f17e4055ba94cd in
/home/vagrant/hbase-rm/output/hbase/target/compat-check/src
INFO:root:Checking out ec79612b9ac6afd73fa8fc7f7f5c8d3bac277679 in
/home/vagrant/hbase-rm/output/hbase/target/compat-check/dst
INFO:root:Building in
/home/vagrant/hbase-rm/output/hbase/target/compat-check/src
INFO:root:Building in
/home/vagrant/hbase-rm/output/hbase/target/compat-check/dst
INFO:root:Will check compatibility between original jars:
/home/vagrant/hbase-rm/output/hbase/target/compat-check/src/hbase-shaded/hbase-shaded-client-byo-hadoop/target/original-hbase-shaded-client-byo-hadoop-2.3.3.jar
/home/vagrant/hbase-rm/output/hbase/target/compat-check/src/hbase-external-blockcache/target/hbase-external-blockcache-2.3.3.jar
/home/vagrant/hbase-rm/output/hbase/target/compat-check/src/hbase-shaded/hbase-shaded-client/target/original-hbase-shaded-client-2.3.3.jar
/home/vagrant/hbase-rm/output/hbase/target/compat-check/src/hbase-zookeeper/target/hbase-zookeeper-2.3.3.jar
/home/vagrant/hbase-rm/output/hbase/target/compat-check/src/hbase-common/target/hbase-common-2.3.3.jar
/home/vagrant/hbase-rm/output/hbase/target/compat-check/dst/hbase-shaded/hbase-shaded-testing-util-tester/target/hbase-shaded-testing-util-tester-2.3.4.jar
/home/vagrant/hbase-rm/output/hbase/target/compat-check/dst/hbase-archetypes/hbase-shaded-client-project/target/hbase-shaded-client-project-2.3.4.jar
/home/vagrant/hbase-rm/output/hbase/target/compat-check/dst/hbase-protocol/target/hbase-protocol-2.3.4.jar
INFO:root:Annotations are:
['org.apache.yetus.audience.InterfaceAudience.Public']
INFO:root:Annotations path:
/home/vagrant/hbase-rm/output/hbase/target/compat-check/annotations.txt
INFO:root:Results: {}
Traceback (most recent call last):
File "./hbase/dev-support/checkcompatibility.py", line 530, in
main()
File "./hbase/dev-support/checkcompatibility.py", line 526, in main
args.compare_warnings))
File "./hbase/dev-support/checkcompatibility.py", line 232, in compare_results
if tool_results[check][issue_type] > known_count]
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25470) Add unitest for HBASE-25445 - SplitWALRemoteProcedure failed to archive split WAL
[ https://issues.apache.org/jira/browse/HBASE-25470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17260265#comment-17260265 ] Viraj Jasani commented on HBASE-25470: -- Hopefully we will not need this as PR on HBASE-25445 is almost ready for commit. > Add unitest for HBASE-25445 - SplitWALRemoteProcedure failed to archive split > WAL > - > > Key: HBASE-25470 > URL: https://issues.apache.org/jira/browse/HBASE-25470 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 3.0.0-alpha-1, 2.4.0, 2.2.6, 2.3.2 >Reporter: mokai >Assignee: Anjan Das >Priority: Major > Fix For: 3.0.0-alpha-1, 2.2.7, 2.5.0, 2.4.1 > > > If 'hbase.wal.dir' and 'hbase.rootdir' are configured to different > filesystem, SplitWALRemoteProcedure archived split WAL failed since > SplitWALManager using wrong fs instance. SplitWALManager should use WAL > corresponding fs instance. > Steps to Reproduce: > * Configure 'hbase.wal.dir' and 'hbase.rootdir' so that they point to > different fs instances. > * Start HBase with multiple RS. > * Create a couple of tables and some rows in them so that the RSs get > assigned with some regions. > * Take any RS with non-zero number of regions offline. > * Check master logs for "Wrong FS" error as shown in the screenshot > attached. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25445) Old WALs archive fails in procedure based WAL split
[ https://issues.apache.org/jira/browse/HBASE-25445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17260264#comment-17260264 ] Viraj Jasani commented on HBASE-25445: -- Thanks [~huaxiangsun]. Since all reviews were happening on PR #2844, [~dasanjan1296] has checked in test already. Hopefully this should be merged very soon (at least before start of day in PST today). > Old WALs archive fails in procedure based WAL split > --- > > Key: HBASE-25445 > URL: https://issues.apache.org/jira/browse/HBASE-25445 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 3.0.0-alpha-1, 2.4.0, 2.2.6, 2.3.2 >Reporter: mokai >Assignee: Anjan Das >Priority: Critical > Fix For: 3.0.0-alpha-1, 2.2.7, 2.3.4, 2.5.0, 2.4.1 > > Attachments: ServerCrashWrongFSError.png > > > If 'hbase.wal.dir' and 'hbase.rootdir' are configured to different > filesystem, SplitWALRemoteProcedure archived split WAL failed since > SplitWALManager using wrong fs instance. SplitWALManager should use WAL > corresponding fs instance. > Steps to Reproduce: > * Configure 'hbase.wal.dir' and 'hbase.rootdir' so that they point to > different fs instances. > * Start HBase with multiple RS. > * Create a couple of tables and some rows in them so that the RSs get > assigned with some regions. > * Take any RS with non-zero number of regions offline. > * Check master logs for "Wrong FS" error as shown in the screenshot > attached. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] virajjasani commented on a change in pull request #2844: HBASE-25445: Use WAL FS instead of master FS in SplitWALManager
virajjasani commented on a change in pull request #2844:
URL: https://github.com/apache/hbase/pull/2844#discussion_r553132840
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestSplitWALManager.java
##
@@ -86,6 +99,59 @@ public void teardown() throws Exception {
TEST_UTIL.shutdownMiniCluster();
}
+ @Test
+ public void testWALArchiveWithDifferentWalAndRootFS() throws Exception{
+HBaseTestingUtility TEST_UTIL_2 = new HBaseTestingUtility();
Review comment:
Since this is not class level constant, no need to keep it in capital
letters, keep it `test_util_2`
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestSplitWALManager.java
##
@@ -86,6 +99,59 @@ public void teardown() throws Exception {
TEST_UTIL.shutdownMiniCluster();
}
+ @Test
+ public void testWALArchiveWithDifferentWalAndRootFS() throws Exception{
+HBaseTestingUtility TEST_UTIL_2 = new HBaseTestingUtility();
+Path dir = TEST_UTIL.getDataTestDirOnTestFS("testWalDir");
+TEST_UTIL_2.getConfiguration().set(CommonFSUtils.HBASE_WAL_DIR,
dir.toString());
+CommonFSUtils.setWALRootDir(TEST_UTIL_2.getConfiguration(), dir);
+TEST_UTIL_2.startMiniCluster(3);
+HMaster TU2_master = TEST_UTIL_2.getHBaseCluster().getMaster();
Review comment:
Same here, this can be `master2` or something similar
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestSplitWALManager.java
##
@@ -86,6 +99,59 @@ public void teardown() throws Exception {
TEST_UTIL.shutdownMiniCluster();
}
+ @Test
+ public void testWALArchiveWithDifferentWalAndRootFS() throws Exception{
+HBaseTestingUtility TEST_UTIL_2 = new HBaseTestingUtility();
+Path dir = TEST_UTIL.getDataTestDirOnTestFS("testWalDir");
+TEST_UTIL_2.getConfiguration().set(CommonFSUtils.HBASE_WAL_DIR,
dir.toString());
+CommonFSUtils.setWALRootDir(TEST_UTIL_2.getConfiguration(), dir);
+TEST_UTIL_2.startMiniCluster(3);
+HMaster TU2_master = TEST_UTIL_2.getHBaseCluster().getMaster();
+LOG.info("The Master FS is pointing to: " +
TU2_master.getMasterFileSystem()
+ .getFileSystem().getUri());
+LOG.info("The WAL FS is pointing to: " + TU2_master.getMasterFileSystem()
+ .getWALFileSystem().getUri());
+Table TABLE = TEST_UTIL_2.createTable(TABLE_NAME, FAMILY);
+TEST_UTIL_2.waitTableAvailable(TABLE_NAME);
+Admin admin = TEST_UTIL_2.getAdmin();
+MasterProcedureEnv env = TEST_UTIL_2.getMiniHBaseCluster().getMaster()
+ .getMasterProcedureExecutor().getEnvironment();
+final ProcedureExecutor executor =
TEST_UTIL_2.getMiniHBaseCluster()
+ .getMaster().getMasterProcedureExecutor();
+List regionInfos = admin.getRegions(TABLE_NAME);
+SplitTableRegionProcedure splitProcedure = new SplitTableRegionProcedure(
+ env, regionInfos.get(0), Bytes.toBytes("row5"));
+// Populate some rows in the table
+LOG.info("Beginning put data to the table: " + TABLE_NAME.toString());
+int rowCount = 5;
+for (int i = 0; i < rowCount; i++) {
+ byte[] row = Bytes.toBytes("row" + i);
+ Put put = new Put(row);
+ put.addColumn(FAMILY, FAMILY, FAMILY);
+ TABLE.put(put);
+}
+executor.submitProcedure(splitProcedure);
+LOG.info("Submitted SplitProcedure.");
+TEST_UTIL_2.waitFor(3, () -> executor.getProcedures().stream()
+ .filter(p -> p instanceof TransitRegionStateProcedure)
+ .map(p -> (TransitRegionStateProcedure) p)
+ .anyMatch(p -> TABLE_NAME.equals(p.getTableName(;
+TEST_UTIL_2.getMiniHBaseCluster().killRegionServer(
+ TEST_UTIL_2.getMiniHBaseCluster().getRegionServer(0).getServerName());
+TEST_UTIL_2.getMiniHBaseCluster().startRegionServer();
+TEST_UTIL_2.waitUntilNoRegionsInTransition();
+Scan scan = new Scan();
+ResultScanner results = TABLE.getScanner(scan);
+int scanRowCount = 0;
+Result result = null;
+while ((result = results.next()) != null) {
Review comment:
Not using `result` will work?
```
while (results.next() != null) {
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2844: HBASE-25445: Use WAL FS instead of master FS in SplitWALManager
Apache-HBase commented on pull request #2844: URL: https://github.com/apache/hbase/pull/2844#issuecomment-755912234 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 31s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 58s | master passed | | +1 :green_heart: | checkstyle | 1m 22s | master passed | | +1 :green_heart: | spotbugs | 2m 38s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 55s | the patch passed | | -0 :warning: | checkstyle | 1m 24s | hbase-server: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 22m 0s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 40s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 12s | The patch does not generate ASF License warnings. | | | | 50m 19s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2844/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2844 | | JIRA Issue | HBASE-25445 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux a1e5e14b038f 4.15.0-126-generic #129-Ubuntu SMP Mon Nov 23 18:53:38 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a414361ed9 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2844/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 85 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2844/3/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2853: HBASE-25471 Move RegionScannerImpl out of HRegion
Apache-HBase commented on pull request #2853: URL: https://github.com/apache/hbase/pull/2853#issuecomment-755906010 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 52s | master passed | | +1 :green_heart: | compile | 0m 56s | master passed | | +1 :green_heart: | shadedjars | 6m 41s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 36s | the patch passed | | +1 :green_heart: | compile | 0m 57s | the patch passed | | +1 :green_heart: | javac | 0m 57s | the patch passed | | +1 :green_heart: | shadedjars | 6m 38s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 140m 15s | hbase-server in the patch passed. | | | | 166m 40s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2853/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2853 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux d7e3fbafcef1 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a414361ed9 | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2853/1/testReport/ | | Max. process+thread count | 4391 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2853/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2853: HBASE-25471 Move RegionScannerImpl out of HRegion
Apache-HBase commented on pull request #2853: URL: https://github.com/apache/hbase/pull/2853#issuecomment-755904072 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 24s | master passed | | +1 :green_heart: | compile | 1m 7s | master passed | | +1 :green_heart: | shadedjars | 6m 46s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 44s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 2s | the patch passed | | +1 :green_heart: | compile | 1m 5s | the patch passed | | +1 :green_heart: | javac | 1m 5s | the patch passed | | +1 :green_heart: | shadedjars | 6m 38s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 133m 14s | hbase-server in the patch passed. | | | | 161m 27s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2853/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2853 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 1cddc59683e8 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a414361ed9 | | Default Java | AdoptOpenJDK-11.0.6+10 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2853/1/testReport/ | | Max. process+thread count | 4438 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2853/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] shenshengli commented on a change in pull request #2816: HBASE-25449 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml
shenshengli commented on a change in pull request #2816: URL: https://github.com/apache/hbase/pull/2816#discussion_r553120672 ## File path: hbase-common/src/main/resources/hbase-default.xml ## @@ -1463,15 +1463,15 @@ possible configurations would overwhelm and obscure the important. dfs.client.read.shortcircuit -false + If set to true, this configuration parameter enables short-circuit local reads. dfs.domain.socket.path -none + Review comment: It is reserved to provide description.This empty config property would not mask the real one.You can verify this from unit tests. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2800: HBASE-25249 Adding StoreContext
Apache-HBase commented on pull request #2800: URL: https://github.com/apache/hbase/pull/2800#issuecomment-755883881 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 38s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 29s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 44s | master passed | | +1 :green_heart: | checkstyle | 1m 41s | master passed | | +1 :green_heart: | spotbugs | 3m 30s | master passed | | -0 :warning: | patch | 1m 10s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 29s | the patch passed | | +1 :green_heart: | checkstyle | 1m 31s | hbase-server: The patch generated 0 new + 42 unchanged - 1 fixed = 42 total (was 43) | | +1 :green_heart: | checkstyle | 0m 20s | The patch passed checkstyle in hbase-mapreduce | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 24m 0s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 4m 3s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 27s | The patch does not generate ASF License warnings. | | | | 55m 48s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2800/12/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2800 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux caa9fb3d873d 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a414361ed9 | | Max. process+thread count | 95 (vs. ulimit of 3) | | modules | C: hbase-server hbase-mapreduce U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2800/12/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2854: HBASE-25333 Add maven enforcer rule to ban VisibleForTesting imports
Apache-HBase commented on pull request #2854: URL: https://github.com/apache/hbase/pull/2854#issuecomment-755878243 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 19s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | ||| _ Patch Compile Tests _ | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +1 :green_heart: | hadoopcheck | 20m 54s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 18s | The patch does not generate ASF License warnings. | | | | 31m 32s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2854/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2854 | | Optional Tests | dupname asflicense hadoopcheck xml | | uname | Linux b98958fc5b59 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a414361ed9 | | Max. process+thread count | 76 (vs. ulimit of 3) | | modules | C: . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2854/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 opened a new pull request #2854: HBASE-25333 Add maven enforcer rule to ban VisibleForTesting imports
Apache9 opened a new pull request #2854: URL: https://github.com/apache/hbase/pull/2854 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2853: HBASE-25471 Move RegionScannerImpl out of HRegion
Apache-HBase commented on pull request #2853: URL: https://github.com/apache/hbase/pull/2853#issuecomment-755868370 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 30s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 18s | master passed | | +1 :green_heart: | checkstyle | 1m 16s | master passed | | +1 :green_heart: | spotbugs | 2m 11s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 50s | the patch passed | | -0 :warning: | checkstyle | 1m 13s | hbase-server: The patch generated 14 new + 225 unchanged - 16 fixed = 239 total (was 241) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 19m 2s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 21s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. | | | | 43m 41s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2853/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2853 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 62263b91c04c 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a414361ed9 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2853/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 84 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2853/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25333) Add maven enforcer rule to ban VisibleForTesting imports
[ https://issues.apache.org/jira/browse/HBASE-25333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-25333: -- Component/s: pom > Add maven enforcer rule to ban VisibleForTesting imports > > > Key: HBASE-25333 > URL: https://issues.apache.org/jira/browse/HBASE-25333 > Project: HBase > Issue Type: Task > Components: build, pom >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > https://github.com/google/error-prone/blob/master/annotations/src/main/java/com/google/errorprone/annotations/RestrictedApi.java > This is a better choice if you want to restrict the usage of a special API. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25333) Add maven enforcer rule to ban VisibleForTesting imports
[ https://issues.apache.org/jira/browse/HBASE-25333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-25333: -- Description: https://github.com/google/error-prone/blob/master/annotations/src/main/java/com/google/errorprone/annotations/RestrictedApi.java This is a better choice if you want to restrict the usage of a special API. > Add maven enforcer rule to ban VisibleForTesting imports > > > Key: HBASE-25333 > URL: https://issues.apache.org/jira/browse/HBASE-25333 > Project: HBase > Issue Type: Task > Components: build >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > https://github.com/google/error-prone/blob/master/annotations/src/main/java/com/google/errorprone/annotations/RestrictedApi.java > This is a better choice if you want to restrict the usage of a special API. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache9 closed pull request #2714: HBASE-25333 Add a yetus check to disable usage of VisibleForTesting a…
Apache9 closed pull request #2714: URL: https://github.com/apache/hbase/pull/2714 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25333) Add maven enforcer rule to ban VisibleForTesting imports
[ https://issues.apache.org/jira/browse/HBASE-25333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-25333: -- Summary: Add maven enforcer rule to ban VisibleForTesting imports (was: Add a yetus check to disable usage of VisibleForTesting annotation on IA.Public and IA.LimitedPrivate classes) > Add maven enforcer rule to ban VisibleForTesting imports > > > Key: HBASE-25333 > URL: https://issues.apache.org/jira/browse/HBASE-25333 > Project: HBase > Issue Type: Task > Components: build >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache9 commented on pull request #2714: HBASE-25333 Add a yetus check to disable usage of VisibleForTesting a…
Apache9 commented on pull request #2714: URL: https://github.com/apache/hbase/pull/2714#issuecomment-755864895 Oh, after searching, I think this annotation is a better choice as error prone could analysis this at compile time https://github.com/google/error-prone/blob/master/annotations/src/main/java/com/google/errorprone/annotations/RestrictedApi.java So let's just ban the VisibleForTesting imports in maven enforcer. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2848: HBASE-25459 WAL can't be cleaned in some scenes
Apache-HBase commented on pull request #2848: URL: https://github.com/apache/hbase/pull/2848#issuecomment-755861293 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 3s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 11s | master passed | | +1 :green_heart: | checkstyle | 1m 15s | master passed | | +1 :green_heart: | spotbugs | 2m 11s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 48s | the patch passed | | +1 :green_heart: | checkstyle | 1m 10s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 18m 57s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 17s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. | | | | 42m 40s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2848/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2848 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 7636f74ed705 4.15.0-126-generic #129-Ubuntu SMP Mon Nov 23 18:53:38 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a414361ed9 | | Max. process+thread count | 84 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2848/2/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-25472) HBASE-24260 breaks JDK7 compilation in branch-1
Wei-Chiu Chuang created HBASE-25472:
---
Summary: HBASE-24260 breaks JDK7 compilation in branch-1
Key: HBASE-25472
URL: https://issues.apache.org/jira/browse/HBASE-25472
Project: HBase
Issue Type: Bug
Reporter: Wei-Chiu Chuang
Found in PR-2799.
{code}
public static String getZKQuorum(Configuration conf) {
String port =
Integer.toString(conf.getInt(HConstants.ZOOKEEPER_CLIENT_PORT, 2181));
String[] serverHosts = conf.getStrings(HConstants.ZOOKEEPER_QUORUM,
"localhost");
for (int i = 0; i < serverHosts.length; i++) {
serverHosts[i] = serverHosts[i] + ":" + port;
}
return String.join(",", serverHosts);
}
{code}
Fails to compile
{quote}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile (default-compile)
on project hbase-it: Compilation failure
[ERROR]
/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2799/src/hbase-it/src/main/java/org/apache/hadoop/hbase/chaos/ChaosUtils.java:[46,18]
cannot find symbol
[ERROR] symbol: method join(java.lang.String,java.lang.String[])
[ERROR] location: class java.lang.String
{quote}
because String.join() is only available since JDK8.
CC: [~lokiore] [~ndimiduk]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2848: HBASE-25459 WAL can't be cleaned in some scenes
Apache-HBase commented on pull request #2848: URL: https://github.com/apache/hbase/pull/2848#issuecomment-755859059 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 14s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 24s | master passed | | +1 :green_heart: | compile | 1m 8s | master passed | | +1 :green_heart: | shadedjars | 6m 45s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 6s | the patch passed | | +1 :green_heart: | compile | 1m 8s | the patch passed | | +1 :green_heart: | javac | 1m 8s | the patch passed | | +1 :green_heart: | shadedjars | 6m 48s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 44s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 7m 19s | hbase-server in the patch failed. | | | | 35m 24s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2848/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2848 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 4fb6311aae02 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a414361ed9 | | Default Java | AdoptOpenJDK-11.0.6+10 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2848/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2848/2/testReport/ | | Max. process+thread count | 565 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2848/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2848: HBASE-25459 WAL can't be cleaned in some scenes
Apache-HBase commented on pull request #2848: URL: https://github.com/apache/hbase/pull/2848#issuecomment-755858159 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 58s | master passed | | +1 :green_heart: | compile | 1m 0s | master passed | | +1 :green_heart: | shadedjars | 6m 31s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 42s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 30s | the patch passed | | +1 :green_heart: | compile | 0m 58s | the patch passed | | +1 :green_heart: | javac | 0m 58s | the patch passed | | +1 :green_heart: | shadedjars | 6m 30s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 6m 44s | hbase-server in the patch failed. | | | | 32m 13s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2848/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2848 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 5da32eb28657 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a414361ed9 | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2848/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2848/2/testReport/ | | Max. process+thread count | 812 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2848/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 closed pull request #2748: HBASE-25373 Remove HTrace completely in code base and try to make use…
Apache9 closed pull request #2748: URL: https://github.com/apache/hbase/pull/2748 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 opened a new pull request #2853: HBASE-25471 Move RegionScannerImpl out of HRegion
Apache9 opened a new pull request #2853: URL: https://github.com/apache/hbase/pull/2853 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-25471) Move RegionScannerImpl out of HRegion
Duo Zhang created HBASE-25471: - Summary: Move RegionScannerImpl out of HRegion Key: HBASE-25471 URL: https://issues.apache.org/jira/browse/HBASE-25471 Project: HBase Issue Type: Improvement Components: regionserver Reporter: Duo Zhang Assignee: Duo Zhang To make HRegion a little smaller. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25446) public static method MultiTableHFileOutputFormat.configureIncrementalLoad actually cannot be invoked outside hbase package
[ https://issues.apache.org/jira/browse/HBASE-25446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17260187#comment-17260187 ] Anoop Sam John commented on HBASE-25446: Seems no alternative and needs a fix. > public static method MultiTableHFileOutputFormat.configureIncrementalLoad > actually cannot be invoked outside hbase package > -- > > Key: HBASE-25446 > URL: https://issues.apache.org/jira/browse/HBASE-25446 > Project: HBase > Issue Type: Bug > Components: mapreduce >Affects Versions: 2.4.0 >Reporter: Zhou Yuliang >Priority: Major > > *MultiTableHFileOutputFormat* provides a public static method > _configureIncrementalLoad_ to configure reducer for multi table hfile output. > See this. > [https://github.com/apache/hbase/blob/master/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/MultiTableHFileOutputFormat.java#L92] > However the *TableInfo* in the parameter is NOT a public class, which makes > it impossible to invoke > *MultiTableHFileOutputFormat*._configureIncrementalLoad_ in other project. > Is there any alternative for the use case? For now I have to create another > class in +org.apache.hadoop.hbase.mapreduce+ package to invoke the > *MultiTableHFileOutputFormat*._configureIncrementalLoad._ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] leyangyueshan commented on pull request #2848: HBASE-25459 WAL can't be cleaned in some scenes
leyangyueshan commented on pull request #2848: URL: https://github.com/apache/hbase/pull/2848#issuecomment-755851385 > Can we have a UT testing this new behaviour? the UT has updated, pelease review again! @wchevreuil This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] leyangyueshan commented on a change in pull request #2848: HBASE-25459 WAL can't be cleaned in some scenes
leyangyueshan commented on a change in pull request #2848:
URL: https://github.com/apache/hbase/pull/2848#discussion_r553082093
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/SequenceIdAccounting.java
##
@@ -250,7 +250,12 @@ void updateStore(byte[] encodedRegionName, byte[]
familyName, Long sequenceId,
*/
private static long getLowestSequenceId(Map sequenceids) {
long lowest = HConstants.NO_SEQNUM;
-for (Long sid: sequenceids.values()) {
+for (Map.Entry entry : sequenceids.entrySet()){
+ if (entry.getKey() instanceof ImmutableByteArray &&
entry.getKey().toString().equals("METAFAMILY")){
Review comment:
entry.getKey() instanceof ImmutableByteArray is for future, but it's
no need at now , I has deleted it !
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] leyangyueshan commented on a change in pull request #2848: HBASE-25459 WAL can't be cleaned in some scenes
leyangyueshan commented on a change in pull request #2848:
URL: https://github.com/apache/hbase/pull/2848#discussion_r553081166
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/SequenceIdAccounting.java
##
@@ -250,7 +250,12 @@ void updateStore(byte[] encodedRegionName, byte[]
familyName, Long sequenceId,
*/
private static long getLowestSequenceId(Map sequenceids) {
long lowest = HConstants.NO_SEQNUM;
-for (Long sid: sequenceids.values()) {
+for (Map.Entry entry : sequenceids.entrySet()){
+ if (entry.getKey() instanceof ImmutableByteArray &&
entry.getKey().toString().equals("METAFAMILY")){
+LOG.info("ColumnFamily is METAFAMILY! continue ...");
Review comment:
no, I think. delete it at a new commit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2816: HBASE-25449 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml
Apache-HBase commented on pull request #2816: URL: https://github.com/apache/hbase/pull/2816#issuecomment-755849822 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 31s | master passed | | +1 :green_heart: | checkstyle | 0m 25s | master passed | | +0 :ok: | refguide | 3m 33s | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | spotbugs | 0m 44s | master passed | | -0 :warning: | patch | 0m 53s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 28s | the patch passed | | +1 :green_heart: | checkstyle | 0m 26s | the patch passed | | -0 :warning: | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 :green_heart: | xml | 0m 4s | The patch has no ill-formed XML file. | | +0 :ok: | refguide | 3m 22s | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | hadoopcheck | 17m 39s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 0m 51s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. | | | | 42m 6s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2816/5/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2816 | | Optional Tests | dupname asflicense refguide xml spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 3f48d819a18f 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a414361ed9 | | refguide | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2816/5/artifact/yetus-general-check/output/branch-site/book.html | | whitespace | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2816/5/artifact/yetus-general-check/output/whitespace-eol.txt | | refguide | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2816/5/artifact/yetus-general-check/output/patch-site/book.html | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-common U: hbase-common | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2816/5/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25287) Forgetting to unbuffer streams results in many CLOSE_WAIT sockets when loading files
[
https://issues.apache.org/jira/browse/HBASE-25287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17260183#comment-17260183
]
Anoop Sam John commented on HBASE-25287:
You mean separate jira for backport to branch-1? This jira having all branch-2
based versions as fixed version already
> Forgetting to unbuffer streams results in many CLOSE_WAIT sockets when
> loading files
>
>
> Key: HBASE-25287
> URL: https://issues.apache.org/jira/browse/HBASE-25287
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.4.0
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.4, 2.5.0, 2.4.1
>
> Attachments: 1605328358304-image.png, 1605328417888-image.png,
> 1605504914256-image.png
>
>
> HBASE-9393 found seek+read will leave many CLOSE_WAIT sockets without stream
> unbuffer, which can free sockets and file descriptors held by the stream.
> In our cluster RSes with about one hundred thousand store files, we found the
> number of CLOSE_WAIT sockets increases with the number of regions opened,
> and can up to the operating system open files limit 100.
>
> {code:java}
> 2020-11-12 20:19:02,452 WARN [1282990092@qtp-220038608-1 - Acceptor0
> SelectChannelConnector@0.0.0.0:16030] mortbay.log: EXCEPTION
> java.io.IOException: Too many open files
> at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
> at
> sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
> at
> sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
> at
> org.mortbay.jetty.nio.SelectChannelConnector$1.acceptChannel(SelectChannelConnector.java:75)
> at
> org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:686)
> at
> org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:192)
> at
> org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
> at
> org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:708)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> {code}
>
> {code:java}
> [hbase@gha-data-hbase-cat0053 hbase]$ ulimit -SHn
> 100
> {code}
>
>
> The reason of the problem is, when store file opened,
> {code:java}
> private void open() throws IOException {
> fileInfo.initHDFSBlocksDistribution();
> long readahead = fileInfo.isNoReadahead() ? 0L : -1L;
> ReaderContext context = fileInfo.createReaderContext(false, readahead,
> ReaderType.PREAD);
> fileInfo.initHFileInfo(context);
> StoreFileReader reader = fileInfo.preStoreFileReaderOpen(context,
> cacheConf);
> if (reader == null) {
> reader = fileInfo.createReader(context, cacheConf);
> fileInfo.getHFileInfo().initMetaAndIndex(reader.getHFileReader());
> }
> {code}
> only createReader() unbuffered the stream. In initMetaAndIndex(), using the
> stream to read blocks, so it needs to unbuffer() the socket , too.
> We can just add try before fileInfo.initHFileInfo(context); and finally
> unbuffer() the stream at the end of the open() function.
> We fixed it on our cluster, the number of CLOSE_WAIT reduced to about 0.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25445) Old WALs archive fails in procedure based WAL split
[ https://issues.apache.org/jira/browse/HBASE-25445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-25445: --- Summary: Old WALs archive fails in procedure based WAL split (was: Old WALs archive fails in procedure based WAL) > Old WALs archive fails in procedure based WAL split > --- > > Key: HBASE-25445 > URL: https://issues.apache.org/jira/browse/HBASE-25445 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 3.0.0-alpha-1, 2.4.0, 2.2.6, 2.3.2 >Reporter: mokai >Assignee: Anjan Das >Priority: Critical > Fix For: 3.0.0-alpha-1, 2.2.7, 2.3.4, 2.5.0, 2.4.1 > > Attachments: ServerCrashWrongFSError.png > > > If 'hbase.wal.dir' and 'hbase.rootdir' are configured to different > filesystem, SplitWALRemoteProcedure archived split WAL failed since > SplitWALManager using wrong fs instance. SplitWALManager should use WAL > corresponding fs instance. > Steps to Reproduce: > * Configure 'hbase.wal.dir' and 'hbase.rootdir' so that they point to > different fs instances. > * Start HBase with multiple RS. > * Create a couple of tables and some rows in them so that the RSs get > assigned with some regions. > * Take any RS with non-zero number of regions offline. > * Check master logs for "Wrong FS" error as shown in the screenshot > attached. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25445) Old WALs archive fails in procedure based WAL
[ https://issues.apache.org/jira/browse/HBASE-25445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-25445: --- Summary: Old WALs archive fails in procedure based WAL (was: SplitWALRemoteProcedure failed to archive split WAL) > Old WALs archive fails in procedure based WAL > - > > Key: HBASE-25445 > URL: https://issues.apache.org/jira/browse/HBASE-25445 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 3.0.0-alpha-1, 2.4.0, 2.2.6, 2.3.2 >Reporter: mokai >Assignee: Anjan Das >Priority: Critical > Fix For: 3.0.0-alpha-1, 2.2.7, 2.3.4, 2.5.0, 2.4.1 > > Attachments: ServerCrashWrongFSError.png > > > If 'hbase.wal.dir' and 'hbase.rootdir' are configured to different > filesystem, SplitWALRemoteProcedure archived split WAL failed since > SplitWALManager using wrong fs instance. SplitWALManager should use WAL > corresponding fs instance. > Steps to Reproduce: > * Configure 'hbase.wal.dir' and 'hbase.rootdir' so that they point to > different fs instances. > * Start HBase with multiple RS. > * Create a couple of tables and some rows in them so that the RSs get > assigned with some regions. > * Take any RS with non-zero number of regions offline. > * Check master logs for "Wrong FS" error as shown in the screenshot > attached. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2816: HBASE-25449 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml
Apache-HBase commented on pull request #2816: URL: https://github.com/apache/hbase/pull/2816#issuecomment-755846474 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 39s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 3s | master passed | | +1 :green_heart: | compile | 0m 25s | master passed | | +1 :green_heart: | shadedjars | 7m 26s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 20s | master passed | | -0 :warning: | patch | 7m 55s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 10s | the patch passed | | +1 :green_heart: | compile | 0m 25s | the patch passed | | +1 :green_heart: | javac | 0m 25s | the patch passed | | +1 :green_heart: | shadedjars | 7m 52s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 24s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 6s | hbase-common in the patch passed. | | | | 29m 53s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2816/5/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2816 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux e0f601514acd 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a414361ed9 | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2816/5/testReport/ | | Max. process+thread count | 221 (vs. ulimit of 3) | | modules | C: hbase-common U: hbase-common | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2816/5/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2816: HBASE-25449 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml
Apache-HBase commented on pull request #2816: URL: https://github.com/apache/hbase/pull/2816#issuecomment-755845788 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 33s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 30s | master passed | | +1 :green_heart: | compile | 0m 28s | master passed | | +1 :green_heart: | shadedjars | 6m 49s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 26s | master passed | | -0 :warning: | patch | 7m 27s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 10s | the patch passed | | +1 :green_heart: | compile | 0m 26s | the patch passed | | +1 :green_heart: | javac | 0m 26s | the patch passed | | +1 :green_heart: | shadedjars | 6m 39s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 24s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 46s | hbase-common in the patch passed. | | | | 27m 16s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2816/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2816 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 35b088b166d5 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a414361ed9 | | Default Java | AdoptOpenJDK-11.0.6+10 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2816/5/testReport/ | | Max. process+thread count | 292 (vs. ulimit of 3) | | modules | C: hbase-common U: hbase-common | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2816/5/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24755) [LOG][RSGroup]Error message is confusing while adding a offline RS to rsgroup
[
https://issues.apache.org/jira/browse/HBASE-24755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17260179#comment-17260179
]
Hudson commented on HBASE-24755:
Results for branch branch-2.4
[build #24 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/24/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/24/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/24/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/24/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/24/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [LOG][RSGroup]Error message is confusing while adding a offline RS to rsgroup
> -
>
> Key: HBASE-24755
> URL: https://issues.apache.org/jira/browse/HBASE-24755
> Project: HBase
> Issue Type: Bug
> Components: rsgroup
>Affects Versions: 2.2.3
>Reporter: mokai
>Assignee: mokai
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.2.7, 2.3.4, 2.5.0, 2.4.1
>
> Attachments: image-2020-07-22-19-34-42-628.png
>
>
> Move a offine RS from default RSGroup to another RSGroup. The error messge
> says 'Source RSGroup for server xxx does not exist.'. It's confusing.
> !image-2020-07-22-19-34-42-628.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25458) HRegion methods cleanup
[ https://issues.apache.org/jira/browse/HBASE-25458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-25458: -- Fix Version/s: 2.5.0 3.0.0-alpha-1 > HRegion methods cleanup > --- > > Key: HBASE-25458 > URL: https://issues.apache.org/jira/browse/HBASE-25458 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.5.0 > > > Narrow down the public interface to make easier for tracing. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25458) HRegion methods cleanup
[ https://issues.apache.org/jira/browse/HBASE-25458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-25458. --- Hadoop Flags: Reviewed Resolution: Fixed Pushed to master and branch-2. Thanks [~meiyi] for reviewing. > HRegion methods cleanup > --- > > Key: HBASE-25458 > URL: https://issues.apache.org/jira/browse/HBASE-25458 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.5.0 > > > Narrow down the public interface to make easier for tracing. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2816: HBASE-25449 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml
Apache-HBase commented on pull request #2816: URL: https://github.com/apache/hbase/pull/2816#issuecomment-755837267 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 37s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 50s | master passed | | +1 :green_heart: | checkstyle | 0m 25s | master passed | | +0 :ok: | refguide | 3m 29s | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | spotbugs | 0m 44s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 26s | the patch passed | | +1 :green_heart: | checkstyle | 0m 24s | the patch passed | | -0 :warning: | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 :green_heart: | xml | 0m 4s | The patch has no ill-formed XML file. | | +0 :ok: | refguide | 3m 26s | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | hadoopcheck | 17m 21s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 0m 52s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. | | | | 43m 17s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2816/4/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2816 | | Optional Tests | dupname asflicense refguide xml spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 0fb6318a2e57 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a414361ed9 | | refguide | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2816/4/artifact/yetus-general-check/output/branch-site/book.html | | whitespace | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2816/4/artifact/yetus-general-check/output/whitespace-eol.txt | | refguide | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2816/4/artifact/yetus-general-check/output/patch-site/book.html | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-common U: hbase-common | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2816/4/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] joshelser commented on a change in pull request #2816: HBASE-25449 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml
joshelser commented on a change in pull request #2816: URL: https://github.com/apache/hbase/pull/2816#discussion_r553067037 ## File path: hbase-common/src/main/resources/hbase-default.xml ## @@ -1463,15 +1463,15 @@ possible configurations would overwhelm and obscure the important. dfs.client.read.shortcircuit -false + If set to true, this configuration parameter enables short-circuit local reads. dfs.domain.socket.path -none + Review comment: I don't think this works either. I think we need to not have this property at all. I think this empty config property would still mask the real one, right? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-25342) Upgrade error prone to 2.4.0
[ https://issues.apache.org/jira/browse/HBASE-25342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-25342. --- Hadoop Flags: Reviewed Resolution: Fixed Thanks [~psomogyi]. Let's resolve it. But maybe we missed this issue in the CHANGES.md for 2.4.0? > Upgrade error prone to 2.4.0 > > > Key: HBASE-25342 > URL: https://issues.apache.org/jira/browse/HBASE-25342 > Project: HBase > Issue Type: Task > Components: build >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.2.7, 2.3.4, 2.4.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2816: HBASE-25449 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml
Apache-HBase commented on pull request #2816: URL: https://github.com/apache/hbase/pull/2816#issuecomment-755834324 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 59s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 1s | master passed | | +1 :green_heart: | compile | 0m 26s | master passed | | +1 :green_heart: | shadedjars | 7m 55s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 24s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 38s | the patch passed | | +1 :green_heart: | compile | 0m 32s | the patch passed | | +1 :green_heart: | javac | 0m 32s | the patch passed | | +1 :green_heart: | shadedjars | 8m 25s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 23s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 2s | hbase-common in the patch passed. | | | | 32m 59s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2816/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2816 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux e90c878bb051 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a414361ed9 | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2816/4/testReport/ | | Max. process+thread count | 221 (vs. ulimit of 3) | | modules | C: hbase-common U: hbase-common | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2816/4/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25342) Upgrade error prone to 2.4.0
[ https://issues.apache.org/jira/browse/HBASE-25342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-25342: -- Fix Version/s: 2.4.0 2.3.4 2.2.7 3.0.0-alpha-1 > Upgrade error prone to 2.4.0 > > > Key: HBASE-25342 > URL: https://issues.apache.org/jira/browse/HBASE-25342 > Project: HBase > Issue Type: Task > Components: build >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0, 2.2.7, 2.3.4 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2816: HBASE-25449 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml
Apache-HBase commented on pull request #2816: URL: https://github.com/apache/hbase/pull/2816#issuecomment-755832554 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 22s | master passed | | +1 :green_heart: | compile | 0m 25s | master passed | | +1 :green_heart: | shadedjars | 6m 38s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 26s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 8s | the patch passed | | +1 :green_heart: | compile | 0m 26s | the patch passed | | +1 :green_heart: | javac | 0m 26s | the patch passed | | +1 :green_heart: | shadedjars | 6m 41s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 24s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 43s | hbase-common in the patch passed. | | | | 26m 57s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2816/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2816 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 8916b8156623 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a414361ed9 | | Default Java | AdoptOpenJDK-11.0.6+10 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2816/4/testReport/ | | Max. process+thread count | 268 (vs. ulimit of 3) | | modules | C: hbase-common U: hbase-common | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2816/4/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] shenshengli commented on a change in pull request #2816: HBASE-25449 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml
shenshengli commented on a change in pull request #2816:
URL: https://github.com/apache/hbase/pull/2816#discussion_r553053984
##
File path:
hbase-common/src/test/java/org/apache/hadoop/hbase/TestHBaseConfiguration.java
##
@@ -115,6 +115,19 @@ public void testSecurityConfCaseInsensitive() {
conf.set("hbase.security.authentication", "KERBeros");
Assert.assertTrue(User.isHBaseSecurityEnabled(conf));
}
+
+ @Test
+ public void testGetConfigOfShortcircuitRead() throws Exception {
+Configuration conf = HBaseConfiguration.create();
+Configuration.addDefaultResource("hdfs-default.xml");
+assertEquals("hdfs-default.xml",
+conf.getPropertySources("dfs.client.read.shortcircuit")[0]);
+assertEquals("false", conf.get("dfs.client.read.shortcircuit"));
+Configuration.addDefaultResource("hdfs-site.xml");
Review comment:
OK, I just want to simulate setting this value explicitly.Changing the
name would work, too.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] shenshengli commented on a change in pull request #2816: HBASE-25449 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml
shenshengli commented on a change in pull request #2816: URL: https://github.com/apache/hbase/pull/2816#discussion_r553052420 ## File path: hbase-common/src/main/resources/hbase-default.xml ## @@ -1463,7 +1463,7 @@ possible configurations would overwhelm and obscure the important. dfs.client.read.shortcircuit -false + Review comment: Yes, I ignored this because it exists in my hbase-site.xml . I'm going to change that. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25445) SplitWALRemoteProcedure failed to archive split WAL
[ https://issues.apache.org/jira/browse/HBASE-25445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17260136#comment-17260136 ] Huaxiang Sun commented on HBASE-25445: -- We are going to spin the RC tomorrow, hopefully, it will be merged to 2.3 by tomorrow, thanks. > SplitWALRemoteProcedure failed to archive split WAL > --- > > Key: HBASE-25445 > URL: https://issues.apache.org/jira/browse/HBASE-25445 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 3.0.0-alpha-1, 2.4.0, 2.2.6, 2.3.2 >Reporter: mokai >Assignee: Anjan Das >Priority: Critical > Fix For: 3.0.0-alpha-1, 2.2.7, 2.3.4, 2.5.0, 2.4.1 > > Attachments: ServerCrashWrongFSError.png > > > If 'hbase.wal.dir' and 'hbase.rootdir' are configured to different > filesystem, SplitWALRemoteProcedure archived split WAL failed since > SplitWALManager using wrong fs instance. SplitWALManager should use WAL > corresponding fs instance. > Steps to Reproduce: > * Configure 'hbase.wal.dir' and 'hbase.rootdir' so that they point to > different fs instances. > * Start HBase with multiple RS. > * Create a couple of tables and some rows in them so that the RSs get > assigned with some regions. > * Take any RS with non-zero number of regions offline. > * Check master logs for "Wrong FS" error as shown in the screenshot > attached. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25445) SplitWALRemoteProcedure failed to archive split WAL
[ https://issues.apache.org/jira/browse/HBASE-25445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17260135#comment-17260135 ] Huaxiang Sun commented on HBASE-25445: -- I created HBASE-25470 as a followup for unitest case. Please merge and backport this Jira to all affected branches, thanks. > SplitWALRemoteProcedure failed to archive split WAL > --- > > Key: HBASE-25445 > URL: https://issues.apache.org/jira/browse/HBASE-25445 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 3.0.0-alpha-1, 2.4.0, 2.2.6, 2.3.2 >Reporter: mokai >Assignee: Anjan Das >Priority: Critical > Fix For: 3.0.0-alpha-1, 2.2.7, 2.3.4, 2.5.0, 2.4.1 > > Attachments: ServerCrashWrongFSError.png > > > If 'hbase.wal.dir' and 'hbase.rootdir' are configured to different > filesystem, SplitWALRemoteProcedure archived split WAL failed since > SplitWALManager using wrong fs instance. SplitWALManager should use WAL > corresponding fs instance. > Steps to Reproduce: > * Configure 'hbase.wal.dir' and 'hbase.rootdir' so that they point to > different fs instances. > * Start HBase with multiple RS. > * Create a couple of tables and some rows in them so that the RSs get > assigned with some regions. > * Take any RS with non-zero number of regions offline. > * Check master logs for "Wrong FS" error as shown in the screenshot > attached. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25470) Add unitest for HBASE-25445 - SplitWALRemoteProcedure failed to archive split WAL
[ https://issues.apache.org/jira/browse/HBASE-25470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxiang Sun updated HBASE-25470: - Fix Version/s: (was: 2.3.4) > Add unitest for HBASE-25445 - SplitWALRemoteProcedure failed to archive split > WAL > - > > Key: HBASE-25470 > URL: https://issues.apache.org/jira/browse/HBASE-25470 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 3.0.0-alpha-1, 2.4.0, 2.2.6, 2.3.2 >Reporter: mokai >Assignee: Anjan Das >Priority: Major > Fix For: 3.0.0-alpha-1, 2.2.7, 2.5.0, 2.4.1 > > > If 'hbase.wal.dir' and 'hbase.rootdir' are configured to different > filesystem, SplitWALRemoteProcedure archived split WAL failed since > SplitWALManager using wrong fs instance. SplitWALManager should use WAL > corresponding fs instance. > Steps to Reproduce: > * Configure 'hbase.wal.dir' and 'hbase.rootdir' so that they point to > different fs instances. > * Start HBase with multiple RS. > * Create a couple of tables and some rows in them so that the RSs get > assigned with some regions. > * Take any RS with non-zero number of regions offline. > * Check master logs for "Wrong FS" error as shown in the screenshot > attached. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25470) Add unitest for HBASE-25445 - SplitWALRemoteProcedure failed to archive split WAL
[ https://issues.apache.org/jira/browse/HBASE-25470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxiang Sun updated HBASE-25470: - Priority: Major (was: Critical) > Add unitest for HBASE-25445 - SplitWALRemoteProcedure failed to archive split > WAL > - > > Key: HBASE-25470 > URL: https://issues.apache.org/jira/browse/HBASE-25470 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 3.0.0-alpha-1, 2.4.0, 2.2.6, 2.3.2 >Reporter: mokai >Assignee: Anjan Das >Priority: Major > Fix For: 3.0.0-alpha-1, 2.2.7, 2.3.4, 2.5.0, 2.4.1 > > > If 'hbase.wal.dir' and 'hbase.rootdir' are configured to different > filesystem, SplitWALRemoteProcedure archived split WAL failed since > SplitWALManager using wrong fs instance. SplitWALManager should use WAL > corresponding fs instance. > Steps to Reproduce: > * Configure 'hbase.wal.dir' and 'hbase.rootdir' so that they point to > different fs instances. > * Start HBase with multiple RS. > * Create a couple of tables and some rows in them so that the RSs get > assigned with some regions. > * Take any RS with non-zero number of regions offline. > * Check master logs for "Wrong FS" error as shown in the screenshot > attached. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25470) Add unitest for HBASE-25445 - SplitWALRemoteProcedure failed to archive split WAL
Huaxiang Sun created HBASE-25470: Summary: Add unitest for HBASE-25445 - SplitWALRemoteProcedure failed to archive split WAL Key: HBASE-25470 URL: https://issues.apache.org/jira/browse/HBASE-25470 Project: HBase Issue Type: Bug Components: wal Affects Versions: 3.0.0-alpha-1, 2.4.0, 2.2.6, 2.3.2 Reporter: mokai Assignee: Anjan Das Fix For: 3.0.0-alpha-1, 2.2.7, 2.3.4, 2.5.0, 2.4.1 If 'hbase.wal.dir' and 'hbase.rootdir' are configured to different filesystem, SplitWALRemoteProcedure archived split WAL failed since SplitWALManager using wrong fs instance. SplitWALManager should use WAL corresponding fs instance. Steps to Reproduce: * Configure 'hbase.wal.dir' and 'hbase.rootdir' so that they point to different fs instances. * Start HBase with multiple RS. * Create a couple of tables and some rows in them so that the RSs get assigned with some regions. * Take any RS with non-zero number of regions offline. * Check master logs for "Wrong FS" error as shown in the screenshot attached. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HBASE-25329) Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
[ https://issues.apache.org/jira/browse/HBASE-25329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17260132#comment-17260132 ] Caroline edited comment on HBASE-25329 at 1/7/21, 12:47 AM: [~apurtell] [~vjasani] [~bharathv] Thanks for the feedback. I'll remove the metrics additions from the patches since this particular Jira concerns dumping RITs into the logs. Do you have any suggestions on how often we should be logging the RITs? For the idea of adding a /rit endpoint in master, I've created a Jira (https://issues.apache.org/jira/browse/HBASE-25469) to track it. Would love to hear suggestions about what info to expose on the endpoint. If someone could provide a servlet code example that would also be helpful, since I'm unfamiliar with them in this code base. Thanks! was (Author: caroliney14): [~apurtell] [~vjasani] [~bharathv] Thanks for the feedback. I'll remove the metrics additions from the patches since this particular Jira concerns dumping RITs into the logs. Do you have any suggestions on how often we should be logging the RITs? For the idea of adding a /rit endpoint in master, I've created a Jira to track it. Would love to hear suggestions about what info to expose on the endpoint. If someone could provide a servlet code example that would also be helpful, since I'm unfamiliar with them in this code base. Thanks! > Dump region hashes in logs for the regions that are stuck in transition for > more than a configured amount of time > - > > Key: HBASE-25329 > URL: https://issues.apache.org/jira/browse/HBASE-25329 > Project: HBase > Issue Type: Improvement >Reporter: Caroline >Assignee: Caroline >Priority: Minor > Attachments: HBASE-25329.branch-1.000.patch, > HBASE-25329.branch-2.000.patch, HBASE-25329.master.000.patch > > > We have metrics for number of RITs as well as number of RITs above a certain > threshold, but we don't have any way of keeping track of the region hashes of > those RITs. It would be beneficial to emit those region hashes as a metric, > as well as log them, so that we don't accidentally lose this information for > debugging the RIT at a later tiime. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25329) Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
[ https://issues.apache.org/jira/browse/HBASE-25329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17260132#comment-17260132 ] Caroline commented on HBASE-25329: -- [~apurtell] [~vjasani] [~bharathv] Thanks for the feedback. I'll remove the metrics additions from the patches since this particular Jira concerns dumping RITs into the logs. Do you have any suggestions on how often we should be logging the RITs? For the idea of adding a /rit endpoint in master, I've created a Jira to track it. Would love to hear suggestions about what info to expose on the endpoint. If someone could provide a servlet code example that would also be helpful, since I'm unfamiliar with them in this code base. Thanks! > Dump region hashes in logs for the regions that are stuck in transition for > more than a configured amount of time > - > > Key: HBASE-25329 > URL: https://issues.apache.org/jira/browse/HBASE-25329 > Project: HBase > Issue Type: Improvement >Reporter: Caroline >Assignee: Caroline >Priority: Minor > Attachments: HBASE-25329.branch-1.000.patch, > HBASE-25329.branch-2.000.patch, HBASE-25329.master.000.patch > > > We have metrics for number of RITs as well as number of RITs above a certain > threshold, but we don't have any way of keeping track of the region hashes of > those RITs. It would be beneficial to emit those region hashes as a metric, > as well as log them, so that we don't accidentally lose this information for > debugging the RIT at a later tiime. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25445) SplitWALRemoteProcedure failed to archive split WAL
[ https://issues.apache.org/jira/browse/HBASE-25445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17260133#comment-17260133 ] Huaxiang Sun commented on HBASE-25445: -- I checked the diff, it is straightforward. Can we create a new Jira as a followup to add a unitest and merge this diff? This will unblock both sides, thanks. > SplitWALRemoteProcedure failed to archive split WAL > --- > > Key: HBASE-25445 > URL: https://issues.apache.org/jira/browse/HBASE-25445 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 3.0.0-alpha-1, 2.4.0, 2.2.6, 2.3.2 >Reporter: mokai >Assignee: Anjan Das >Priority: Critical > Fix For: 3.0.0-alpha-1, 2.2.7, 2.3.4, 2.5.0, 2.4.1 > > Attachments: ServerCrashWrongFSError.png > > > If 'hbase.wal.dir' and 'hbase.rootdir' are configured to different > filesystem, SplitWALRemoteProcedure archived split WAL failed since > SplitWALManager using wrong fs instance. SplitWALManager should use WAL > corresponding fs instance. > Steps to Reproduce: > * Configure 'hbase.wal.dir' and 'hbase.rootdir' so that they point to > different fs instances. > * Start HBase with multiple RS. > * Create a couple of tables and some rows in them so that the RSs get > assigned with some regions. > * Take any RS with non-zero number of regions offline. > * Check master logs for "Wrong FS" error as shown in the screenshot > attached. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HBASE-25329) Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
[ https://issues.apache.org/jira/browse/HBASE-25329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17260132#comment-17260132 ] Caroline edited comment on HBASE-25329 at 1/7/21, 12:46 AM: [~apurtell] [~vjasani] [~bharathv] Thanks for the feedback. I'll remove the metrics additions from the patches since this particular Jira concerns dumping RITs into the logs. Do you have any suggestions on how often we should be logging the RITs? For the idea of adding a /rit endpoint in master, I've created a Jira to track it. Would love to hear suggestions about what info to expose on the endpoint. If someone could provide a servlet code example that would also be helpful, since I'm unfamiliar with them in this code base. Thanks! was (Author: caroliney14): [~apurtell] [~vjasani] [~bharathv] Thanks for the feedback. I'll remove the metrics additions from the patches since this particular Jira concerns dumping RITs into the logs. Do you have any suggestions on how often we should be logging the RITs? For the idea of adding a /rit endpoint in master, I've created a Jira to track it. Would love to hear suggestions about what info to expose on the endpoint. If someone could provide a servlet code example that would also be helpful, since I'm unfamiliar with them in this code base. Thanks! > Dump region hashes in logs for the regions that are stuck in transition for > more than a configured amount of time > - > > Key: HBASE-25329 > URL: https://issues.apache.org/jira/browse/HBASE-25329 > Project: HBase > Issue Type: Improvement >Reporter: Caroline >Assignee: Caroline >Priority: Minor > Attachments: HBASE-25329.branch-1.000.patch, > HBASE-25329.branch-2.000.patch, HBASE-25329.master.000.patch > > > We have metrics for number of RITs as well as number of RITs above a certain > threshold, but we don't have any way of keeping track of the region hashes of > those RITs. It would be beneficial to emit those region hashes as a metric, > as well as log them, so that we don't accidentally lose this information for > debugging the RIT at a later tiime. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25469) Create RIT servlet in HMaster to track more detailed RIT info not captured in metrics
Caroline created HBASE-25469: Summary: Create RIT servlet in HMaster to track more detailed RIT info not captured in metrics Key: HBASE-25469 URL: https://issues.apache.org/jira/browse/HBASE-25469 Project: HBase Issue Type: Improvement Reporter: Caroline -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work started] (HBASE-25468) Update git-jira-audit fallback_actions file with recent exceptions
[ https://issues.apache.org/jira/browse/HBASE-25468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-25468 started by Nick Dimiduk. > Update git-jira-audit fallback_actions file with recent exceptions > -- > > Key: HBASE-25468 > URL: https://issues.apache.org/jira/browse/HBASE-25468 > Project: HBase > Issue Type: Task > Components: community >Affects Versions: 3.0.0-alpha-1 >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 3.0.0-alpha-1 > > > I recently ran the git/Jira audit tool ; we have some new ill-formed commits > to handle manually. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2852: HBASE-25468 Update git-jira-audit fallback_actions file with recent exceptions
Apache-HBase commented on pull request #2852: URL: https://github.com/apache/hbase/pull/2852#issuecomment-755767113 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 43s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 7s | Maven dependency ordering for patch | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | ||| _ Other Tests _ | | +0 :ok: | asflicense | 0m 0s | ASF License check generated no output? | | | | 3m 8s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2852/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2852 | | Optional Tests | dupname asflicense | | uname | Linux aa31577a69de 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a414361ed9 | | Max. process+thread count | 44 (vs. ulimit of 3) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2852/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2852: HBASE-25468 Update git-jira-audit fallback_actions file with recent exceptions
Apache-HBase commented on pull request #2852: URL: https://github.com/apache/hbase/pull/2852#issuecomment-755766860 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 5s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 7s | Maven dependency ordering for patch | ||| _ Other Tests _ | | | | 2m 30s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2852/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2852 | | Optional Tests | | | uname | Linux 1d16dd22579a 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a414361ed9 | | Max. process+thread count | 47 (vs. ulimit of 3) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2852/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2852: HBASE-25468 Update git-jira-audit fallback_actions file with recent exceptions
Apache-HBase commented on pull request #2852: URL: https://github.com/apache/hbase/pull/2852#issuecomment-755766639 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 24s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 8s | Maven dependency ordering for patch | ||| _ Other Tests _ | | | | 1m 55s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2852/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2852 | | Optional Tests | | | uname | Linux d32b1c2bbb42 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a414361ed9 | | Max. process+thread count | 55 (vs. ulimit of 3) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2852/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk opened a new pull request #2852: HBASE-25468 Update git-jira-audit fallback_actions file with recent exceptions
ndimiduk opened a new pull request #2852: URL: https://github.com/apache/hbase/pull/2852 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-25468) Update git-jira-audit fallback_actions file with recent exceptions
Nick Dimiduk created HBASE-25468: Summary: Update git-jira-audit fallback_actions file with recent exceptions Key: HBASE-25468 URL: https://issues.apache.org/jira/browse/HBASE-25468 Project: HBase Issue Type: Task Components: community Affects Versions: 3.0.0-alpha-1 Reporter: Nick Dimiduk Assignee: Nick Dimiduk Fix For: 3.0.0-alpha-1 I recently ran the git/Jira audit tool ; we have some new ill-formed commits to handle manually. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2800: HBASE-25249 Adding StoreContext
Apache-HBase commented on pull request #2800: URL: https://github.com/apache/hbase/pull/2800#issuecomment-755761858 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 0s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 20s | master passed | | +1 :green_heart: | compile | 1m 23s | master passed | | +1 :green_heart: | shadedjars | 7m 7s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 56s | master passed | | -0 :warning: | patch | 8m 25s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 38s | the patch passed | | +1 :green_heart: | compile | 2m 3s | the patch passed | | +1 :green_heart: | javac | 2m 3s | the patch passed | | +1 :green_heart: | shadedjars | 7m 16s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 54s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 210m 15s | hbase-server in the patch failed. | | +1 :green_heart: | unit | 13m 30s | hbase-mapreduce in the patch passed. | | | | 256m 54s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2800/11/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2800 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux ec977a29957b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a414361ed9 | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2800/11/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2800/11/testReport/ | | Max. process+thread count | 3735 (vs. ulimit of 3) | | modules | C: hbase-server hbase-mapreduce U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2800/11/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] taklwu commented on a change in pull request #2800: HBASE-25249 Adding StoreContext
taklwu commented on a change in pull request #2800:
URL: https://github.com/apache/hbase/pull/2800#discussion_r552986081
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreContext.java
##
@@ -0,0 +1,194 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.regionserver;
+
+import java.net.InetSocketAddress;
+import java.util.Collection;
+import java.util.function.Supplier;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.CellComparator;
+import org.apache.hadoop.hbase.client.ColumnFamilyDescriptor;
+import org.apache.hadoop.hbase.io.HeapSize;
+import org.apache.hadoop.hbase.io.crypto.Encryption;
+import org.apache.hadoop.hbase.io.hfile.CacheConfig;
+import org.apache.hadoop.hbase.util.ClassSize;
+import org.apache.yetus.audience.InterfaceAudience;
+
+/**
+ * This carries the immutable information and references on some of the meta
data about the HStore.
+ * This meta data can be used across the HFileWriter/Readers and other HStore
consumers without the
+ * need of passing around the complete store.
+ */
+@InterfaceAudience.Private
+public final class StoreContext implements HeapSize {
+ public static final long FIXED_OVERHEAD =
ClassSize.estimateBase(HStore.class, false);
+
+ private final int blockSize;
+ private final Encryption.Context encryptionContext;
+ private final CacheConfig cacheConf;
+ private final HRegionFileSystem regionFileSystem;
+ private final CellComparator comparator;
+ private final BloomType bloomFilterType;
+ private final Supplier> compactedFilesSupplier;
+ private final Supplier favoredNodesSupplier;
+ private final ColumnFamilyDescriptor family;
+ private final Path familyStoreDirectoryPath;
+ private final RegionCoprocessorHost coprocessorHost;
+
+ private StoreContext(Builder builder) {
+this.blockSize = builder.blockSize;
+this.encryptionContext = builder.encryptionContext;
+this.cacheConf = builder.cacheConf;
+this.regionFileSystem = builder.regionFileSystem;
+this.comparator = builder.comparator;
+this.bloomFilterType = builder.bloomFilterType;
+this.compactedFilesSupplier = builder.compactedFilesSupplier;
+this.favoredNodesSupplier = builder.favoredNodesSupplier;
+this.family = builder.family;
+this.familyStoreDirectoryPath = builder.familyStoreDirectoryPath;
+this.coprocessorHost = builder.coprocessorHost;
+ }
+
+ public int getBlockSize() {
+return blockSize;
+ }
+
+ public Encryption.Context getEncryptionContext() {
+return encryptionContext;
+ }
+
+ public CacheConfig getCacheConf() {
+return cacheConf;
+ }
+
+ public HRegionFileSystem getRegionFileSystem() {
+return regionFileSystem;
+ }
+
+ public CellComparator getComparator() {
+return comparator;
+ }
+
+ public BloomType getBloomFilterType() {
+return bloomFilterType;
+ }
+
+ public Supplier> getCompactedFilesSupplier() {
+return compactedFilesSupplier;
+ }
+
+ public InetSocketAddress[] getFavoredNodes() {
+return favoredNodesSupplier.get();
+ }
+
+ public ColumnFamilyDescriptor getFamily() {
+return family;
+ }
+
+ public Path getFamilyStoreDirectoryPath() {
+return familyStoreDirectoryPath;
+ }
+
+ public RegionCoprocessorHost getCoprocessorHost() {
+return coprocessorHost;
+ }
+
+ public static Builder getBuilder() {
+return new Builder();
+ }
+
+ @Override
+ public long heapSize() {
+return FIXED_OVERHEAD;
+ }
Review comment:
@apurtell ping again on the `heapSize()` and implements `HeapSize` for
StoreContext class, could you have another look? I should have fixed it, but
will wait for your approval/resolve in 1 day or 2 day before merging it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] ndimiduk commented on a change in pull request #2754: HBASE-25375 Provide a VM-based release environment
ndimiduk commented on a change in pull request #2754: URL: https://github.com/apache/hbase/pull/2754#discussion_r552984966 ## File path: dev-support/release-vm/README.md ## @@ -0,0 +1,141 @@ + + +# HBase Release Env + +This is a vagrant project that provides a virtual machine environment suitable +for running an Apache HBase release. + +Requires: +* [VirtualBox](http://virtualbox.org) +* [Vagrant](http://virtualbox.org) +* The private portion of your signing key avilable in the local GPG agent +* The private portion of your Github authentication key available in either the local GPG agent or + local SSH agent + +## Usage + +Unlock the local keyring before proceeding (this should prompt you for your GPG passphrase). For +example, assuming you have an authentication key configured in your keyring, this will do the +trick. + +All terminal commands used below are assumed to be run with the current working directory as the +location containing the `Vagrantfile`. + +The term "Host" is used to mean the environment that runs the Vagrant process. The term "Guest" is +used to mean the virtual machine managed by the Host. + +### Ensure credentials work from the Host OS + +The ssh- and gpg-agent forwarding configuration used here assumes that your credentials work +on the Host. Verify both are working before you proceed with the Guest. Additionally, using the +credentials requires you to unlock the respective keyring, the state of which is persisted by the +agent process or processes running on the Host. + +See instructions in [`create-release`](../create-release/README.txt) regarding proper +configuration of ssh- and gpg-agents. + +Assuming the git repo origin is on GitHub, the following command will ensure that your ssh +credentials are working. On the Host, run: + +```sh +host:~$ ssh -T g...@github.com +Hi ! You've successfully authenticated, but GitHub does not provide shell access. +``` + +Likewise, ensure you have an encryption key that can be used to sign a file. Again, on the Host, +run: Review comment: No, not down here. See the second paragraph added up above under the "Usage" heading. I wrote it that way so that if this readme gets moved around later, the anchor of reference holds true. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2800: HBASE-25249 Adding StoreContext
Apache-HBase commented on pull request #2800: URL: https://github.com/apache/hbase/pull/2800#issuecomment-755726798 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 49s | master passed | | +1 :green_heart: | compile | 1m 47s | master passed | | +1 :green_heart: | shadedjars | 8m 15s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 11s | master passed | | -0 :warning: | patch | 9m 48s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 56s | the patch passed | | +1 :green_heart: | compile | 1m 55s | the patch passed | | +1 :green_heart: | javac | 1m 55s | the patch passed | | +1 :green_heart: | shadedjars | 8m 24s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 15s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 139m 10s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 10m 29s | hbase-mapreduce in the patch passed. | | | | 185m 42s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2800/11/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2800 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 8eaafe54df56 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a414361ed9 | | Default Java | AdoptOpenJDK-11.0.6+10 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2800/11/testReport/ | | Max. process+thread count | 4067 (vs. ulimit of 3) | | modules | C: hbase-server hbase-mapreduce U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2800/11/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25405) Backport HBASE-25287 to branch-1
[ https://issues.apache.org/jira/browse/HBASE-25405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HBASE-25405: Fix Version/s: 1.7.0 > Backport HBASE-25287 to branch-1 > > > Key: HBASE-25405 > URL: https://issues.apache.org/jira/browse/HBASE-25405 > Project: HBase > Issue Type: Sub-task >Reporter: Wei-Chiu Chuang >Assignee: Wei-Chiu Chuang >Priority: Major > Fix For: 1.7.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25405) Backport HBASE-25287 to branch-1
[ https://issues.apache.org/jira/browse/HBASE-25405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HBASE-25405. - Resolution: Fixed > Backport HBASE-25287 to branch-1 > > > Key: HBASE-25405 > URL: https://issues.apache.org/jira/browse/HBASE-25405 > Project: HBase > Issue Type: Sub-task >Reporter: Wei-Chiu Chuang >Assignee: Wei-Chiu Chuang >Priority: Major > Fix For: 1.7.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2850: HBASE-25467 Fix error prone problems on branch-2.2
Apache-HBase commented on pull request #2850: URL: https://github.com/apache/hbase/pull/2850#issuecomment-755703933 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 35s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-2.2 Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 11s | branch-2.2 passed | | +1 :green_heart: | compile | 1m 23s | branch-2.2 passed | | +1 :green_heart: | checkstyle | 2m 0s | branch-2.2 passed | | +1 :green_heart: | shadedjars | 4m 5s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 4s | branch-2.2 passed | | +0 :ok: | spotbugs | 3m 25s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 4m 43s | branch-2.2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 45s | the patch passed | | +1 :green_heart: | compile | 1m 29s | the patch passed | | +1 :green_heart: | javac | 1m 29s | the patch passed | | +1 :green_heart: | checkstyle | 2m 34s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 50s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 31m 39s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 2.10.0 or 3.1.2 3.2.1. | | +1 :green_heart: | javadoc | 1m 8s | the patch passed | | +1 :green_heart: | findbugs | 5m 56s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 4m 16s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 158m 57s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 1m 13s | The patch does not generate ASF License warnings. | | | | 241m 23s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2850/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2850 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux d34da259d66f 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2850/out/precommit/personality/provided.sh | | git revision | branch-2.2 / 821f422175 | | Default Java | Oracle Corporation-1.8.0_275-b01 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2850/2/testReport/ | | Max. process+thread count | 3553 (vs. ulimit of 12500) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2850/2/console | | versions | git=2.20.1 maven=3.6.3 findbugs=3.1.11 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] jojochuang merged pull request #2799: Backport HBASE-25287 to branch-1
jojochuang merged pull request #2799: URL: https://github.com/apache/hbase/pull/2799 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] jojochuang commented on pull request #2799: Backport HBASE-25287 to branch-1
jojochuang commented on pull request #2799: URL: https://github.com/apache/hbase/pull/2799#issuecomment-755689035 Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #2754: HBASE-25375 Provide a VM-based release environment
saintstack commented on a change in pull request #2754: URL: https://github.com/apache/hbase/pull/2754#discussion_r552954576 ## File path: dev-support/release-vm/README.md ## @@ -0,0 +1,141 @@ + + +# HBase Release Env + +This is a vagrant project that provides a virtual machine environment suitable +for running an Apache HBase release. + +Requires: +* [VirtualBox](http://virtualbox.org) +* [Vagrant](http://virtualbox.org) +* The private portion of your signing key avilable in the local GPG agent +* The private portion of your Github authentication key available in either the local GPG agent or + local SSH agent + +## Usage + +Unlock the local keyring before proceeding (this should prompt you for your GPG passphrase). For +example, assuming you have an authentication key configured in your keyring, this will do the +trick. + +All terminal commands used below are assumed to be run with the current working directory as the +location containing the `Vagrantfile`. + +The term "Host" is used to mean the environment that runs the Vagrant process. The term "Guest" is +used to mean the virtual machine managed by the Host. + +### Ensure credentials work from the Host OS + +The ssh- and gpg-agent forwarding configuration used here assumes that your credentials work +on the Host. Verify both are working before you proceed with the Guest. Additionally, using the +credentials requires you to unlock the respective keyring, the state of which is persisted by the +agent process or processes running on the Host. + +See instructions in [`create-release`](../create-release/README.txt) regarding proper +configuration of ssh- and gpg-agents. + +Assuming the git repo origin is on GitHub, the following command will ensure that your ssh +credentials are working. On the Host, run: + +```sh +host:~$ ssh -T g...@github.com +Hi ! You've successfully authenticated, but GitHub does not provide shell access. +``` + +Likewise, ensure you have an encryption key that can be used to sign a file. Again, on the Host, +run: Review comment: No cd into release-vm dir? ## File path: dev-support/release-vm/puppet/production/data/nodes/rmvm.yaml ## @@ -0,0 +1,34 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +--- +classes: + - docker + - packages::manage + - ssh + +packages::latest: + - curl + - git + - gnupg + - gnupg-agent + - maven Review comment: +1 on stripping them. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25405) Backport HBASE-25287 to branch-1
[ https://issues.apache.org/jira/browse/HBASE-25405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17260041#comment-17260041 ] Wei-Chiu Chuang commented on HBASE-25405: - The actual PR link is https://github.com/apache/hbase/pull/2799 > Backport HBASE-25287 to branch-1 > > > Key: HBASE-25405 > URL: https://issues.apache.org/jira/browse/HBASE-25405 > Project: HBase > Issue Type: Sub-task >Reporter: Wei-Chiu Chuang >Assignee: Wei-Chiu Chuang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2754: HBASE-25375 Provide a VM-based release environment
Apache-HBase commented on pull request #2754: URL: https://github.com/apache/hbase/pull/2754#issuecomment-755593965 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 9s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 9s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 9s | Maven dependency ordering for patch | ||| _ Other Tests _ | | | | 3m 31s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2754/8/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2754 | | Optional Tests | | | uname | Linux 7833592c5c02 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a414361ed9 | | Max. process+thread count | 42 (vs. ulimit of 3) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2754/8/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2754: HBASE-25375 Provide a VM-based release environment
Apache-HBase commented on pull request #2754: URL: https://github.com/apache/hbase/pull/2754#issuecomment-755591822 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 27s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 8s | Maven dependency ordering for patch | ||| _ Other Tests _ | | | | 1m 58s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2754/8/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2754 | | Optional Tests | | | uname | Linux 35a361842d24 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a414361ed9 | | Max. process+thread count | 55 (vs. ulimit of 3) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2754/8/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2754: HBASE-25375 Provide a VM-based release environment
Apache-HBase commented on pull request #2754: URL: https://github.com/apache/hbase/pull/2754#issuecomment-755588370 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 5s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +0 :ok: | yamllint | 0m 0s | yamllint was not available. | | +0 :ok: | markdownlint | 0m 0s | markdownlint was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 7s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 8s | Maven dependency ordering for patch | | +1 :green_heart: | shellcheck | 0m 0s | There were no new shellcheck issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | ||| _ Other Tests _ | | +0 :ok: | asflicense | 0m 0s | ASF License check generated no output? | | | | 3m 37s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2754/8/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2754 | | Optional Tests | dupname asflicense shellcheck shelldocs yamllint markdownlint | | uname | Linux b1db0c8e0b00 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a414361ed9 | | Max. process+thread count | 42 (vs. ulimit of 3) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2754/8/console | | versions | git=2.17.1 maven=3.6.3 shellcheck=0.4.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24755) [LOG][RSGroup]Error message is confusing while adding a offline RS to rsgroup
[
https://issues.apache.org/jira/browse/HBASE-24755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17260018#comment-17260018
]
Hudson commented on HBASE-24755:
Results for branch branch-2.2
[build #145 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/145/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/145//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/145//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/145//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/145//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> [LOG][RSGroup]Error message is confusing while adding a offline RS to rsgroup
> -
>
> Key: HBASE-24755
> URL: https://issues.apache.org/jira/browse/HBASE-24755
> Project: HBase
> Issue Type: Bug
> Components: rsgroup
>Affects Versions: 2.2.3
>Reporter: mokai
>Assignee: mokai
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.2.7, 2.3.4, 2.5.0, 2.4.1
>
> Attachments: image-2020-07-22-19-34-42-628.png
>
>
> Move a offine RS from default RSGroup to another RSGroup. The error messge
> says 'Source RSGroup for server xxx does not exist.'. It's confusing.
> !image-2020-07-22-19-34-42-628.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HBASE-25445) SplitWALRemoteProcedure failed to archive split WAL
[ https://issues.apache.org/jira/browse/HBASE-25445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17260005#comment-17260005 ] Viraj Jasani edited comment on HBASE-25445 at 1/6/21, 7:26 PM: --- FYI [~ndimiduk] [~huaxiangsun] Would you like to wait for some time to get this merged before spinning 2.3.4 RC? We are just waiting for a unit test and it's good to go. I just included 2.3.4 in fix versions, please update it as per your decision. was (Author: vjasani): FYI [~ndimiduk] [~huaxiangsun] Would you like to wait for some time to get this merged before spinning 2.3.4 RC? We are just waiting for a unit test and it's good to go. > SplitWALRemoteProcedure failed to archive split WAL > --- > > Key: HBASE-25445 > URL: https://issues.apache.org/jira/browse/HBASE-25445 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 3.0.0-alpha-1, 2.4.0, 2.2.6, 2.3.2 >Reporter: mokai >Assignee: Anjan Das >Priority: Critical > Fix For: 3.0.0-alpha-1, 2.2.7, 2.3.4, 2.5.0, 2.4.1 > > Attachments: ServerCrashWrongFSError.png > > > If 'hbase.wal.dir' and 'hbase.rootdir' are configured to different > filesystem, SplitWALRemoteProcedure archived split WAL failed since > SplitWALManager using wrong fs instance. SplitWALManager should use WAL > corresponding fs instance. > Steps to Reproduce: > * Configure 'hbase.wal.dir' and 'hbase.rootdir' so that they point to > different fs instances. > * Start HBase with multiple RS. > * Create a couple of tables and some rows in them so that the RSs get > assigned with some regions. > * Take any RS with non-zero number of regions offline. > * Check master logs for "Wrong FS" error as shown in the screenshot > attached. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HBASE-25445) SplitWALRemoteProcedure failed to archive split WAL
[ https://issues.apache.org/jira/browse/HBASE-25445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17260005#comment-17260005 ] Viraj Jasani edited comment on HBASE-25445 at 1/6/21, 7:21 PM: --- FYI [~ndimiduk] [~huaxiangsun] Would you like to wait for some time to get this merged before spinning 2.3.4 RC? We are just waiting for a unit test and it's good to go. was (Author: vjasani): FYI [~ndimiduk] [~huaxiangsun] Would you like to wait for some time to get this merged before spinning 2.3.4 RC? We are just waiting for a test and it's good to go. > SplitWALRemoteProcedure failed to archive split WAL > --- > > Key: HBASE-25445 > URL: https://issues.apache.org/jira/browse/HBASE-25445 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 3.0.0-alpha-1, 2.4.0, 2.2.6, 2.3.2 >Reporter: mokai >Assignee: Anjan Das >Priority: Critical > Fix For: 3.0.0-alpha-1, 2.2.7, 2.3.4, 2.5.0, 2.4.1 > > Attachments: ServerCrashWrongFSError.png > > > If 'hbase.wal.dir' and 'hbase.rootdir' are configured to different > filesystem, SplitWALRemoteProcedure archived split WAL failed since > SplitWALManager using wrong fs instance. SplitWALManager should use WAL > corresponding fs instance. > Steps to Reproduce: > * Configure 'hbase.wal.dir' and 'hbase.rootdir' so that they point to > different fs instances. > * Start HBase with multiple RS. > * Create a couple of tables and some rows in them so that the RSs get > assigned with some regions. > * Take any RS with non-zero number of regions offline. > * Check master logs for "Wrong FS" error as shown in the screenshot > attached. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25445) SplitWALRemoteProcedure failed to archive split WAL
[ https://issues.apache.org/jira/browse/HBASE-25445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated HBASE-25445: - Fix Version/s: (was: 2.3.5) 2.3.4 > SplitWALRemoteProcedure failed to archive split WAL > --- > > Key: HBASE-25445 > URL: https://issues.apache.org/jira/browse/HBASE-25445 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 3.0.0-alpha-1, 2.4.0, 2.2.6, 2.3.2 >Reporter: mokai >Assignee: Anjan Das >Priority: Critical > Fix For: 3.0.0-alpha-1, 2.2.7, 2.3.4, 2.5.0, 2.4.1 > > Attachments: ServerCrashWrongFSError.png > > > If 'hbase.wal.dir' and 'hbase.rootdir' are configured to different > filesystem, SplitWALRemoteProcedure archived split WAL failed since > SplitWALManager using wrong fs instance. SplitWALManager should use WAL > corresponding fs instance. > Steps to Reproduce: > * Configure 'hbase.wal.dir' and 'hbase.rootdir' so that they point to > different fs instances. > * Start HBase with multiple RS. > * Create a couple of tables and some rows in them so that the RSs get > assigned with some regions. > * Take any RS with non-zero number of regions offline. > * Check master logs for "Wrong FS" error as shown in the screenshot > attached. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25445) SplitWALRemoteProcedure failed to archive split WAL
[ https://issues.apache.org/jira/browse/HBASE-25445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17260005#comment-17260005 ] Viraj Jasani commented on HBASE-25445: -- FYI [~ndimiduk] [~huaxiangsun] Would you like to wait for some time to get this merged before spinning 2.3.4 RC? We are just waiting for a test and it's good to go. > SplitWALRemoteProcedure failed to archive split WAL > --- > > Key: HBASE-25445 > URL: https://issues.apache.org/jira/browse/HBASE-25445 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 3.0.0-alpha-1, 2.4.0, 2.2.6, 2.3.2 >Reporter: mokai >Assignee: Anjan Das >Priority: Critical > Fix For: 3.0.0-alpha-1, 2.2.7, 2.5.0, 2.4.1, 2.3.5 > > Attachments: ServerCrashWrongFSError.png > > > If 'hbase.wal.dir' and 'hbase.rootdir' are configured to different > filesystem, SplitWALRemoteProcedure archived split WAL failed since > SplitWALManager using wrong fs instance. SplitWALManager should use WAL > corresponding fs instance. > Steps to Reproduce: > * Configure 'hbase.wal.dir' and 'hbase.rootdir' so that they point to > different fs instances. > * Start HBase with multiple RS. > * Create a couple of tables and some rows in them so that the RSs get > assigned with some regions. > * Take any RS with non-zero number of regions offline. > * Check master logs for "Wrong FS" error as shown in the screenshot > attached. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25445) SplitWALRemoteProcedure failed to archive split WAL
[ https://issues.apache.org/jira/browse/HBASE-25445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated HBASE-25445: - Fix Version/s: 2.3.5 2.4.1 2.5.0 2.2.7 3.0.0-alpha-1 > SplitWALRemoteProcedure failed to archive split WAL > --- > > Key: HBASE-25445 > URL: https://issues.apache.org/jira/browse/HBASE-25445 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 3.0.0-alpha-1, 2.4.0, 2.2.6, 2.3.2 >Reporter: mokai >Assignee: Anjan Das >Priority: Critical > Fix For: 3.0.0-alpha-1, 2.2.7, 2.5.0, 2.4.1, 2.3.5 > > Attachments: ServerCrashWrongFSError.png > > > If 'hbase.wal.dir' and 'hbase.rootdir' are configured to different > filesystem, SplitWALRemoteProcedure archived split WAL failed since > SplitWALManager using wrong fs instance. SplitWALManager should use WAL > corresponding fs instance. > Steps to Reproduce: > * Configure 'hbase.wal.dir' and 'hbase.rootdir' so that they point to > different fs instances. > * Start HBase with multiple RS. > * Create a couple of tables and some rows in them so that the RSs get > assigned with some regions. > * Take any RS with non-zero number of regions offline. > * Check master logs for "Wrong FS" error as shown in the screenshot > attached. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2800: HBASE-25249 Adding StoreContext
Apache-HBase commented on pull request #2800: URL: https://github.com/apache/hbase/pull/2800#issuecomment-755541061 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 34s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 28s | master passed | | +1 :green_heart: | checkstyle | 1m 40s | master passed | | +1 :green_heart: | spotbugs | 3m 3s | master passed | | -0 :warning: | patch | 1m 2s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 1s | the patch passed | | +1 :green_heart: | checkstyle | 1m 16s | hbase-server: The patch generated 0 new + 42 unchanged - 1 fixed = 42 total (was 43) | | +1 :green_heart: | checkstyle | 0m 19s | The patch passed checkstyle in hbase-mapreduce | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 20m 51s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 3m 5s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 25s | The patch does not generate ASF License warnings. | | | | 48m 22s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2800/11/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2800 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux c956ffd7b703 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a414361ed9 | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-server hbase-mapreduce U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2800/11/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25433) There is no limit on the table name length when creating a table
[ https://issues.apache.org/jira/browse/HBASE-25433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17259989#comment-17259989 ] Huaxiang Sun commented on HBASE-25433: -- moving out to 2.3.5. > There is no limit on the table name length when creating a table > > > Key: HBASE-25433 > URL: https://issues.apache.org/jira/browse/HBASE-25433 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 2.2.3 >Reporter: ZouWeiMing >Priority: Major > Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.1, 2.3.5 > > Attachments: attachment_tableNameLengthLimit.html > > > It occurs > Exeception(ERROR：java.util.concurrent.ExecutionExceptionjava.io.IOExpcetionException > in createDir) because of 8000 bytes length limits on HDFS directory name if > the length of HBase table name misses validation when creating. > So,we decide to verify the length of table name when creating a > table.IllegalArgumentExcetion will be thrown when length is larger than 8000 > bytes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25433) There is no limit on the table name length when creating a table
[ https://issues.apache.org/jira/browse/HBASE-25433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxiang Sun updated HBASE-25433: - Fix Version/s: (was: 2.3.4) 2.3.5 > There is no limit on the table name length when creating a table > > > Key: HBASE-25433 > URL: https://issues.apache.org/jira/browse/HBASE-25433 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 2.2.3 >Reporter: ZouWeiMing >Priority: Major > Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.1, 2.3.5 > > Attachments: attachment_tableNameLengthLimit.html > > > It occurs > Exeception(ERROR：java.util.concurrent.ExecutionExceptionjava.io.IOExpcetionException > in createDir) because of 8000 bytes length limits on HDFS directory name if > the length of HBase table name misses validation when creating. > So,we decide to verify the length of table name when creating a > table.IllegalArgumentExcetion will be thrown when length is larger than 8000 > bytes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25421) There is no limit on the column family length when creating a table
[ https://issues.apache.org/jira/browse/HBASE-25421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17259987#comment-17259987 ] Huaxiang Sun commented on HBASE-25421: -- move out to 2.3.5. > There is no limit on the column family length when creating a table > --- > > Key: HBASE-25421 > URL: https://issues.apache.org/jira/browse/HBASE-25421 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 2.2.3 >Reporter: LiDingSheng >Priority: Major > Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.1, 2.3.5 > > Attachments: attachment.html > > > Table can be created successfully when the length of column family is larger > than 127 bytes,however,it occurs exception when putting data into the > table(ERROR：Family > 127). > So,We decide to verify the length of column family when creating a > table.IllegalArgumentException will be thrown when length is larger than 127 > bytes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25421) There is no limit on the column family length when creating a table
[ https://issues.apache.org/jira/browse/HBASE-25421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxiang Sun updated HBASE-25421: - Fix Version/s: (was: 2.3.4) 2.3.5 > There is no limit on the column family length when creating a table > --- > > Key: HBASE-25421 > URL: https://issues.apache.org/jira/browse/HBASE-25421 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 2.2.3 >Reporter: LiDingSheng >Priority: Major > Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.1, 2.3.5 > > Attachments: attachment.html > > > Table can be created successfully when the length of column family is larger > than 127 bytes,however,it occurs exception when putting data into the > table(ERROR：Family > 127). > So,We decide to verify the length of column family when creating a > table.IllegalArgumentException will be thrown when length is larger than 127 > bytes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25287) Forgetting to unbuffer streams results in many CLOSE_WAIT sockets when loading files
[
https://issues.apache.org/jira/browse/HBASE-25287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Huaxiang Sun updated HBASE-25287:
-
Resolution: Fixed
Status: Resolved (was: Patch Available)
Resolving here as [~weichiu] has a separate Jira to back port to branch-2.
> Forgetting to unbuffer streams results in many CLOSE_WAIT sockets when
> loading files
>
>
> Key: HBASE-25287
> URL: https://issues.apache.org/jira/browse/HBASE-25287
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.4.0
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.4, 2.5.0, 2.4.1
>
> Attachments: 1605328358304-image.png, 1605328417888-image.png,
> 1605504914256-image.png
>
>
> HBASE-9393 found seek+read will leave many CLOSE_WAIT sockets without stream
> unbuffer, which can free sockets and file descriptors held by the stream.
> In our cluster RSes with about one hundred thousand store files, we found the
> number of CLOSE_WAIT sockets increases with the number of regions opened,
> and can up to the operating system open files limit 100.
>
> {code:java}
> 2020-11-12 20:19:02,452 WARN [1282990092@qtp-220038608-1 - Acceptor0
> SelectChannelConnector@0.0.0.0:16030] mortbay.log: EXCEPTION
> java.io.IOException: Too many open files
> at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
> at
> sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
> at
> sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
> at
> org.mortbay.jetty.nio.SelectChannelConnector$1.acceptChannel(SelectChannelConnector.java:75)
> at
> org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:686)
> at
> org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:192)
> at
> org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
> at
> org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:708)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> {code}
>
> {code:java}
> [hbase@gha-data-hbase-cat0053 hbase]$ ulimit -SHn
> 100
> {code}
>
>
> The reason of the problem is, when store file opened,
> {code:java}
> private void open() throws IOException {
> fileInfo.initHDFSBlocksDistribution();
> long readahead = fileInfo.isNoReadahead() ? 0L : -1L;
> ReaderContext context = fileInfo.createReaderContext(false, readahead,
> ReaderType.PREAD);
> fileInfo.initHFileInfo(context);
> StoreFileReader reader = fileInfo.preStoreFileReaderOpen(context,
> cacheConf);
> if (reader == null) {
> reader = fileInfo.createReader(context, cacheConf);
> fileInfo.getHFileInfo().initMetaAndIndex(reader.getHFileReader());
> }
> {code}
> only createReader() unbuffered the stream. In initMetaAndIndex(), using the
> stream to read blocks, so it needs to unbuffer() the socket , too.
> We can just add try before fileInfo.initHFileInfo(context); and finally
> unbuffer() the stream at the end of the open() function.
> We fixed it on our cluster, the number of CLOSE_WAIT reduced to about 0.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25034) Table regions details on master GUI display slowly.
[ https://issues.apache.org/jira/browse/HBASE-25034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxiang Sun updated HBASE-25034: - Fix Version/s: (was: 2.3.4) 2.3.5 > Table regions details on master GUI display slowly. > --- > > Key: HBASE-25034 > URL: https://issues.apache.org/jira/browse/HBASE-25034 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.3.2 >Reporter: DingSheng Li >Priority: Major > Labels: newbie > Fix For: 3.0.0-alpha-1, 2.5.0, 2.3.5 > > Attachments: The table display after pagination.html > > > When a table has a large number of regions (e.g.,a single table contains more > than 100,000 regions), it takes about 20 to 30 minutes to display the table > regions on the master GUI, which is unacceptable to users. After testing, we > find that web page rendering takes up the most time,and this can be solved by > pagination query. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25034) Table regions details on master GUI display slowly.
[ https://issues.apache.org/jira/browse/HBASE-25034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17259983#comment-17259983 ] Huaxiang Sun commented on HBASE-25034: -- move out to 2.3.5. > Table regions details on master GUI display slowly. > --- > > Key: HBASE-25034 > URL: https://issues.apache.org/jira/browse/HBASE-25034 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.3.2 >Reporter: DingSheng Li >Priority: Major > Labels: newbie > Fix For: 3.0.0-alpha-1, 2.5.0, 2.3.5 > > Attachments: The table display after pagination.html > > > When a table has a large number of regions (e.g.,a single table contains more > than 100,000 regions), it takes about 20 to 30 minutes to display the table > regions on the master GUI, which is unacceptable to users. After testing, we > find that web page rendering takes up the most time,and this can be solved by > pagination query. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25019) check if we're impacted by MASSEMBLY-941 and mitigate if needed
[ https://issues.apache.org/jira/browse/HBASE-25019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxiang Sun updated HBASE-25019: - Fix Version/s: (was: 2.3.4) 2.3.5 > check if we're impacted by MASSEMBLY-941 and mitigate if needed > --- > > Key: HBASE-25019 > URL: https://issues.apache.org/jira/browse/HBASE-25019 > Project: HBase > Issue Type: Task > Components: scripts >Affects Versions: 3.0.0-alpha-1, 2.3.0, 2.3.1, 1.6.0, 1.7.0 >Reporter: Sean Busbey >Priority: Minor > Fix For: 3.0.0-alpha-1, 1.7.0, 2.5.0, 2.3.5 > > > MASSEMBLY-941 notes a bug starting in version 3.2.0 of the assembly plugin > where scripts lose their executable bit. > We've had this version since updating to the apache parent pom version 22 in > HBASE-23675. We should check our release artifacts to see if we are impacted > and if so downgrade the assembly plugin in our poms to 3.1.1. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25019) check if we're impacted by MASSEMBLY-941 and mitigate if needed
[ https://issues.apache.org/jira/browse/HBASE-25019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17259982#comment-17259982 ] Huaxiang Sun commented on HBASE-25019: -- move out to 2.3.5. > check if we're impacted by MASSEMBLY-941 and mitigate if needed > --- > > Key: HBASE-25019 > URL: https://issues.apache.org/jira/browse/HBASE-25019 > Project: HBase > Issue Type: Task > Components: scripts >Affects Versions: 3.0.0-alpha-1, 2.3.0, 2.3.1, 1.6.0, 1.7.0 >Reporter: Sean Busbey >Priority: Minor > Fix For: 3.0.0-alpha-1, 1.7.0, 2.5.0, 2.3.5 > > > MASSEMBLY-941 notes a bug starting in version 3.2.0 of the assembly plugin > where scripts lose their executable bit. > We've had this version since updating to the apache parent pom version 22 in > HBASE-23675. We should check our release artifacts to see if we are impacted > and if so downgrade the assembly plugin in our poms to 3.1.1. -- This message was sent by Atlassian Jira (v8.3.4#803005)
