[GitHub] [hbase] Apache-HBase commented on issue #528: HBASE-22890 Verify the files when RegionServer is starting and BucketCache is in file mode
Apache-HBase commented on issue #528: HBASE-22890 Verify the files when RegionServer is starting and BucketCache is in file mode URL: https://github.com/apache/hbase/pull/528#issuecomment-531651656 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 10m 13s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 0s | No case conflicting files found. | | :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. | ||| _ branch-1 Compile Tests _ | | :green_heart: | mvninstall | 8m 1s | branch-1 passed | | :green_heart: | compile | 2m 3s | branch-1 passed | | :green_heart: | checkstyle | 1m 44s | branch-1 passed | | :broken_heart: | shadedjars | 0m 11s | branch has 10 errors when building our shaded downstream artifacts. | | :green_heart: | javadoc | 0m 36s | branch-1 passed | | :blue_heart: | spotbugs | 2m 24s | Used deprecated FindBugs config; considering switching to SpotBugs. | | :green_heart: | findbugs | 2m 22s | branch-1 passed | ||| _ Patch Compile Tests _ | | :green_heart: | mvninstall | 1m 50s | the patch passed | | :green_heart: | compile | 1m 47s | the patch passed | | :green_heart: | javac | 1m 47s | the patch passed | | :broken_heart: | checkstyle | 1m 39s | hbase-server: The patch generated 2 new + 46 unchanged - 6 fixed = 48 total (was 52) | | :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | :broken_heart: | shadedjars | 0m 10s | patch has 10 errors when building our shaded downstream artifacts. | | :green_heart: | hadoopcheck | 4m 43s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | :green_heart: | javadoc | 0m 30s | the patch passed | | :broken_heart: | findbugs | 2m 22s | hbase-server generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) | ||| _ Other Tests _ | | :green_heart: | unit | 110m 35s | hbase-server in the patch passed. | | :green_heart: | asflicense | 0m 25s | The patch does not generate ASF License warnings. | | | | 152m 31s | | | Reason | Tests | |---:|:--| | FindBugs | module:hbase-server | | | org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.retrieveFromFile(int[]) ignores result of java.io.ObjectInputStream.read(byte[]) At BucketCache.java: At BucketCache.java:[line 1073] | | | org.apache.hadoop.hbase.io.hfile.bucket.FileIOEngine.readPersistentChecksum(String) ignores result of java.io.ObjectInputStream.read(byte[]) At FileIOEngine.java: At FileIOEngine.java:[line 328] | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-528/37/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/528 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux a1f4cc976144 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-528/out/precommit/personality/provided.sh | | git revision | branch-1 / 20d7da5 | | Default Java | 1.8.0_222 | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-528/37/artifact/out/branch-shadedjars.txt | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-528/37/artifact/out/diff-checkstyle-hbase-server.txt | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-528/37/artifact/out/patch-shadedjars.txt | | findbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-528/37/artifact/out/new-findbugs-hbase-server.html | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-528/37/testReport/ | | Max. process+thread count | 4157 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-528/37/console | | versions | git=1.9.1 maven=3.0.5 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.11.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu
[GitHub] [hbase] Reidddddd commented on a change in pull request #528: HBASE-22890 Verify the files when RegionServer is starting and BucketCache is in file mode
Reidd commented on a change in pull request #528: HBASE-22890 Verify the
files when RegionServer is starting and BucketCache is in file mode
URL: https://github.com/apache/hbase/pull/528#discussion_r324501589
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/FileIOEngine.java
##
@@ -267,6 +317,99 @@ void refreshFileConnection(int accessFileNum, IOException
ioe) throws IOExceptio
}
}
+ /**
+ * Read the persistent checksum from persistent path
+ * @param persistentPath the backingMap persistent path
+ * @return the persistent checksum
+ */
+ private byte[] readPersistentChecksum(String persistentPath) {
+try (ObjectInputStream ois = new ObjectInputStream(new
FileInputStream(persistentPath))) {
+ byte[] PBMagic = new byte[ProtobufUtil.PB_MAGIC.length];
+ ois.read(PBMagic);
+ if (Bytes.equals(ProtobufUtil.PB_MAGIC, PBMagic)) {
+int length = ois.readInt();
+byte[] persistentChecksum = new byte[length];
+ois.read(persistentChecksum);
+return persistentChecksum;
+ } else {
+// if the persistent file is not start with PB_MAGIC, it's an old
version file
+oldVersion = true;
+ }
+} catch (IOException ioex) {
+ LOG.warn("Failed read persistent checksum, because of " + ioex);
+ return null;
+}
+return null;
+ }
+
+ @Override
+ public void deleteCacheDataFile() {
+if (filePaths == null) {
+ return;
+}
+for (String file : filePaths) {
+ new File(file).delete();
+}
+ }
+
+ @Override
+ public byte[] calculateChecksum()
+throws IOException, NoSuchAlgorithmException {
+
Review comment:
Unnecessary empty line.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Reidddddd commented on a change in pull request #528: HBASE-22890 Verify the files when RegionServer is starting and BucketCache is in file mode
Reidd commented on a change in pull request #528: HBASE-22890 Verify the
files when RegionServer is starting and BucketCache is in file mode
URL: https://github.com/apache/hbase/pull/528#discussion_r324501125
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/FileIOEngine.java
##
@@ -19,46 +19,96 @@
package org.apache.hadoop.hbase.io.hfile.bucket;
import java.io.File;
+import java.io.FileInputStream;
import java.io.IOException;
+import java.io.ObjectInputStream;
import java.io.RandomAccessFile;
import java.nio.ByteBuffer;
import java.nio.channels.ClosedByInterruptException;
import java.nio.channels.ClosedChannelException;
import java.nio.channels.FileChannel;
+import java.security.MessageDigest;
+import java.security.NoSuchAlgorithmException;
import java.util.Arrays;
import java.util.concurrent.locks.ReentrantLock;
import com.google.common.annotations.VisibleForTesting;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.apache.hadoop.hbase.classification.InterfaceAudience;
+import org.apache.hadoop.hbase.protobuf.ProtobufUtil;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.util.Shell;
import org.apache.hadoop.util.StringUtils;
/**
* IO engine that stores data to a file on the local file system.
*/
@InterfaceAudience.Private
-public class FileIOEngine implements IOEngine {
+public class FileIOEngine implements PersistentIOEngine {
private static final Log LOG = LogFactory.getLog(FileIOEngine.class);
public static final String FILE_DELIMITER = ",";
+ private static final DuFileCommand du = new DuFileCommand(new String[]
{"du", ""});
Review comment:
Upper case for static final member.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Work started] (HBASE-23026) docker run command should not quote JAVA_VOL
[ https://issues.apache.org/jira/browse/HBASE-23026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-23026 started by Peter Somogyi. - > docker run command should not quote JAVA_VOL > > > Key: HBASE-23026 > URL: https://issues.apache.org/jira/browse/HBASE-23026 > Project: HBase > Issue Type: Bug > Components: hbase-operator-tools >Affects Versions: hbase-operator-tools-1.0.0 >Reporter: Peter Somogyi >Assignee: Peter Somogyi >Priority: Major > Fix For: hbase-operator-tools-1.0.0 > > > The JAVA_VOL variable shouldn't be quoted when calling docker run in > do-release-docker.sh. > + docker run -ti --env-file /Users/peter.somogyi/projects/rm/env.list > --volume /Users/peter.somogyi/projects/rm:/opt/hbase-rm '' hbase-rm:latest > docker: invalid reference format. > A similar error occurs when -j is passed to the script: > unknown flag: --volume > /Users/peter.somogyi/.sdkman/candidates/java/current:/opt/hbase-java -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [hbase-operator-tools] asf-ci commented on issue #34: HBASE-23026 docker run command should not quote JAVA_VOL
asf-ci commented on issue #34: HBASE-23026 docker run command should not quote JAVA_VOL URL: https://github.com/apache/hbase-operator-tools/pull/34#issuecomment-531598144 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/PreCommit-HBASE-OPERATOR-TOOLS-Build/107/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase-operator-tools] petersomogyi opened a new pull request #34: HBASE-23026 docker run command should not quote JAVA_VOL
petersomogyi opened a new pull request #34: HBASE-23026 docker run command should not quote JAVA_VOL URL: https://github.com/apache/hbase-operator-tools/pull/34 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (HBASE-23026) docker run command should not quote JAVA_VOL
Peter Somogyi created HBASE-23026: - Summary: docker run command should not quote JAVA_VOL Key: HBASE-23026 URL: https://issues.apache.org/jira/browse/HBASE-23026 Project: HBase Issue Type: Bug Components: hbase-operator-tools Affects Versions: hbase-operator-tools-1.0.0 Reporter: Peter Somogyi Assignee: Peter Somogyi Fix For: hbase-operator-tools-1.0.0 The JAVA_VOL variable shouldn't be quoted when calling docker run in do-release-docker.sh. + docker run -ti --env-file /Users/peter.somogyi/projects/rm/env.list --volume /Users/peter.somogyi/projects/rm:/opt/hbase-rm '' hbase-rm:latest docker: invalid reference format. A similar error occurs when -j is passed to the script: unknown flag: --volume /Users/peter.somogyi/.sdkman/candidates/java/current:/opt/hbase-java -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22955) Branches-1 precommit and nightly yetus jobs are using jdk8 for jdk7 jobs
[
https://issues.apache.org/jira/browse/HBASE-22955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16930074#comment-16930074
]
Sean Busbey commented on HBASE-22955:
-
filed YETUS-913 for the squashing of JAVA_HOME
> Branches-1 precommit and nightly yetus jobs are using jdk8 for jdk7 jobs
>
>
> Key: HBASE-22955
> URL: https://issues.apache.org/jira/browse/HBASE-22955
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.5.0, 1.3.6, 1.4.11
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
>
> branch-1 nightly job is failing while trying to do our yetus shaded jar test
> because the enforcer plugin complains about a jdk8 jdk-tools jar being
> present.
>
> looking at the yetus footer, it's because jdk8 is being used instead of jdk7:
> {code:java}
> [2019-08-30T13:30:34.353Z] | maven | version: Apache Maven 3.0.5 |
> [2019-08-30T13:30:34.353Z] | Default Java | 1.8.0_222 |
> {code}
> Same thing in the footer of the jdk7 build (which passes, even though it
> shouldn't)
> {code}
> [2019-08-30T15:43:29.103Z] | maven | version: Apache Maven 3.0.5 |
> [2019-08-30T15:43:29.103Z] | Default Java | 1.8.0_222 |
> {code}
> Both builds appear to be properly running in Docker mode.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Resolved] (HBASE-23025) Do not quote GPG command
[
https://issues.apache.org/jira/browse/HBASE-23025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Somogyi resolved HBASE-23025.
---
Resolution: Fixed
Merged PR #33 to master.
> Do not quote GPG command
>
>
> Key: HBASE-23025
> URL: https://issues.apache.org/jira/browse/HBASE-23025
> Project: HBase
> Issue Type: Bug
> Components: hbase-operator-tools
>Affects Versions: hbase-operator-tools-1.0.0
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Fix For: hbase-operator-tools-1.0.0
>
>
> Release making script fails on this line:
> "$GPG" --passphrase "$GPG_PASSPHRASE" --export-secret-key --armor "$GPG_KEY"
> > "$GPG_KEY_FILE"
>
> {noformat}
> ./dev-support/create-release/do-release-docker.sh: line 118: gpg
> --pinentry-mode loopback --no-tty --batch: command not found
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [hbase-operator-tools] petersomogyi commented on issue #33: HBASE-23025 Do not quote GPG command
petersomogyi commented on issue #33: HBASE-23025 Do not quote GPG command URL: https://github.com/apache/hbase-operator-tools/pull/33#issuecomment-531591786 I don't think we have any of these currently. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase-operator-tools] petersomogyi merged pull request #33: HBASE-23025 Do not quote GPG command
petersomogyi merged pull request #33: HBASE-23025 Do not quote GPG command URL: https://github.com/apache/hbase-operator-tools/pull/33 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase-operator-tools] busbey commented on issue #33: HBASE-23025 Do not quote GPG command
busbey commented on issue #33: HBASE-23025 Do not quote GPG command URL: https://github.com/apache/hbase-operator-tools/pull/33#issuecomment-531591564 It'll only be needed if we need to have either a GPG command or one of the args with a space or something the shell would parse as a glob. Probably fine to do that later as part of a general shellcheck cleanup. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase-operator-tools] petersomogyi commented on issue #33: HBASE-23025 Do not quote GPG command
petersomogyi commented on issue #33: HBASE-23025 Do not quote GPG command URL: https://github.com/apache/hbase-operator-tools/pull/33#issuecomment-531587975 > Should the GPG variable be an array instead? Probably that would also work. I'm not that proficient in bash to judge but if you recommend array here I can take a look. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22956) [HBCK2/hbase-operator-tools] Make first release, 1.0.0
[ https://issues.apache.org/jira/browse/HBASE-22956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16930016#comment-16930016 ] Peter Somogyi commented on HBASE-22956: --- I ran into HBASE-23025 while working on the release. > [HBCK2/hbase-operator-tools] Make first release, 1.0.0 > -- > > Key: HBASE-22956 > URL: https://issues.apache.org/jira/browse/HBASE-22956 > Project: HBase > Issue Type: Task > Components: hbase-operator-tools, hbck2 >Affects Versions: hbase-operator-tools-1.0.0 >Reporter: stack >Assignee: Peter Somogyi >Priority: Major > > Make our first release of hbck2/hbase-operator-tools. > First release should have the coverage hbck1 had at least. When the parent > for this issue is done, we'll be at hbck1+. Let us release then (week or > two?). > A release will help operators who have been struggling having to build hbck2 > against different hbase versions. The release should be a "fat > jar"/completely contained with all dependency satisfied so operator can just > fire up hbck2 w/o having to build against an hbase or provide some magic mix > of jars to satisfy hbck2 tool needs. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [hbase-operator-tools] asf-ci commented on issue #33: HBASE-23025 Do not quote GPG command
asf-ci commented on issue #33: HBASE-23025 Do not quote GPG command URL: https://github.com/apache/hbase-operator-tools/pull/33#issuecomment-531582663 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/PreCommit-HBASE-OPERATOR-TOOLS-Build/106/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase-operator-tools] petersomogyi opened a new pull request #33: HBASE-23025 Do not quote GPG command
petersomogyi opened a new pull request #33: HBASE-23025 Do not quote GPG command URL: https://github.com/apache/hbase-operator-tools/pull/33 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (HBASE-23025) Do not quote GPG command
Peter Somogyi created HBASE-23025:
-
Summary: Do not quote GPG command
Key: HBASE-23025
URL: https://issues.apache.org/jira/browse/HBASE-23025
Project: HBase
Issue Type: Bug
Components: hbase-operator-tools
Affects Versions: hbase-operator-tools-1.0.0
Reporter: Peter Somogyi
Assignee: Peter Somogyi
Fix For: hbase-operator-tools-1.0.0
Release making script fails on this line:
"$GPG" --passphrase "$GPG_PASSPHRASE" --export-secret-key --armor "$GPG_KEY" >
"$GPG_KEY_FILE"
{noformat}
./dev-support/create-release/do-release-docker.sh: line 118: gpg
--pinentry-mode loopback --no-tty --batch: command not found
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [hbase] Apache-HBase commented on issue #559: HBASE-22944 Space Quota: TableNotFoundException: hbase:quota is thrown when region server is restarted.
Apache-HBase commented on issue #559: HBASE-22944 Space Quota: TableNotFoundException: hbase:quota is thrown when region server is restarted. URL: https://github.com/apache/hbase/pull/559#issuecomment-531578211 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 0m 35s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 0s | No case conflicting files found. | | :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | :green_heart: | mvninstall | 5m 27s | master passed | | :green_heart: | compile | 0m 59s | master passed | | :green_heart: | checkstyle | 1m 19s | master passed | | :green_heart: | shadedjars | 4m 45s | branch has no errors when building our shaded downstream artifacts. | | :green_heart: | javadoc | 0m 40s | master passed | | :blue_heart: | spotbugs | 4m 11s | Used deprecated FindBugs config; considering switching to SpotBugs. | | :green_heart: | findbugs | 4m 9s | master passed | ||| _ Patch Compile Tests _ | | :green_heart: | mvninstall | 5m 9s | the patch passed | | :green_heart: | compile | 1m 0s | the patch passed | | :green_heart: | javac | 1m 0s | the patch passed | | :green_heart: | checkstyle | 1m 18s | the patch passed | | :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | :green_heart: | shadedjars | 4m 51s | patch has no errors when building our shaded downstream artifacts. | | :green_heart: | hadoopcheck | 16m 6s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | :green_heart: | javadoc | 0m 36s | the patch passed | | :green_heart: | findbugs | 4m 21s | the patch passed | ||| _ Other Tests _ | | :green_heart: | unit | 171m 43s | hbase-server in the patch passed. | | :green_heart: | asflicense | 0m 36s | The patch does not generate ASF License warnings. | | | | 230m 9s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-559/15/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/559 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 75df86f43dc5 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-559/out/precommit/personality/provided.sh | | git revision | master / 1cb4f687c1 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-559/15/testReport/ | | Max. process+thread count | 4987 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-559/15/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-22956) [HBCK2/hbase-operator-tools] Make first release, 1.0.0
[ https://issues.apache.org/jira/browse/HBASE-22956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi updated HBASE-22956: -- Issue Type: Task (was: Bug) > [HBCK2/hbase-operator-tools] Make first release, 1.0.0 > -- > > Key: HBASE-22956 > URL: https://issues.apache.org/jira/browse/HBASE-22956 > Project: HBase > Issue Type: Task > Components: hbase-operator-tools, hbck2 >Affects Versions: hbase-operator-tools-1.0.0 >Reporter: stack >Assignee: Peter Somogyi >Priority: Major > > Make our first release of hbck2/hbase-operator-tools. > First release should have the coverage hbck1 had at least. When the parent > for this issue is done, we'll be at hbck1+. Let us release then (week or > two?). > A release will help operators who have been struggling having to build hbck2 > against different hbase versions. The release should be a "fat > jar"/completely contained with all dependency satisfied so operator can just > fire up hbck2 w/o having to build against an hbase or provide some magic mix > of jars to satisfy hbck2 tool needs. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HBASE-22956) [HBCK2/hbase-operator-tools] Make first release, 1.0.0
[ https://issues.apache.org/jira/browse/HBASE-22956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi updated HBASE-22956: -- Affects Version/s: hbase-operator-tools-1.0.0 > [HBCK2/hbase-operator-tools] Make first release, 1.0.0 > -- > > Key: HBASE-22956 > URL: https://issues.apache.org/jira/browse/HBASE-22956 > Project: HBase > Issue Type: Bug > Components: hbase-operator-tools, hbck2 >Affects Versions: hbase-operator-tools-1.0.0 >Reporter: stack >Assignee: Peter Somogyi >Priority: Major > > Make our first release of hbck2/hbase-operator-tools. > First release should have the coverage hbck1 had at least. When the parent > for this issue is done, we'll be at hbck1+. Let us release then (week or > two?). > A release will help operators who have been struggling having to build hbck2 > against different hbase versions. The release should be a "fat > jar"/completely contained with all dependency satisfied so operator can just > fire up hbck2 w/o having to build against an hbase or provide some magic mix > of jars to satisfy hbck2 tool needs. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HBASE-22956) [HBCK2/hbase-operator-tools] Make first release, 1.0.0
[ https://issues.apache.org/jira/browse/HBASE-22956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi updated HBASE-22956: -- Component/s: hbase-operator-tools > [HBCK2/hbase-operator-tools] Make first release, 1.0.0 > -- > > Key: HBASE-22956 > URL: https://issues.apache.org/jira/browse/HBASE-22956 > Project: HBase > Issue Type: Bug > Components: hbase-operator-tools, hbck2 >Reporter: stack >Assignee: Peter Somogyi >Priority: Major > > Make our first release of hbck2/hbase-operator-tools. > First release should have the coverage hbck1 had at least. When the parent > for this issue is done, we'll be at hbck1+. Let us release then (week or > two?). > A release will help operators who have been struggling having to build hbck2 > against different hbase versions. The release should be a "fat > jar"/completely contained with all dependency satisfied so operator can just > fire up hbck2 w/o having to build against an hbase or provide some magic mix > of jars to satisfy hbck2 tool needs. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (HBASE-22956) [HBCK2/hbase-operator-tools] Make first release, 1.0.0
[ https://issues.apache.org/jira/browse/HBASE-22956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi reassigned HBASE-22956: - Assignee: Peter Somogyi > [HBCK2/hbase-operator-tools] Make first release, 1.0.0 > -- > > Key: HBASE-22956 > URL: https://issues.apache.org/jira/browse/HBASE-22956 > Project: HBase > Issue Type: Bug > Components: hbck2 >Reporter: stack >Assignee: Peter Somogyi >Priority: Major > > Make our first release of hbck2/hbase-operator-tools. > First release should have the coverage hbck1 had at least. When the parent > for this issue is done, we'll be at hbck1+. Let us release then (week or > two?). > A release will help operators who have been struggling having to build hbck2 > against different hbase versions. The release should be a "fat > jar"/completely contained with all dependency satisfied so operator can just > fire up hbck2 w/o having to build against an hbase or provide some magic mix > of jars to satisfy hbck2 tool needs. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [hbase] Apache-HBase commented on issue #626: HBASE-23017 Verify the file integrity in persistent IOEngine
Apache-HBase commented on issue #626: HBASE-23017 Verify the file integrity in persistent IOEngine URL: https://github.com/apache/hbase/pull/626#issuecomment-531563092 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 3m 1s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 0s | No case conflicting files found. | | :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. | ||| _ master Compile Tests _ | | :green_heart: | mvninstall | 5m 52s | master passed | | :green_heart: | compile | 0m 59s | master passed | | :green_heart: | checkstyle | 1m 29s | master passed | | :green_heart: | shadedjars | 4m 56s | branch has no errors when building our shaded downstream artifacts. | | :green_heart: | javadoc | 0m 37s | master passed | | :blue_heart: | spotbugs | 4m 31s | Used deprecated FindBugs config; considering switching to SpotBugs. | | :green_heart: | findbugs | 4m 29s | master passed | ||| _ Patch Compile Tests _ | | :green_heart: | mvninstall | 5m 27s | the patch passed | | :green_heart: | compile | 0m 56s | the patch passed | | :green_heart: | javac | 0m 56s | the patch passed | | :broken_heart: | checkstyle | 1m 28s | hbase-server: The patch generated 1 new + 31 unchanged - 0 fixed = 32 total (was 31) | | :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | :green_heart: | shadedjars | 4m 55s | patch has no errors when building our shaded downstream artifacts. | | :green_heart: | hadoopcheck | 18m 51s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | :green_heart: | javadoc | 0m 36s | the patch passed | | :broken_heart: | findbugs | 4m 36s | hbase-server generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) | ||| _ Other Tests _ | | :broken_heart: | unit | 227m 51s | hbase-server in the patch failed. | | :green_heart: | asflicense | 0m 30s | The patch does not generate ASF License warnings. | | | | 293m 58s | | | Reason | Tests | |---:|:--| | FindBugs | module:hbase-server | | | org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.retrieveFromFile(int[]) ignores result of java.io.FileInputStream.read(byte[]) At BucketCache.java: At BucketCache.java:[line 1104] | | | org.apache.hadoop.hbase.io.hfile.bucket.PersistentIOEngine.readPersistentChecksum(String) ignores result of java.io.FileInputStream.read(byte[]) At PersistentIOEngine.java: At PersistentIOEngine.java:[line 89] | | Failed junit tests | hadoop.hbase.client.TestSizeFailures | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.2 Server=19.03.2 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-626/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/626 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 5afc131ebe1b 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-626/out/precommit/personality/provided.sh | | git revision | master / 1cb4f687c1 | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-626/2/artifact/out/diff-checkstyle-hbase-server.txt | | findbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-626/2/artifact/out/new-findbugs-hbase-server.html | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-626/2/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-626/2/testReport/ | | Max. process+thread count | 4740 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-626/2/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service
[GitHub] [hbase] Apache-HBase commented on issue #627: HBASE-23024 : Replace initcause with Constructor arg
Apache-HBase commented on issue #627: HBASE-23024 : Replace initcause with Constructor arg URL: https://github.com/apache/hbase/pull/627#issuecomment-531562815 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 2m 58s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 0s | No case conflicting files found. | | :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :yellow_heart: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | :blue_heart: | mvndep | 0m 33s | Maven dependency ordering for branch | | :green_heart: | mvninstall | 5m 48s | master passed | | :green_heart: | compile | 1m 43s | master passed | | :green_heart: | checkstyle | 2m 31s | master passed | | :green_heart: | shadedjars | 4m 56s | branch has no errors when building our shaded downstream artifacts. | | :green_heart: | javadoc | 1m 22s | master passed | | :blue_heart: | spotbugs | 4m 22s | Used deprecated FindBugs config; considering switching to SpotBugs. | | :green_heart: | findbugs | 6m 16s | master passed | ||| _ Patch Compile Tests _ | | :blue_heart: | mvndep | 0m 15s | Maven dependency ordering for patch | | :green_heart: | mvninstall | 5m 29s | the patch passed | | :green_heart: | compile | 1m 45s | the patch passed | | :green_heart: | javac | 1m 45s | the patch passed | | :green_heart: | checkstyle | 2m 31s | the patch passed | | :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | :green_heart: | shadedjars | 4m 56s | patch has no errors when building our shaded downstream artifacts. | | :green_heart: | hadoopcheck | 17m 17s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | :green_heart: | javadoc | 1m 17s | the patch passed | | :green_heart: | findbugs | 6m 46s | the patch passed | ||| _ Other Tests _ | | :green_heart: | unit | 2m 55s | hbase-common in the patch passed. | | :green_heart: | unit | 1m 49s | hbase-client in the patch passed. | | :broken_heart: | unit | 233m 5s | hbase-server in the patch failed. | | :green_heart: | asflicense | 1m 14s | The patch does not generate ASF License warnings. | | | | 312m 47s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.0 Server=19.03.0 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-627/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/627 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 521d67797323 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-627/out/precommit/personality/provided.sh | | git revision | master / 1cb4f687c1 | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-627/1/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-627/1/testReport/ | | Max. process+thread count | 4748 (vs. ulimit of 1) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-627/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #559: HBASE-22944 Space Quota: TableNotFoundException: hbase:quota is thrown when region server is restarted.
Apache-HBase commented on issue #559: HBASE-22944 Space Quota: TableNotFoundException: hbase:quota is thrown when region server is restarted. URL: https://github.com/apache/hbase/pull/559#issuecomment-531560964 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 1m 30s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 1s | No case conflicting files found. | | :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | :green_heart: | mvninstall | 5m 36s | master passed | | :green_heart: | compile | 0m 57s | master passed | | :green_heart: | checkstyle | 1m 18s | master passed | | :green_heart: | shadedjars | 4m 47s | branch has no errors when building our shaded downstream artifacts. | | :green_heart: | javadoc | 0m 39s | master passed | | :blue_heart: | spotbugs | 3m 52s | Used deprecated FindBugs config; considering switching to SpotBugs. | | :green_heart: | findbugs | 3m 49s | master passed | ||| _ Patch Compile Tests _ | | :green_heart: | mvninstall | 5m 6s | the patch passed | | :green_heart: | compile | 0m 57s | the patch passed | | :green_heart: | javac | 0m 57s | the patch passed | | :green_heart: | checkstyle | 1m 17s | the patch passed | | :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | :green_heart: | shadedjars | 4m 41s | patch has no errors when building our shaded downstream artifacts. | | :green_heart: | hadoopcheck | 15m 47s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | :green_heart: | javadoc | 0m 36s | the patch passed | | :green_heart: | findbugs | 3m 48s | the patch passed | ||| _ Other Tests _ | | :green_heart: | unit | 166m 15s | hbase-server in the patch passed. | | :green_heart: | asflicense | 0m 35s | The patch does not generate ASF License warnings. | | | | 224m 13s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-559/14/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/559 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux accf7342673f 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-559/out/precommit/personality/provided.sh | | git revision | master / 1cb4f687c1 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-559/14/testReport/ | | Max. process+thread count | 5077 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-559/14/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22514) Move rsgroup feature into core of HBase
[
https://issues.apache.org/jira/browse/HBASE-22514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16929947#comment-16929947
]
Hudson commented on HBASE-22514:
Results for branch HBASE-22514
[build #110 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/110/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/110//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/110//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/110//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/110//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Move rsgroup feature into core of HBase
> ---
>
> Key: HBASE-22514
> URL: https://issues.apache.org/jira/browse/HBASE-22514
> Project: HBase
> Issue Type: Umbrella
> Components: Admin, Client, rsgroup
>Reporter: Yechao Chen
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HBASE-22514.master.001.patch,
> image-2019-05-31-18-25-38-217.png
>
>
> The class RSGroupAdminClient is not public
> we need to use java api RSGroupAdminClient to manager RSG
> so RSGroupAdminClient should be public
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HBASE-22978) Online slow response log
[
https://issues.apache.org/jira/browse/HBASE-22978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16929945#comment-16929945
]
Viraj Jasani commented on HBASE-22978:
--
{quote}{{get_slow_responses [ ... , ] [ , \{
} ]}}
Retrieve, decode, and pretty print the contents of the too slow response ring
buffer maintained by the given list of servers; or all servers in the cluster
if no list is provided. Optionally provide a map of parameters for filtering as
additional argument.
{quote}
{quote}I like the idea of persisting the complete slow log with best effort,
though. Reminds me of the [MySQL slow log dump
tool|https://dev.mysql.com/doc/refman/8.0/en/mysqldumpslow.html]. Initially I
was thinking this could be part of general performance surveillance, where
sampling is good enough, but maybe there could be a tough to debug case that's
rare so would also be hard to catch in the ring buffers. For that we'd
configure a slow log directory in site configuration, presumably in HDFS, into
which regionservers would each append to a file, rolling at some configured
bound.
{quote}
shell triggered get_slow_responses should retrieve records maintained by ring
buffers only? or it should read records from HDFS?
If shell reads only the buffer only, we might miss on details of hard to catch
rare cases (already written to HDFS)?
> Online slow response log
>
>
> Key: HBASE-22978
> URL: https://issues.apache.org/jira/browse/HBASE-22978
> Project: HBase
> Issue Type: New Feature
> Components: Admin, Operability, regionserver, shell
>Reporter: Andrew Purtell
>Priority: Minor
>
> Today when an individual RPC exceeds a configurable time bound we log a
> complaint by way of the logging subsystem. These log lines look like:
> {noformat}
> 2019-08-30 22:10:36,195 WARN [,queue=15,port=60020] ipc.RpcServer -
> (responseTooSlow):
> {"call":"Scan(org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ScanRequest)",
> "starttimems":1567203007549,
> "responsesize":6819737,
> "method":"Scan",
> "param":"region { type: REGION_NAME value:
> \"tsdb,\\000\\000\\215\\f)o\\024\\302\\220\\000\\000\\000\\000\\000\\001\\000\\000\\000\\000\\000\\006\\000\\000\\000\\000\\000\\005\\000\\000",
> "processingtimems":28646,
> "client":"10.253.196.215:41116",
> "queuetimems":22453,
> "class":"HRegionServer"}
> {noformat}
> Unfortunately we often truncate the request parameters, like in the above
> example. We do this because the human readable representation is verbose, the
> rate of too slow warnings may be high, and the combination of these things
> can overwhelm the log capture system. The truncation is unfortunate because
> it eliminates much of the utility of the warnings. For example, the region
> name, the start and end keys, and the filter hierarchy are all important
> clues for debugging performance problems caused by moderate to low
> selectivity queries or queries made at a high rate.
> We can maintain an in-memory ring buffer of requests that were judged to be
> too slow in addition to the responseTooSlow logging. The in-memory
> representation can be complete and compressed. A new admin API and shell
> command can provide access to the ring buffer for online performance
> debugging. A modest sizing of the ring buffer will prevent excessive memory
> utilization for a minor performance debugging feature by limiting the total
> number of retained records. There is some chance a high rate of requests will
> cause information on other interesting requests to be overwritten before it
> can be read. This is the nature of a ring buffer and an acceptable trade off.
> The write request types do not require us to retain all information submitted
> in the request. We don't need to retain all key-values in the mutation, which
> may be too large to comfortably retain. We only need a unique set of row
> keys, or even a min/max range, and total counts.
> The consumers of this information will be debugging tools. We can afford to
> apply fast compression to ring buffer entries (if codec support is
> available), something like snappy or zstandard, and decompress on the fly
> when servicing the retrieval API request. This will minimize the impact of
> retaining more information about slow requests than we do today.
> This proposal is for retention of request information only, the same
> information provided by responseTooSlow warnings. Total size of response
> serialization, possibly also total cell or row counts, should be sufficient
> to characterize the response.
> Optionally persist new entries added to the ring buffer into one or more
> files in HDFS in a write-behind manner. If the HDFS writer blocks or falls
> behind and we are unable to persist an entry before it is overwritten, that
> is fine. Response too
[GitHub] [hbase] Apache-HBase commented on issue #528: HBASE-22890 Verify the files when RegionServer is starting and BucketCache is in file mode
Apache-HBase commented on issue #528: HBASE-22890 Verify the files when RegionServer is starting and BucketCache is in file mode URL: https://github.com/apache/hbase/pull/528#issuecomment-531554168 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 11m 11s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 0s | No case conflicting files found. | | :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. | ||| _ branch-1 Compile Tests _ | | :green_heart: | mvninstall | 8m 0s | branch-1 passed | | :green_heart: | compile | 1m 53s | branch-1 passed | | :green_heart: | checkstyle | 1m 34s | branch-1 passed | | :broken_heart: | shadedjars | 0m 14s | branch has 10 errors when building our shaded downstream artifacts. | | :green_heart: | javadoc | 0m 37s | branch-1 passed | | :blue_heart: | spotbugs | 2m 24s | Used deprecated FindBugs config; considering switching to SpotBugs. | | :green_heart: | findbugs | 2m 22s | branch-1 passed | ||| _ Patch Compile Tests _ | | :green_heart: | mvninstall | 1m 47s | the patch passed | | :green_heart: | compile | 1m 45s | the patch passed | | :green_heart: | javac | 1m 45s | the patch passed | | :broken_heart: | checkstyle | 1m 57s | hbase-server: The patch generated 2 new + 46 unchanged - 6 fixed = 48 total (was 52) | | :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | :broken_heart: | shadedjars | 0m 12s | patch has 10 errors when building our shaded downstream artifacts. | | :green_heart: | hadoopcheck | 4m 36s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | :green_heart: | javadoc | 0m 31s | the patch passed | | :broken_heart: | findbugs | 2m 27s | hbase-server generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) | ||| _ Other Tests _ | | :green_heart: | unit | 114m 44s | hbase-server in the patch passed. | | :green_heart: | asflicense | 0m 32s | The patch does not generate ASF License warnings. | | | | 157m 41s | | | Reason | Tests | |---:|:--| | FindBugs | module:hbase-server | | | org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.retrieveFromFile(int[]) ignores result of java.io.ObjectInputStream.read(byte[]) At BucketCache.java: At BucketCache.java:[line 1073] | | | org.apache.hadoop.hbase.io.hfile.bucket.FileIOEngine.readPersistentChecksum(String) ignores result of java.io.ObjectInputStream.read(byte[]) At FileIOEngine.java: At FileIOEngine.java:[line 328] | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-528/36/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/528 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux d9eec913af2b 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-528/out/precommit/personality/provided.sh | | git revision | branch-1 / 20d7da5 | | Default Java | 1.8.0_222 | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-528/36/artifact/out/branch-shadedjars.txt | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-528/36/artifact/out/diff-checkstyle-hbase-server.txt | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-528/36/artifact/out/patch-shadedjars.txt | | findbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-528/36/artifact/out/new-findbugs-hbase-server.html | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-528/36/testReport/ | | Max. process+thread count | 4420 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-528/36/console | | versions | git=1.9.1 maven=3.0.5 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.11.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub
[GitHub] [hbase] shardul-cr7 commented on a change in pull request #559: HBASE-22944 Space Quota: TableNotFoundException: hbase:quota is thrown when region server is restarted.
shardul-cr7 commented on a change in pull request #559: HBASE-22944 Space
Quota: TableNotFoundException: hbase:quota is thrown when region server is
restarted.
URL: https://github.com/apache/hbase/pull/559#discussion_r324452121
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/SpaceQuotaRefresherChore.java
##
@@ -74,6 +76,12 @@ public
SpaceQuotaRefresherChore(RegionServerSpaceQuotaManager manager, Connectio
@Override
protected void chore() {
try {
+ // check whether Quota table is present or not.
+ if (!quotaTablePresent && !checkQuotaTableExists()) {
+LOG.info("Quota table not found, skipping quota manager cache
refresh.");
+quotaTablePresent = true;
Review comment:
Thanks for the review. Made the changes. :)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] shardul-cr7 commented on a change in pull request #559: HBASE-22944 Space Quota: TableNotFoundException: hbase:quota is thrown when region server is restarted.
shardul-cr7 commented on a change in pull request #559: HBASE-22944 Space
Quota: TableNotFoundException: hbase:quota is thrown when region server is
restarted.
URL: https://github.com/apache/hbase/pull/559#discussion_r324451813
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/SpaceQuotaRefresherChore.java
##
@@ -74,6 +76,12 @@ public
SpaceQuotaRefresherChore(RegionServerSpaceQuotaManager manager, Connectio
@Override
protected void chore() {
try {
+ // check whether Quota table is present or not.
+ if (!quotaTablePresent && !checkQuotaTableExists()) {
+LOG.info("Quota table not found, skipping quota manager cache
refresh.");
+quotaTablePresent = true;
Review comment:
ohh yes..I should set it after the if condition...my bad :)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] srdo commented on a change in pull request #361: HBase-22027: Split non-MR related parts of TokenUtil off into a Clien…
srdo commented on a change in pull request #361: HBase-22027: Split non-MR related parts of TokenUtil off into a Clien… URL: https://github.com/apache/hbase/pull/361#discussion_r324451511 ## File path: hbase-client/src/main/java/org/apache/hadoop/hbase/security/token/ClientTokenUtil.java ## @@ -0,0 +1,200 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.hadoop.hbase.security.token; + +import com.google.protobuf.ByteString; Review comment: I'm not sure I follow. How does making ClientTokenUtil an interface affect class shading? The implementation of the ClientTokenUtil interface would still have to reference the unshaded classes, right? If you have an idea for refactoring this, I'd be happy to merge it into this PR. If you put up a PR against the HBASE-22027 branch at https://github.com/srdo/hbase, I could update this PR with your changes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] ZhaoBQ commented on a change in pull request #528: HBASE-22890 Verify the files when RegionServer is starting and BucketCache is in file mode
ZhaoBQ commented on a change in pull request #528: HBASE-22890 Verify the files
when RegionServer is starting and BucketCache is in file mode
URL: https://github.com/apache/hbase/pull/528#discussion_r324450723
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/PersistentIOEngine.java
##
@@ -0,0 +1,58 @@
+/**
+ * Copyright The Apache Software Foundation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.hadoop.hbase.io.hfile.bucket;
+
+import java.io.IOException;
+import java.security.NoSuchAlgorithmException;
+
+import org.apache.hadoop.hbase.classification.InterfaceAudience;
+
+/**
+ * A class implementing PersistentIOEngine interface supports persistent and
file integrity verify
+ * for {@link BucketCache}
+ */
+@InterfaceAudience.Private
+public interface PersistentIOEngine extends IOEngine {
+
+ /**
+ * Delete bucketcache files
+ */
+ void deleteCacheDataFile();
+
+ /**
+ * Using an encryption algorithm to calculate a checksum, the default
encryption algorithm is MD5
+ * @return the checksum which is convert to HexString
+ * @throws IOException something happened like file not exists
+ * @throws NoSuchAlgorithmException no such algorithm
+ */
+ byte[] calculateChecksum()
+throws IOException, NoSuchAlgorithmException;
+
+ /**
+ * Whether the persistence file is old version, it's for back compatibility
+ * @return true if the persistence file is old version
Review comment:
It has been modified. Thanks!
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Updated] (HBASE-23024) Replace Exception.initCause() with Constructor args
[ https://issues.apache.org/jira/browse/HBASE-23024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated HBASE-23024: - Fix Version/s: 2.3.0 1.5.0 3.0.0 Status: Patch Available (was: In Progress) > Replace Exception.initCause() with Constructor args > --- > > Key: HBASE-23024 > URL: https://issues.apache.org/jira/browse/HBASE-23024 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Minor > Fix For: 3.0.0, 1.5.0, 2.3.0 > > > We can use parameterized constructor for providing root cause of some > Exceptions based on the availability of such constructors to avoid calling > .initCause() for Exceptions. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Work started] (HBASE-23024) Replace Exception.initCause() with Constructor args
[ https://issues.apache.org/jira/browse/HBASE-23024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-23024 started by Viraj Jasani. > Replace Exception.initCause() with Constructor args > --- > > Key: HBASE-23024 > URL: https://issues.apache.org/jira/browse/HBASE-23024 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Minor > > We can use parameterized constructor for providing root cause of some > Exceptions based on the availability of such constructors to avoid calling > .initCause() for Exceptions. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [hbase] virajjasani opened a new pull request #627: HBASE-23024 : Replace initcause with Constructor arg
virajjasani opened a new pull request #627: HBASE-23024 : Replace initcause with Constructor arg URL: https://github.com/apache/hbase/pull/627 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (HBASE-23024) Replace Exception.initCause() with Constructor args
Viraj Jasani created HBASE-23024: Summary: Replace Exception.initCause() with Constructor args Key: HBASE-23024 URL: https://issues.apache.org/jira/browse/HBASE-23024 Project: HBase Issue Type: Improvement Affects Versions: 3.0.0 Reporter: Viraj Jasani Assignee: Viraj Jasani We can use parameterized constructor for providing root cause of some Exceptions based on the availability of such constructors to avoid calling .initCause() for Exceptions. -- This message was sent by Atlassian Jira (v8.3.2#803003)
