[jira] [Commented] (HBASE-22784) OldWALs not cleared in a replication slave cluster (cyclic replication bw 2 clusters)
[ https://issues.apache.org/jira/browse/HBASE-22784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919332#comment-16919332 ] Solvannan R M commented on HBASE-22784: --- [~wchevreuil] Thanks for the quick fix. Currently, all our clusters are running HBase 1.4.9. Would it be possible to provide a patch for 1.4.9 till the stable release of 1.4.11 or 1.5.0? > OldWALs not cleared in a replication slave cluster (cyclic replication bw 2 > clusters) > - > > Key: HBASE-22784 > URL: https://issues.apache.org/jira/browse/HBASE-22784 > Project: HBase > Issue Type: Bug > Components: regionserver, Replication >Affects Versions: 1.4.9, 1.4.10 >Reporter: Solvannan R M >Assignee: Wellington Chevreuil >Priority: Blocker > Fix For: 1.5.0, 1.4.11 > > Attachments: HBASE-22784.branch-1.001.patch, > HBASE-22784.branch-1.002.patch, HBASE-22784.branch-1.003.patch, > HBASE-22784.branch-1.004.patch > > > When a cluster is passive (receiving edits only via replication) in a cyclic > replication setup of 2 clusters, OldWALs size keeps on growing. On analysing, > we observed the following behaviour. > # New entry is added to WAL (Edit replicated from other cluster). > # ReplicationSourceWALReaderThread(RSWALRT) reads and applies the configured > filters (due to cyclic replication setup, ClusterMarkingEntryFilter discards > new entry from other cluster). > # Entry is null, RSWALRT neither updates the batch stats > (WALEntryBatch.lastWalPosition) nor puts it in the entryBatchQueue. > # ReplicationSource thread is blocked in entryBachQueue.take(). > # So ReplicationSource#updateLogPosition has never invoked and WAL file is > never cleared from ReplicationQueue. > # Hence LogCleaner on the master, doesn't deletes the oldWAL files from > hadoop. > NOTE: When a new edit is added via hbase-client, ReplicationSource thread > process and clears the oldWAL files from replication queues and hence master > cleans up the WALs > Please provide us a solution > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [hbase] Reidddddd commented on a change in pull request #528: HBASE-22890 Verify the files when RegionServer is starting and BucketCache is in file mode
Reidd commented on a change in pull request #528: HBASE-22890 Verify the
files when RegionServer is starting and BucketCache is in file mode
URL: https://github.com/apache/hbase/pull/528#discussion_r319439221
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
##
@@ -370,9 +388,15 @@ private IOEngine getIOEngineFromName(String ioEngineName,
long capacity)
// In order to make the usage simple, we only need the prefix 'files:' in
// document whether one or multiple file(s), but also support 'file:' for
// the compatibility
- String[] filePaths =
+ filePaths =
ioEngineName.substring(ioEngineName.indexOf(":") +
1).split(FileIOEngine.FILE_DELIMITER);
- return new FileIOEngine(capacity, filePaths);
+ hasKey = FileIOEngineUtils.readLongFormPersistencePath(persistencePath)
== -1;
+ if (hasKey) {
+return new FileIOEngine(algorithm, persistencePath, capacity,
filePaths);
+ } else {
+return new FileIOEngine(capacity,filePaths);
Review comment:
I don't like the idea of keeping the older constructor, the `hasKey` check
can put into the new constructor as well.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #548: HBASE-22930 Set unique name to longCompactions/shortCompactions threads
Apache-HBase commented on issue #548: HBASE-22930 Set unique name to longCompactions/shortCompactions threads URL: https://github.com/apache/hbase/pull/548#issuecomment-526569207 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 48 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 | mvninstall | 404 | master passed | | +1 | compile | 69 | master passed | | +1 | checkstyle | 98 | master passed | | +1 | shadedjars | 303 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 39 | master passed | | 0 | spotbugs | 250 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 249 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 297 | the patch passed | | +1 | compile | 57 | the patch passed | | +1 | javac | 57 | the patch passed | | +1 | checkstyle | 83 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 282 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 978 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 36 | the patch passed | | +1 | findbugs | 249 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 10364 | hbase-server in the patch passed. | | +1 | asflicense | 27 | The patch does not generate ASF License warnings. | | | | 13971 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-548/6/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/548 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux d4a523e76468 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-548/out/precommit/personality/provided.sh | | git revision | master / 090c55f3ff | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-548/6/testReport/ | | Max. process+thread count | 4627 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-548/6/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22881) Fix non-daemon threads in hbase server implementation
[
https://issues.apache.org/jira/browse/HBASE-22881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919400#comment-16919400
]
Hudson commented on HBASE-22881:
Results for branch branch-2.1
[build #1537 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1537/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1537//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1537//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1537//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Fix non-daemon threads in hbase server implementation
> -
>
> Key: HBASE-22881
> URL: https://issues.apache.org/jira/browse/HBASE-22881
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.1.7, 2.2.2
>
>
> "pool-8-thread-3" #7252 prio=5 os_prio=0 tid=0x7f91040044c0 nid=0xd71e
> waiting on condition [0x7f8f4d209000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0005c0e49ed0> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
>Locked ownable synchronizers:
> - None
> "pool-8-thread-2" #7251 prio=5 os_prio=0 tid=0x7f910c010be0 nid=0xd71d
> waiting on condition [0x7f8f4daab000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0005c0e49ed0> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
>Locked ownable synchronizers:
> - None
> "pool-8-thread-1" #7250 prio=5 os_prio=0 tid=0x7f9119d0 nid=0xd71c
> waiting on condition [0x7f8f4da6a000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0005c0e49ed0> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
>Locked ownable synchronizers:
> - None
> "pool-5-thread-3" #7248 prio=5 os_prio=0 tid=0x7f9238005ad0 nid=0xd71a
> waiting on condition [0x7f8f4cb65000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for
[GitHub] [hbase] Apache-HBase commented on issue #559: HBASE-22944 TableNotFoundException: hbase:quota is thrown when region server is restarted.
Apache-HBase commented on issue #559: HBASE-22944 TableNotFoundException: hbase:quota is thrown when region server is restarted. URL: https://github.com/apache/hbase/pull/559#issuecomment-526567438 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 40 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | +1 | mvninstall | 345 | master passed | | +1 | compile | 55 | master passed | | +1 | checkstyle | 78 | master passed | | +1 | shadedjars | 290 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 39 | master passed | | 0 | spotbugs | 261 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 259 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 316 | the patch passed | | +1 | compile | 57 | the patch passed | | +1 | javac | 57 | the patch passed | | +1 | checkstyle | 76 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 299 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1089 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 40 | the patch passed | | +1 | findbugs | 295 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 9537 | hbase-server in the patch passed. | | +1 | asflicense | 29 | The patch does not generate ASF License warnings. | | | | 13288 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-559/6/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/559 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux a46eccf22eaf 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-559/out/precommit/personality/provided.sh | | git revision | master / 090c55f3ff | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-559/6/testReport/ | | Max. process+thread count | 4458 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-559/6/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #553: HBASE-22939 SpaceQuotas- Bulkload from different hdfs failed when space quotas are turned on.
Apache-HBase commented on issue #553: HBASE-22939 SpaceQuotas- Bulkload from different hdfs failed when space quotas are turned on. URL: https://github.com/apache/hbase/pull/553#issuecomment-526498453 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 164 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 | mvninstall | 476 | master passed | | +1 | compile | 75 | master passed | | +1 | checkstyle | 94 | master passed | | +1 | shadedjars | 345 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 47 | master passed | | 0 | spotbugs | 297 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 295 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 437 | the patch passed | | +1 | compile | 75 | the patch passed | | +1 | javac | 75 | the patch passed | | +1 | checkstyle | 97 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 364 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1312 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 47 | the patch passed | | +1 | findbugs | 292 | the patch passed | ||| _ Other Tests _ | | -1 | unit | 16053 | hbase-server in the patch failed. | | +1 | asflicense | 25 | The patch does not generate ASF License warnings. | | | | 20703 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.TestSplitMerge | | | hadoop.hbase.regionserver.wal.TestAsyncLogRolling | | | hadoop.hbase.regionserver.TestNewVersionBehaviorFromClientSide | | | hadoop.hbase.regionserver.TestRegionServerMetrics | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-553/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/553 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 4368a0e7fadf 4.15.0-48-generic #51-Ubuntu SMP Wed Apr 3 08:28:49 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-553/out/precommit/personality/provided.sh | | git revision | master / 090c55f3ff | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-553/3/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-553/3/testReport/ | | Max. process+thread count | 4606 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-553/3/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-22939) SpaceQuotas- Bulkload from different hdfs failed when space quotas are turned on.
[
https://issues.apache.org/jira/browse/HBASE-22939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiran Wu updated HBASE-22939:
-
Attachment: HBASE-22939-v2.patch
> SpaceQuotas- Bulkload from different hdfs failed when space quotas are turned
> on.
> -
>
> Key: HBASE-22939
> URL: https://issues.apache.org/jira/browse/HBASE-22939
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Yiran Wu
>Assignee: Yiran Wu
>Priority: Major
> Attachments: HBASE-22939-v0.patch, HBASE-22939-v1.patch,
> HBASE-22939-v2.patch
>
>
> {code:java}
> Caused by:
> org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(java.io.IOException):
> java.io.IOException: Wrong FS:
> hdfs://172.16.159.148:8020/tmp/bkldOutPut/fm1/327d2de5db4d4f0da667bfdf77105d4d,
> expected: hdfs://snake
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:433)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318)
> Caused by: java.lang.IllegalArgumentException: Wrong FS:
> hdfs://172.16.159.148:8020/tmp/bkldOutPut/fm1/327d2de5db4d4f0da667bfdf77105d4d,
> expected: hdfs://snake
> at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:665)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:214)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1440)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1437)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1437)
> at
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:442)
> at
> org.apache.hadoop.hbase.quotas.policies.AbstractViolationPolicyEnforcement.getFileSize(AbstractViolationPolicyEnforcement.java:95)
> at
> org.apache.hadoop.hbase.quotas.policies.MissingSnapshotViolationPolicyEnforcement.computeBulkLoadSize(MissingSnapshotViolationPolicyEnforcement.java:53)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.bulkLoadHFile(RSRpcServices.java:2407)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:42004)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:374)
> ... 3 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [hbase] Apache-HBase commented on issue #528: HBASE-22890 Verify the files when RegionServer is starting and BucketCache is in file mode
Apache-HBase commented on issue #528: HBASE-22890 Verify the files when RegionServer is starting and BucketCache is in file mode URL: https://github.com/apache/hbase/pull/528#issuecomment-526521221 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 51 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ branch-1 Compile Tests _ | | +1 | mvninstall | 498 | branch-1 passed | | +1 | compile | 44 | branch-1 passed | | +1 | checkstyle | 98 | branch-1 passed | | -1 | shadedjars | 14 | branch has 10 errors when building our shaded downstream artifacts. | | +1 | javadoc | 41 | branch-1 passed | | 0 | spotbugs | 146 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 144 | branch-1 passed | | -0 | patch | 154 | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 115 | the patch passed | | +1 | compile | 46 | the patch passed | | +1 | javac | 46 | the patch passed | | -1 | checkstyle | 104 | hbase-server: The patch generated 11 new + 51 unchanged - 0 fixed = 62 total (was 51) | | +1 | whitespace | 0 | The patch has no whitespace issues. | | -1 | shadedjars | 13 | patch has 10 errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 323 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 | javadoc | 33 | the patch passed | | +1 | findbugs | 139 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 6776 | hbase-server in the patch passed. | | +1 | asflicense | 30 | The patch does not generate ASF License warnings. | | | | 8670 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-528/18/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/528 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 55d8594b2716 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-528/out/precommit/personality/provided.sh | | git revision | branch-1 / 73e0304 | | Default Java | 1.8.0_222 | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-528/18/artifact/out/branch-shadedjars.txt | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-528/18/artifact/out/diff-checkstyle-hbase-server.txt | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-528/18/artifact/out/patch-shadedjars.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-528/18/testReport/ | | Max. process+thread count | 4414 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-528/18/console | | versions | git=1.9.1 maven=3.0.5 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Reidddddd commented on a change in pull request #528: HBASE-22890 Verify the files when RegionServer is starting and BucketCache is in file mode
Reidd commented on a change in pull request #528: HBASE-22890 Verify the
files when RegionServer is starting and BucketCache is in file mode
URL: https://github.com/apache/hbase/pull/528#discussion_r319440632
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
##
@@ -1597,4 +1631,12 @@ float getMultiFactor() {
float getMemoryFactor() {
return memoryFactor;
}
+
+ public String getAlgorithm() {
Review comment:
Mark annotation if it is for test only
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Reidddddd commented on a change in pull request #528: HBASE-22890 Verify the files when RegionServer is starting and BucketCache is in file mode
Reidd commented on a change in pull request #528: HBASE-22890 Verify the
files when RegionServer is starting and BucketCache is in file mode
URL: https://github.com/apache/hbase/pull/528#discussion_r319440400
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
##
@@ -1029,19 +1053,23 @@ private void persistToFile() throws IOException {
}
fos = new FileOutputStream(persistencePath, false);
oos = new ObjectOutputStream(fos);
+ oos.writeLong(-1);
Review comment:
HBase has a PB_MAGIC_STR in `ProtobufUtil`, I suggest use that instead of
`-1`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-22784) OldWALs not cleared in a replication slave cluster (cyclic replication bw 2 clusters)
[ https://issues.apache.org/jira/browse/HBASE-22784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919402#comment-16919402 ] Wellington Chevreuil commented on HBASE-22784: -- [~solvannan] , we don't do patched releases of apache hbase. If you are running a commercial distribution (i.e. cdh or hdp), you would need to request such release for the related vendor. If you are running apache hbase distribution, then you will need to wait for 1.4.11, or, since this patch has been committed to branch-1.4, you can checkout this branch and build it on your own. > OldWALs not cleared in a replication slave cluster (cyclic replication bw 2 > clusters) > - > > Key: HBASE-22784 > URL: https://issues.apache.org/jira/browse/HBASE-22784 > Project: HBase > Issue Type: Bug > Components: regionserver, Replication >Affects Versions: 1.4.9, 1.4.10 >Reporter: Solvannan R M >Assignee: Wellington Chevreuil >Priority: Blocker > Fix For: 1.5.0, 1.4.11 > > Attachments: HBASE-22784.branch-1.001.patch, > HBASE-22784.branch-1.002.patch, HBASE-22784.branch-1.003.patch, > HBASE-22784.branch-1.004.patch > > > When a cluster is passive (receiving edits only via replication) in a cyclic > replication setup of 2 clusters, OldWALs size keeps on growing. On analysing, > we observed the following behaviour. > # New entry is added to WAL (Edit replicated from other cluster). > # ReplicationSourceWALReaderThread(RSWALRT) reads and applies the configured > filters (due to cyclic replication setup, ClusterMarkingEntryFilter discards > new entry from other cluster). > # Entry is null, RSWALRT neither updates the batch stats > (WALEntryBatch.lastWalPosition) nor puts it in the entryBatchQueue. > # ReplicationSource thread is blocked in entryBachQueue.take(). > # So ReplicationSource#updateLogPosition has never invoked and WAL file is > never cleared from ReplicationQueue. > # Hence LogCleaner on the master, doesn't deletes the oldWAL files from > hadoop. > NOTE: When a new edit is added via hbase-client, ReplicationSource thread > process and clears the oldWAL files from replication queues and hence master > cleans up the WALs > Please provide us a solution > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [hbase] Apache-HBase commented on issue #435: HBASE-22767 System table RIT STUCK if their RSGroup has no highest ve…
Apache-HBase commented on issue #435: HBASE-22767 System table RIT STUCK if their RSGroup has no highest ve… URL: https://github.com/apache/hbase/pull/435#issuecomment-526512657 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 32 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | 0 | shelldocs | 0 | Shelldocs was not available. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | 0 | mvndep | 33 | Maven dependency ordering for branch | | +1 | mvninstall | 347 | master passed | | +1 | compile | 113 | master passed | | +1 | checkstyle | 127 | master passed | | +1 | shadedjars | 315 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 90 | master passed | | 0 | spotbugs | 266 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 372 | master passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 17 | Maven dependency ordering for patch | | +1 | mvninstall | 331 | the patch passed | | +1 | compile | 108 | the patch passed | | +1 | javac | 108 | the patch passed | | -1 | checkstyle | 88 | hbase-server: The patch generated 1 new + 9 unchanged - 0 fixed = 10 total (was 9) | | +1 | shellcheck | 0 | There were no new shellcheck issues. | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 303 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1038 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 85 | the patch passed | | -1 | findbugs | 65 | hbase-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | +1 | unit | 179 | hbase-common in the patch passed. | | +1 | unit | 10883 | hbase-server in the patch passed. | | +1 | unit | 411 | hbase-rsgroup in the patch passed. | | +1 | asflicense | 62 | The patch does not generate ASF License warnings. | | | | 15831 | | | Reason | Tests | |---:|:--| | FindBugs | module:hbase-common | | | org.apache.hadoop.hbase.Version.() invokes inefficient new String(String) constructor At Version.java:inefficient new String(String) constructor At Version.java:[line 10] | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-435/90/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/435 | | Optional Tests | dupname asflicense shellcheck shelldocs javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 80b3c7730c37 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-435/out/precommit/personality/provided.sh | | git revision | master / 090c55f3ff | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-435/90/artifact/out/diff-checkstyle-hbase-server.txt | | findbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-435/90/artifact/out/new-findbugs-hbase-common.html | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-435/90/testReport/ | | Max. process+thread count | 4890 (vs. ulimit of 1) | | modules | C: hbase-common hbase-server hbase-rsgroup U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-435/90/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) shellcheck=0.4.4 findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] ravowlga123 commented on a change in pull request #564: HBASE-22900
ravowlga123 commented on a change in pull request #564: HBASE-22900
URL: https://github.com/apache/hbase/pull/564#discussion_r319418141
##
File path: pom.xml
##
@@ -1631,6 +1632,11 @@
httpcore
${httpcore.version}
+
+org.apache.httpcomponents
+httpclient
Review comment:
httpcore was already included in the pom.xml for this branch that's why I
added httpclient only.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-22928) ScanMetrics counter update may not happen in case of exception in TableRecordReaderImpl
[
https://issues.apache.org/jira/browse/HBASE-22928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919290#comment-16919290
]
Reid Chan commented on HBASE-22928:
---
Nice catch, LGTM.
> ScanMetrics counter update may not happen in case of exception in
> TableRecordReaderImpl
> ---
>
> Key: HBASE-22928
> URL: https://issues.apache.org/jira/browse/HBASE-22928
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Minor
>
> ScanMetrics counter update may not happen in case of exception in
> TableRecordReaderImpl.nextKeyValue(),
> There are multiple cases as below,
> 1. In case of DoNotRetryIOException
> {code}
> // do not retry if the exception tells us not to do so
> if (e instanceof DoNotRetryIOException) {
> throw e;
> }
> {code}
> 2. While creating a new scanner,
> {code}
> if (lastSuccessfulRow == null) {
> restart(scan.getStartRow());
> } else {
> restart(lastSuccessfulRow);
> scanner.next(); // skip presumed already mapped row
> }
> {code}
> 3. In case of IOException
> {code}
> catch (IOException ioe) {
> if (logScannerActivity) {
> long now = System.currentTimeMillis();
> LOG.info("Mapper took " + (now-timestamp)
> + "ms to process " + rowcount + " rows");
> LOG.info(ioe.toString(), ioe);
> String lastRow = lastSuccessfulRow == null ?
> "null" : Bytes.toStringBinary(lastSuccessfulRow);
> LOG.info("lastSuccessfulRow=" + lastRow);
> }
> throw ioe;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [hbase] Apache-HBase commented on issue #551: HBASE-22896 TestHRegion.testFlushMarkersWALFail is flaky
Apache-HBase commented on issue #551: HBASE-22896 TestHRegion.testFlushMarkersWALFail is flaky URL: https://github.com/apache/hbase/pull/551#issuecomment-526496579 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 45 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | +1 | mvninstall | 435 | master passed | | +1 | compile | 68 | master passed | | +1 | checkstyle | 79 | master passed | | +1 | shadedjars | 278 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 37 | master passed | | 0 | spotbugs | 245 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 244 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 299 | the patch passed | | +1 | compile | 56 | the patch passed | | +1 | javac | 56 | the patch passed | | +1 | checkstyle | 75 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 286 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 953 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 36 | the patch passed | | +1 | findbugs | 255 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 10258 | hbase-server in the patch passed. | | +1 | asflicense | 29 | The patch does not generate ASF License warnings. | | | | 13812 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-551/5/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/551 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux d9a026e531c7 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-551/out/precommit/personality/provided.sh | | git revision | master / 090c55f3ff | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-551/5/testReport/ | | Max. process+thread count | 4453 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-551/5/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #562: HBASE-20688
Apache-HBase commented on issue #562: HBASE-20688 URL: https://github.com/apache/hbase/pull/562#issuecomment-526569665 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 38 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 | mvninstall | 409 | master passed | | 0 | refguide | 453 | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 376 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | 0 | refguide | 453 | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | ||| _ Other Tests _ | | +1 | asflicense | 20 | The patch does not generate ASF License warnings. | | | | 1814 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-562/5/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/562 | | JIRA Issue | HBASE-20688 | | Optional Tests | dupname asflicense refguide | | uname | Linux c2846a7e8929 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-562/out/precommit/personality/provided.sh | | git revision | master / 090c55f3ff | | refguide | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-562/5/artifact/out/branch-site/book.html | | refguide | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-562/5/artifact/out/patch-site/book.html | | Max. process+thread count | 96 (vs. ulimit of 1) | | modules | C: . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-562/5/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-20688) Refguide has "HBase Backup" section and a chapter named "Backup and Restore"; neither refers to the other
[
https://issues.apache.org/jira/browse/HBASE-20688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919461#comment-16919461
]
HBase QA commented on HBASE-20688:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
38s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
49s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 7m
33s{color} | {color:blue} branch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 7m
33s{color} | {color:blue} patch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 14s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.1 Server=19.03.1 base:
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-562/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hbase/pull/562 |
| JIRA Issue | HBASE-20688 |
| Optional Tests | dupname asflicense refguide |
| uname | Linux c2846a7e8929 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-562/out/precommit/personality/provided.sh
|
| git revision | master / 090c55f3ff |
| refguide |
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-562/5/artifact/out/branch-site/book.html
|
| refguide |
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-562/5/artifact/out/patch-site/book.html
|
| Max. process+thread count | 96 (vs. ulimit of 1) |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-562/5/console |
| versions | git=2.11.0 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.10.0 http://yetus.apache.org |
This message was automatically generated.
> Refguide has "HBase Backup" section and a chapter named "Backup and Restore";
> neither refers to the other
> -
>
> Key: HBASE-20688
> URL: https://issues.apache.org/jira/browse/HBASE-20688
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: Rabi Kumar K C
>Priority: Major
>
> The two backup sections are not connected or related. It'd be confusing to
> the user. Needs addressing.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [hbase] Apache-HBase commented on issue #560: HBASE-22945 Show quota infos in master UI
Apache-HBase commented on issue #560: HBASE-22945 Show quota infos in master UI URL: https://github.com/apache/hbase/pull/560#issuecomment-526499130 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 78 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | 0 | mvndep | 34 | Maven dependency ordering for branch | | +1 | mvninstall | 361 | master passed | | +1 | compile | 89 | master passed | | +1 | checkstyle | 124 | master passed | | +1 | shadedjars | 308 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 62 | master passed | | 0 | spotbugs | 317 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 392 | master passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 19 | Maven dependency ordering for patch | | +1 | mvninstall | 466 | the patch passed | | +1 | compile | 109 | the patch passed | | +1 | javac | 109 | the patch passed | | +1 | checkstyle | 142 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 387 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1425 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 73 | the patch passed | | +1 | findbugs | 350 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 122 | hbase-client in the patch passed. | | -1 | unit | 17337 | hbase-server in the patch failed. | | +1 | asflicense | 50 | The patch does not generate ASF License warnings. | | | | 22469 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.master.TestMasterShutdown | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.0 Server=19.03.0 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-560/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/560 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux e7c60ae0f812 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4 22:49:08 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-560/out/precommit/personality/provided.sh | | git revision | master / 090c55f3ff | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-560/3/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-560/3/testReport/ | | Max. process+thread count | 5084 (vs. ulimit of 1) | | modules | C: hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-560/3/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #510: HBASE-22878 Show table throttle quotas in table jsp
Apache-HBase commented on issue #510: HBASE-22878 Show table throttle quotas in table jsp URL: https://github.com/apache/hbase/pull/510#issuecomment-526528998 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 41 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | 0 | mvndep | 35 | Maven dependency ordering for branch | | +1 | mvninstall | 310 | master passed | | +1 | compile | 82 | master passed | | +1 | checkstyle | 109 | master passed | | +1 | shadedjars | 267 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 62 | master passed | | 0 | spotbugs | 244 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 312 | master passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 17 | Maven dependency ordering for patch | | +1 | mvninstall | 298 | the patch passed | | +1 | compile | 81 | the patch passed | | +1 | javac | 81 | the patch passed | | +1 | checkstyle | 107 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 264 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 937 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 57 | the patch passed | | +1 | findbugs | 320 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 108 | hbase-client in the patch passed. | | +1 | unit | 9262 | hbase-server in the patch passed. | | +1 | asflicense | 60 | The patch does not generate ASF License warnings. | | | | 13119 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-510/37/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/510 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 971dfc809ec4 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-510/out/precommit/personality/provided.sh | | git revision | master / 090c55f3ff | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-510/37/testReport/ | | Max. process+thread count | 4554 (vs. ulimit of 1) | | modules | C: hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-510/37/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #494: HBASE-22380 break circle replication when doing bulkload
Apache-HBase commented on issue #494: HBASE-22380 break circle replication when doing bulkload URL: https://github.com/apache/hbase/pull/494#issuecomment-526554596 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 44 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ branch-2 Compile Tests _ | | 0 | mvndep | 14 | Maven dependency ordering for branch | | +1 | mvninstall | 331 | branch-2 passed | | +1 | compile | 123 | branch-2 passed | | +1 | checkstyle | 145 | branch-2 passed | | +1 | shadedjars | 261 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 75 | branch-2 passed | | 0 | spotbugs | 183 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 421 | branch-2 passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 17 | Maven dependency ordering for patch | | +1 | mvninstall | 302 | the patch passed | | +1 | compile | 126 | the patch passed | | +1 | cc | 126 | the patch passed | | +1 | javac | 126 | the patch passed | | +1 | checkstyle | 139 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 257 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1149 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | hbaseprotoc | 133 | the patch passed | | +1 | javadoc | 80 | the patch passed | | -1 | findbugs | 257 | hbase-server generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | +1 | unit | 51 | hbase-protocol-shaded in the patch passed. | | +1 | unit | 225 | hbase-client in the patch passed. | | -1 | unit | 10015 | hbase-server in the patch failed. | | +1 | asflicense | 83 | The patch does not generate ASF License warnings. | | | | 15014 | | | Reason | Tests | |---:|:--| | FindBugs | module:hbase-server | | | org.apache.hadoop.hbase.replication.regionserver.ReplicationSink.replicateEntries(List, CellScanner, String, String, String) makes inefficient use of keySet iterator instead of entrySet iterator At ReplicationSink.java:makes inefficient use of keySet iterator instead of entrySet iterator At ReplicationSink.java:[line 256] | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-494/41/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/494 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc | | uname | Linux 1401e96a9823 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-494/out/precommit/personality/provided.sh | | git revision | branch-2 / 4a4208f872 | | Default Java | 1.8.0_181 | | findbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-494/41/artifact/out/new-findbugs-hbase-server.html | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-494/41/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-494/41/testReport/ | | Max. process+thread count | 4653 (vs. ulimit of 1) | | modules | C: hbase-protocol-shaded hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-494/41/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Reidddddd commented on a change in pull request #528: HBASE-22890 Verify the files when RegionServer is starting and BucketCache is in file mode
Reidd commented on a change in pull request #528: HBASE-22890 Verify the
files when RegionServer is starting and BucketCache is in file mode
URL: https://github.com/apache/hbase/pull/528#discussion_r319465640
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/util/FileIOEngineUtils.java
##
@@ -0,0 +1,133 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.util;
+
+import org.apache.hadoop.util.Shell;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.ObjectInputStream;
+import java.security.MessageDigest;
+import java.security.NoSuchAlgorithmException;
+
+public final class FileIOEngineUtils {
+
+ private FileIOEngineUtils() {
+ }
+
+ /**
+ * When HBase is stoping, a key will be written to persistencePath
+ * @param persistencePath the backingMap persistence path
+ * @return the key
+ * @throws IOException something happened like file not exists
+ */
+ public static String getPreFilesKey(String persistencePath) throws
IOException {
+try (ObjectInputStream ois = new ObjectInputStream(new
FileInputStream(persistencePath))) {
+ ois.readLong();
+ String result = ois.readUTF();
+ return result;
+}
+ }
+
+ /**
+ * ReadLong from persistencePath
+ * @param persistencePath the backingMap persistence path
+ * @return -1 means has key
+ */
+ public static long readLongFormPersistencePath(String persistencePath) {
+if (persistencePath == null) {
+ return 0;
+}
+try (ObjectInputStream ois = new ObjectInputStream(new
FileInputStream(persistencePath))) {
+ Long result = ois.readLong();
+ return result;
+} catch (IOException e) {
+ return 0;
+}
+ }
+ /**
+ * delete bucketcache files
+ * @param filePaths the cache data files
+ */
+ public static void deleteCacheDataFile(String[] filePaths) {
+if (filePaths == null) {
+ return;
+}
+for (String file : filePaths) {
+ File file1 = new File(file);
+ file1.delete();
+}
+ }
+
+ /**
+ * Using an encryption algorithm to get a key, the default encryption
algorithm is MD5
+ * @param filePaths the cache data files
+ * @param algorithmName the encryption algorithm
+ * @return the key which is convert to HexString
+ * @throws IOException something happened like file not exists
+ * @throws NoSuchAlgorithmException no such algorithm
+ */
+ public static String getFilesKey(String[] filePaths, String algorithmName)
+throws IOException, NoSuchAlgorithmException {
+if (filePaths == null) {
+ return null;
+}
+StringBuffer sb = new StringBuffer();
+for (String filePath : filePaths){
+ File file = new File(filePath);
+ if (file.exists()){
+sb.append((getFileSize(filePath)+""+file.lastModified()));
+ } else {
+throw new IOException("Cache file " + filePath + " is not exists.");
+ }
+}
+MessageDigest messageDigest = MessageDigest.getInstance(algorithmName);
+messageDigest.update(sb.toString().getBytes());
+return bytes2HexString(messageDigest.digest());
+ }
+
+ /**
+ * Using Linux command du to get file's real size
+ * @param filePath the file
+ * @return file's real size
+ * @throws IOException something happened like file not exists
+ */
+ private static long getFileSize(String filePath) throws IOException {
+String[] commands = {"du",filePath};
+Shell.ShellCommandExecutor shell = new
Shell.ShellCommandExecutor(commands);
Review comment:
Why not just create a static member, every `getFileSize` creates a
`ShellCommandExecutor` is not a good practice.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #476: HBASE-11062 hbtop
Apache-HBase commented on issue #476: HBASE-11062 hbtop URL: https://github.com/apache/hbase/pull/476#issuecomment-526565973 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 48 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | 0 | shelldocs | 1 | Shelldocs was not available. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 1 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 22 new or modified test files. | ||| _ master Compile Tests _ | | 0 | mvndep | 37 | Maven dependency ordering for branch | | +1 | mvninstall | 425 | master passed | | +1 | compile | 233 | master passed | | +1 | checkstyle | 197 | master passed | | +1 | shadedjars | 352 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 214 | master passed | | 0 | spotbugs | 42 | Used deprecated FindBugs config; considering switching to SpotBugs. | | 0 | findbugs | 42 | branch/hbase-assembly no findbugs output file (findbugsXml.xml) | ||| _ Patch Compile Tests _ | | 0 | mvndep | 17 | Maven dependency ordering for patch | | +1 | mvninstall | 395 | the patch passed | | +1 | compile | 226 | the patch passed | | +1 | javac | 226 | the patch passed | | +1 | checkstyle | 183 | the patch passed | | +1 | shellcheck | 2 | There were no new shellcheck issues. | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | xml | 5 | The patch has no ill-formed XML file. | | +1 | shadedjars | 343 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1209 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 233 | the patch passed | | 0 | findbugs | 34 | hbase-assembly has no data from findbugs | ||| _ Other Tests _ | | +1 | unit | 15418 | root in the patch passed. | | +1 | asflicense | 107 | The patch does not generate ASF License warnings. | | | | 22332 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-476/62/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/476 | | Optional Tests | dupname asflicense shellcheck shelldocs javac javadoc unit shadedjars hadoopcheck xml compile spotbugs findbugs hbaseanti checkstyle | | uname | Linux ffd621a532f5 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-476/out/precommit/personality/provided.sh | | git revision | master / 090c55f3ff | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-476/62/testReport/ | | Max. process+thread count | 5679 (vs. ulimit of 1) | | modules | C: hbase-hbtop . hbase-assembly U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-476/62/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) shellcheck=0.4.4 findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #479: HBASE-22802 Avoid temp ByteBuffer allocation in FileIOEngine#read
Apache-HBase commented on issue #479: HBASE-22802 Avoid temp ByteBuffer allocation in FileIOEngine#read URL: https://github.com/apache/hbase/pull/479#issuecomment-526491254 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 38 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | 0 | mvndep | 36 | Maven dependency ordering for branch | | +1 | mvninstall | 365 | master passed | | +1 | compile | 83 | master passed | | +1 | checkstyle | 113 | master passed | | +1 | shadedjars | 288 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 60 | master passed | | 0 | spotbugs | 296 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 344 | master passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 20 | Maven dependency ordering for patch | | +1 | mvninstall | 396 | the patch passed | | +1 | compile | 95 | the patch passed | | +1 | javac | 95 | the patch passed | | +1 | checkstyle | 8 | hbase-common: The patch generated 0 new + 48 unchanged - 2 fixed = 48 total (was 50) | | +1 | checkstyle | 103 | The patch passed checkstyle in hbase-server | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 345 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1190 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 66 | the patch passed | | +1 | findbugs | 353 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 197 | hbase-common in the patch passed. | | +1 | unit | 9721 | hbase-server in the patch passed. | | +1 | asflicense | 54 | The patch does not generate ASF License warnings. | | | | 14358 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-479/62/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/479 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 186bc5956fe2 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-479/out/precommit/personality/provided.sh | | git revision | master / 090c55f3ff | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-479/62/testReport/ | | Max. process+thread count | 4659 (vs. ulimit of 1) | | modules | C: hbase-common hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-479/62/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-22760) Stop/Resume Snapshot Auto-Cleanup activity with shell command
[ https://issues.apache.org/jira/browse/HBASE-22760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated HBASE-22760: - Attachment: HBASE-22760.branch-2.000.patch > Stop/Resume Snapshot Auto-Cleanup activity with shell command > - > > Key: HBASE-22760 > URL: https://issues.apache.org/jira/browse/HBASE-22760 > Project: HBase > Issue Type: Improvement > Components: Admin, shell, snapshots >Affects Versions: 3.0.0, 1.5.0, 2.3.0 >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 1.4.11, 2.1.7 > > Attachments: HBASE-22760.branch-2.000.patch, > HBASE-22760.master.003.patch, HBASE-22760.master.004.patch, > HBASE-22760.master.005.patch, HBASE-22760.master.008.patch, > HBASE-22760.master.009.patch > > > For any scheduled snapshot backup activity, we would like to disable > auto-cleaner for snapshot based on TTL. However, as per HBASE-22648 we have a > config to disable snapshot auto-cleaner: > hbase.master.cleaner.snapshot.disable, which would take effect only upon > HMaster restart just similar to any other hbase-site configs. > For any running cluster, we should be able to stop/resume auto-cleanup > activity for snapshot based on shell command. Something similar to below > command should be able to stop/start cleanup chore: > hbase(main):001:0> snapshot_auto_cleanup_switch false (disable > auto-cleaner) > hbase(main):001:0> snapshot_auto_cleanup_switch true (enable auto-cleaner) -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [hbase] Apache-HBase commented on issue #547: HBASE-22899 logging improvements for snapshot operations w/large manifests
Apache-HBase commented on issue #547: HBASE-22899 logging improvements for snapshot operations w/large manifests URL: https://github.com/apache/hbase/pull/547#issuecomment-526501696 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 39 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 | mvninstall | 348 | master passed | | +1 | compile | 58 | master passed | | +1 | checkstyle | 77 | master passed | | +1 | shadedjars | 274 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 40 | master passed | | 0 | spotbugs | 247 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 244 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 297 | the patch passed | | +1 | compile | 56 | the patch passed | | +1 | javac | 56 | the patch passed | | +1 | checkstyle | 75 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 270 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 941 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 36 | the patch passed | | +1 | findbugs | 327 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 10395 | hbase-server in the patch passed. | | +1 | asflicense | 29 | The patch does not generate ASF License warnings. | | | | 13891 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-547/6/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/547 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux e4dcc698ff5b 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-547/out/precommit/personality/provided.sh | | git revision | master / 090c55f3ff | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-547/6/testReport/ | | Max. process+thread count | 4390 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-547/6/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] ravowlga123 commented on issue #562: HBASE-20688
ravowlga123 commented on issue #562: HBASE-20688 URL: https://github.com/apache/hbase/pull/562#issuecomment-526506138 The erroneous commit has been removed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #361: HBase-22027: Split non-MR related parts of TokenUtil off into a Clien…
Apache-HBase commented on issue #361: HBase-22027: Split non-MR related parts of TokenUtil off into a Clien… URL: https://github.com/apache/hbase/pull/361#issuecomment-526506643 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 40 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ master Compile Tests _ | | 0 | mvndep | 34 | Maven dependency ordering for branch | | +1 | mvninstall | 341 | master passed | | +1 | compile | 84 | master passed | | +1 | checkstyle | 121 | master passed | | +1 | shadedjars | 309 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 63 | master passed | | 0 | spotbugs | 233 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 305 | master passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 17 | Maven dependency ordering for patch | | +1 | mvninstall | 346 | the patch passed | | +1 | compile | 88 | the patch passed | | +1 | javac | 88 | the patch passed | | +1 | checkstyle | 129 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 303 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1037 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 57 | the patch passed | | +1 | findbugs | 373 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 120 | hbase-client in the patch passed. | | -1 | unit | 10808 | hbase-server in the patch failed. | | +1 | asflicense | 52 | The patch does not generate ASF License warnings. | | | | 15051 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-361/145/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/361 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 802e2ed61235 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-361/out/precommit/personality/provided.sh | | git revision | master / 090c55f3ff | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-361/145/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-361/145/testReport/ | | Max. process+thread count | 4833 (vs. ulimit of 1) | | modules | C: hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-361/145/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22874) Canary should not be IA.Public
[
https://issues.apache.org/jira/browse/HBASE-22874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919326#comment-16919326
]
HBase QA commented on HBASE-22874:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
3s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 4m
20s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
29s{color} | {color:red} hbase-server: The patch generated 11 new + 18
unchanged - 1 fixed = 29 total (was 19) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
0s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 13s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}224m 0s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}286m 0s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.1 Server=19.03.1 base:
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-565/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hbase/pull/565 |
| JIRA Issue | HBASE-22874 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux e07a774c2fa6 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24
10:55:24 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-565/out/precommit/personality/provided.sh
|
| git revision | master / 090c55f3ff |
| Default Java | 1.8.0_181 |
|
[GitHub] [hbase] Apache-HBase commented on issue #565: [HBASE-22874] Canary should not be IA.Public
Apache-HBase commented on issue #565: [HBASE-22874] Canary should not be IA.Public URL: https://github.com/apache/hbase/pull/565#issuecomment-526511987 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 74 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 | mvninstall | 362 | master passed | | +1 | compile | 59 | master passed | | +1 | checkstyle | 91 | master passed | | +1 | shadedjars | 303 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 39 | master passed | | 0 | spotbugs | 260 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 258 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 331 | the patch passed | | +1 | compile | 56 | the patch passed | | +1 | javac | 56 | the patch passed | | -1 | checkstyle | 89 | hbase-server: The patch generated 11 new + 18 unchanged - 1 fixed = 29 total (was 19) | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 300 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1033 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 35 | the patch passed | | +1 | findbugs | 259 | the patch passed | ||| _ Other Tests _ | | -1 | unit | 13440 | hbase-server in the patch failed. | | +1 | asflicense | 29 | The patch does not generate ASF License warnings. | | | | 17160 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-565/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/565 | | JIRA Issue | HBASE-22874 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux e07a774c2fa6 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-565/out/precommit/personality/provided.sh | | git revision | master / 090c55f3ff | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-565/1/artifact/out/diff-checkstyle-hbase-server.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-565/1/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-565/1/testReport/ | | Max. process+thread count | 4967 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-565/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Reidddddd commented on a change in pull request #528: HBASE-22890 Verify the files when RegionServer is starting and BucketCache is in file mode
Reidd commented on a change in pull request #528: HBASE-22890 Verify the
files when RegionServer is starting and BucketCache is in file mode
URL: https://github.com/apache/hbase/pull/528#discussion_r319465860
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/util/FileIOEngineUtils.java
##
@@ -0,0 +1,133 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.util;
+
+import org.apache.hadoop.util.Shell;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.ObjectInputStream;
+import java.security.MessageDigest;
+import java.security.NoSuchAlgorithmException;
+
+public final class FileIOEngineUtils {
+
+ private FileIOEngineUtils() {
+ }
+
+ /**
+ * When HBase is stoping, a key will be written to persistencePath
+ * @param persistencePath the backingMap persistence path
+ * @return the key
+ * @throws IOException something happened like file not exists
+ */
+ public static String getPreFilesKey(String persistencePath) throws
IOException {
+try (ObjectInputStream ois = new ObjectInputStream(new
FileInputStream(persistencePath))) {
+ ois.readLong();
+ String result = ois.readUTF();
+ return result;
+}
+ }
+
+ /**
+ * ReadLong from persistencePath
+ * @param persistencePath the backingMap persistence path
+ * @return -1 means has key
+ */
+ public static long readLongFormPersistencePath(String persistencePath) {
+if (persistencePath == null) {
+ return 0;
+}
+try (ObjectInputStream ois = new ObjectInputStream(new
FileInputStream(persistencePath))) {
+ Long result = ois.readLong();
+ return result;
+} catch (IOException e) {
+ return 0;
+}
+ }
+ /**
+ * delete bucketcache files
+ * @param filePaths the cache data files
+ */
+ public static void deleteCacheDataFile(String[] filePaths) {
+if (filePaths == null) {
+ return;
+}
+for (String file : filePaths) {
+ File file1 = new File(file);
+ file1.delete();
+}
+ }
+
+ /**
+ * Using an encryption algorithm to get a key, the default encryption
algorithm is MD5
+ * @param filePaths the cache data files
+ * @param algorithmName the encryption algorithm
+ * @return the key which is convert to HexString
+ * @throws IOException something happened like file not exists
+ * @throws NoSuchAlgorithmException no such algorithm
+ */
+ public static String getFilesKey(String[] filePaths, String algorithmName)
+throws IOException, NoSuchAlgorithmException {
+if (filePaths == null) {
+ return null;
+}
+StringBuffer sb = new StringBuffer();
+for (String filePath : filePaths){
+ File file = new File(filePath);
+ if (file.exists()){
+sb.append((getFileSize(filePath)+""+file.lastModified()));
+ } else {
+throw new IOException("Cache file " + filePath + " is not exists.");
+ }
+}
+MessageDigest messageDigest = MessageDigest.getInstance(algorithmName);
+messageDigest.update(sb.toString().getBytes());
+return bytes2HexString(messageDigest.digest());
Review comment:
byte[] to string, then string to byte[]? why not just return byte[]
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Reidddddd commented on a change in pull request #528: HBASE-22890 Verify the files when RegionServer is starting and BucketCache is in file mode
Reidd commented on a change in pull request #528: HBASE-22890 Verify the
files when RegionServer is starting and BucketCache is in file mode
URL: https://github.com/apache/hbase/pull/528#discussion_r319466015
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/bucket/TestVerifyBucketCacheFile.java
##
@@ -0,0 +1,268 @@
+/**
+ * Copyright The Apache Software Foundation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.hadoop.hbase.io.hfile.bucket;
+
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.HBaseTestingUtility;
+import org.apache.hadoop.hbase.io.hfile.BlockCacheKey;
+import org.apache.hadoop.hbase.io.hfile.CacheTestUtils;
+import org.apache.hadoop.hbase.io.hfile.Cacheable;
+import org.apache.hadoop.hbase.testclassification.SmallTests;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import java.io.BufferedWriter;
+import java.io.File;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.ObjectOutputStream;
+import java.io.OutputStreamWriter;
+import java.security.NoSuchAlgorithmException;
+import java.util.Arrays;
+import java.util.concurrent.ConcurrentMap;
+
+import static org.junit.Assert.assertTrue;
+import static org.junit.Assert.assertEquals;
+
+@RunWith(Parameterized.class) @Category(SmallTests.class) public class
TestVerifyBucketCacheFile {
Review comment:
nit, new line.
Doc for the this test.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Reidddddd commented on a change in pull request #528: HBASE-22890 Verify the files when RegionServer is starting and BucketCache is in file mode
Reidd commented on a change in pull request #528: HBASE-22890 Verify the
files when RegionServer is starting and BucketCache is in file mode
URL: https://github.com/apache/hbase/pull/528#discussion_r319466169
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/bucket/TestVerifyBucketCacheFile.java
##
@@ -0,0 +1,268 @@
+/**
+ * Copyright The Apache Software Foundation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.hadoop.hbase.io.hfile.bucket;
+
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.HBaseTestingUtility;
+import org.apache.hadoop.hbase.io.hfile.BlockCacheKey;
+import org.apache.hadoop.hbase.io.hfile.CacheTestUtils;
+import org.apache.hadoop.hbase.io.hfile.Cacheable;
+import org.apache.hadoop.hbase.testclassification.SmallTests;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import java.io.BufferedWriter;
+import java.io.File;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.ObjectOutputStream;
+import java.io.OutputStreamWriter;
+import java.security.NoSuchAlgorithmException;
+import java.util.Arrays;
+import java.util.concurrent.ConcurrentMap;
+
+import static org.junit.Assert.assertTrue;
+import static org.junit.Assert.assertEquals;
+
+@RunWith(Parameterized.class) @Category(SmallTests.class) public class
TestVerifyBucketCacheFile {
+ @Parameterized.Parameters(name = "{index}: blockSize={0}, bucketSizes={1}")
+ public static Iterable data() {
+return Arrays.asList(new Object[][] { { 8192, null }, { 16 * 1024,
+new int[] { 2 * 1024 + 1024, 4 * 1024 + 1024, 8 * 1024 + 1024, 16 *
1024 + 1024,
+28 * 1024 + 1024, 32 * 1024 + 1024, 64 * 1024 + 1024, 96 * 1024 + 1024,
+128 * 1024 + 1024 } } });
+ }
+
+ @Parameterized.Parameter(0)
+ public int constructedBlockSize;
+
+ @Parameterized.Parameter(1)
+ public int[] constructedBlockSizes;
+
+ final long capacitySize = 32 * 1024 * 1024;
+ final int writeThreads = BucketCache.DEFAULT_WRITER_THREADS;
+ final int writerQLen = BucketCache.DEFAULT_WRITER_QUEUE_ITEMS;
+
+ @Test public void testRetrieveFromFile() throws Exception {
Review comment:
nit, new line style.
Doc for each test method.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-22928) ScanMetrics counter update may not happen in case of exception in TableRecordReaderImpl
[
https://issues.apache.org/jira/browse/HBASE-22928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919341#comment-16919341
]
Pankaj Kumar commented on HBASE-22928:
--
Thanks for reviewing [~reidchan]...
> ScanMetrics counter update may not happen in case of exception in
> TableRecordReaderImpl
> ---
>
> Key: HBASE-22928
> URL: https://issues.apache.org/jira/browse/HBASE-22928
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Minor
>
> ScanMetrics counter update may not happen in case of exception in
> TableRecordReaderImpl.nextKeyValue(),
> There are multiple cases as below,
> 1. In case of DoNotRetryIOException
> {code}
> // do not retry if the exception tells us not to do so
> if (e instanceof DoNotRetryIOException) {
> throw e;
> }
> {code}
> 2. While creating a new scanner,
> {code}
> if (lastSuccessfulRow == null) {
> restart(scan.getStartRow());
> } else {
> restart(lastSuccessfulRow);
> scanner.next(); // skip presumed already mapped row
> }
> {code}
> 3. In case of IOException
> {code}
> catch (IOException ioe) {
> if (logScannerActivity) {
> long now = System.currentTimeMillis();
> LOG.info("Mapper took " + (now-timestamp)
> + "ms to process " + rowcount + " rows");
> LOG.info(ioe.toString(), ioe);
> String lastRow = lastSuccessfulRow == null ?
> "null" : Bytes.toStringBinary(lastSuccessfulRow);
> LOG.info("lastSuccessfulRow=" + lastRow);
> }
> throw ioe;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [hbase] Reidddddd commented on a change in pull request #528: HBASE-22890 Verify the files when RegionServer is starting and BucketCache is in file mode
Reidd commented on a change in pull request #528: HBASE-22890 Verify the
files when RegionServer is starting and BucketCache is in file mode
URL: https://github.com/apache/hbase/pull/528#discussion_r319440808
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/FileIOEngine.java
##
@@ -59,6 +61,43 @@ public FileIOEngine(long capacity, String... filePaths)
throws IOException {
this.fileChannels = new FileChannel[filePaths.length];
this.rafs = new RandomAccessFile[filePaths.length];
this.channelLocks = new ReentrantLock[filePaths.length];
+init();
+ }
+
+ public FileIOEngine(String algorithmName, String persistencePath,
+long capacity, String... filePaths) throws IOException {
+this.sizePerFile = capacity / filePaths.length;
+this.capacity = this.sizePerFile * filePaths.length;
+this.filePaths = filePaths;
+this.fileChannels = new FileChannel[filePaths.length];
+this.rafs = new RandomAccessFile[filePaths.length];
+this.channelLocks = new ReentrantLock[filePaths.length];
+if (persistencePath != null){
+ try {
+String preFilesKey = FileIOEngineUtils.getPreFilesKey(persistencePath);
+String filesKey = FileIOEngineUtils.getFilesKey(filePaths,
algorithmName);
+LOG.info("preFilesKey: "+ preFilesKey + " filesKey: " + filesKey);
Review comment:
More verbose language for logging.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Reidddddd commented on a change in pull request #528: HBASE-22890 Verify the files when RegionServer is starting and BucketCache is in file mode
Reidd commented on a change in pull request #528: HBASE-22890 Verify the
files when RegionServer is starting and BucketCache is in file mode
URL: https://github.com/apache/hbase/pull/528#discussion_r319440717
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
##
@@ -1597,4 +1631,12 @@ float getMultiFactor() {
float getMemoryFactor() {
return memoryFactor;
}
+
+ public String getAlgorithm() {
+return algorithm;
+ }
+
+ public UniqueIndexMap getDeserialiserMap() {
Review comment:
Ditto
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-22945) Show quota infos in master UI
[ https://issues.apache.org/jira/browse/HBASE-22945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919450#comment-16919450 ] Yi Mei commented on HBASE-22945: [~jatsakthi] I think this page is used to show all quota infos, feel free to show space quotas if they are useful. > Show quota infos in master UI > - > > Key: HBASE-22945 > URL: https://issues.apache.org/jira/browse/HBASE-22945 > Project: HBase > Issue Type: Sub-task >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Attachments: 1.png, 2.png > > > Add a page in master UI to show the following quota infos: > if rpc throttle is enabled; > if exceed throttle quota is enabled; > namespace throtlles; > user throttles. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [hbase] shardul-cr7 commented on issue #559: HBASE-22944 TableNotFoundException: hbase:quota is thrown when region server is restarted.
shardul-cr7 commented on issue #559: HBASE-22944 TableNotFoundException: hbase:quota is thrown when region server is restarted. URL: https://github.com/apache/hbase/pull/559#issuecomment-526578747 Hi @jatsakthi, Can you review this? :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22874) Canary should not be IA.Public
[
https://issues.apache.org/jira/browse/HBASE-22874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919627#comment-16919627
]
HBase QA commented on HBASE-22874:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
40s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
45s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 4m
14s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
19s{color} | {color:red} hbase-server: The patch generated 11 new + 18
unchanged - 1 fixed = 29 total (was 19) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
42s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
15m 46s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}155m
52s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}213m 31s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.1 Server=19.03.1 base:
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-565/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hbase/pull/565 |
| JIRA Issue | HBASE-22874 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 74c7edf9db91 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-565/out/precommit/personality/provided.sh
|
| git revision | master / 090c55f3ff |
| Default Java | 1.8.0_181 |
[GitHub] [hbase] Apache-HBase commented on issue #565: [HBASE-22874] Canary should not be IA.Public
Apache-HBase commented on issue #565: [HBASE-22874] Canary should not be IA.Public URL: https://github.com/apache/hbase/pull/565#issuecomment-526637259 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 40 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 | mvninstall | 333 | master passed | | +1 | compile | 59 | master passed | | +1 | checkstyle | 82 | master passed | | +1 | shadedjars | 285 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 41 | master passed | | 0 | spotbugs | 254 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 251 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 309 | the patch passed | | +1 | compile | 57 | the patch passed | | +1 | javac | 57 | the patch passed | | -1 | checkstyle | 79 | hbase-server: The patch generated 11 new + 18 unchanged - 1 fixed = 29 total (was 19) | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 282 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 946 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 34 | the patch passed | | +1 | findbugs | 258 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 9352 | hbase-server in the patch passed. | | +1 | asflicense | 29 | The patch does not generate ASF License warnings. | | | | 12811 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-565/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/565 | | JIRA Issue | HBASE-22874 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 74c7edf9db91 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-565/out/precommit/personality/provided.sh | | git revision | master / 090c55f3ff | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-565/2/artifact/out/diff-checkstyle-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-565/2/testReport/ | | Max. process+thread count | 4359 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-565/2/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase-operator-tools] petersomogyi opened a new pull request #20: HBASE-22952 HBCK2 replication command is incompatible with 2.0.x
petersomogyi opened a new pull request #20: HBASE-22952 HBCK2 replication command is incompatible with 2.0.x URL: https://github.com/apache/hbase-operator-tools/pull/20 Adds supported versions to the `replication` command. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #552: HBASE-22893
Apache-HBase commented on issue #552: HBASE-22893 URL: https://github.com/apache/hbase/pull/552#issuecomment-526652865 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 41 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 | mvninstall | 341 | master passed | | 0 | refguide | 349 | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 301 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | 0 | refguide | 359 | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | ||| _ Other Tests _ | | +1 | asflicense | 18 | The patch does not generate ASF License warnings. | | | | 1472 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-552/5/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/552 | | JIRA Issue | HBASE-22893 | | Optional Tests | dupname asflicense refguide | | uname | Linux ffed4e19f444 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-552/out/precommit/personality/provided.sh | | git revision | master / 090c55f3ff | | refguide | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-552/5/artifact/out/branch-site/book.html | | refguide | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-552/5/artifact/out/patch-site/book.html | | Max. process+thread count | 96 (vs. ulimit of 1) | | modules | C: . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-552/5/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Work started] (HBASE-22952) HBCK2 replication command is incompatible with 2.0.x
[
https://issues.apache.org/jira/browse/HBASE-22952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-22952 started by Peter Somogyi.
-
> HBCK2 replication command is incompatible with 2.0.x
>
>
> Key: HBASE-22952
> URL: https://issues.apache.org/jira/browse/HBASE-22952
> Project: HBase
> Issue Type: Bug
> Components: hbase-operator-tools
>Affects Versions: operator-1.0.0
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Critical
> Fix For: operator-1.0.0
>
>
> HBCK2 does not have a valid version check for `replication` command on 2.0
> releases.
> This command relies on ReplicationQueueStorage and ReplicationStorageFactory
> classes which were added in 2.1.0 so running HBCK2 replication on a 2.0
> cluster causes runtime error.
> {noformat}
> $ bin/hbase hbck -j hbase-hbck2-1.0.0-SNAPSHOT.jar replication
> 17:36:16.362 [main] WARN org.apache.hadoop.util.NativeCodeLoader - Unable to
> load native-hadoop library for your platform... using builtin-java classes
> where applicable
> 17:36:16.634 [main] INFO org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient
> - Connect 0x201a4587 to localhost:2181 with session timeout=9ms, retries
> 30, retry interval 1000ms, keepAlive=6ms
> 17:36:17.246 [main] INFO
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper - Process
> identifier=HBCK2 connecting to ZooKeeper ensemble=localhost:2181
> 17:36:17.253 [main] INFO
> org.apache.hadoop.hbase.client.ConnectionImplementation - Closing master
> protocol: MasterService
> 17:36:17.253 [main] INFO org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient
> - Close zookeeper connection 0x201a4587 to localhost:2181
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/hbase/replication/ReplicationStorageFactory
> at
> org.apache.hbase.hbck1.ReplicationChecker.(ReplicationChecker.java:59)
> at
> org.apache.hbase.hbck1.HBaseFsck.checkAndFixReplication(HBaseFsck.java:3772)
> at org.apache.hbase.ReplicationFsck.fsck(ReplicationFsck.java:65)
> at org.apache.hbase.HBCK2.doCommandLine(HBCK2.java:571)
> at org.apache.hbase.HBCK2.run(HBCK2.java:470)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at org.apache.hbase.HBCK2.main(HBCK2.java:620)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.replication.ReplicationStorageFactory
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 8 more
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [hbase] Apache-HBase commented on issue #561: HBASE-22946 Fix TableNotFound when grant/revoke if AccessController is not loaded
Apache-HBase commented on issue #561: HBASE-22946 Fix TableNotFound when grant/revoke if AccessController is not loaded URL: https://github.com/apache/hbase/pull/561#issuecomment-526593350 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 46 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | +1 | mvninstall | 421 | master passed | | +1 | compile | 70 | master passed | | +1 | checkstyle | 101 | master passed | | +1 | shadedjars | 358 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 44 | master passed | | 0 | spotbugs | 286 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 283 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 381 | the patch passed | | +1 | compile | 70 | the patch passed | | +1 | javac | 70 | the patch passed | | +1 | checkstyle | 94 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 343 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1214 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 46 | the patch passed | | +1 | findbugs | 344 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 9703 | hbase-server in the patch passed. | | +1 | asflicense | 29 | The patch does not generate ASF License warnings. | | | | 14021 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-561/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/561 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 2c525f2f7fb5 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-561/out/precommit/personality/provided.sh | | git revision | master / 090c55f3ff | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-561/4/testReport/ | | Max. process+thread count | 4785 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-561/4/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22939) SpaceQuotas- Bulkload from different hdfs failed when space quotas are turned on.
[
https://issues.apache.org/jira/browse/HBASE-22939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919531#comment-16919531
]
HBase QA commented on HBASE-22939:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
54s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
52s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
52s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
16m 50s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}252m 2s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
48s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}312m 21s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestTags |
| | hadoop.hbase.master.procedure.TestSCPWithReplicas |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.1 Server=19.03.1 base:
https://builds.apache.org/job/PreCommit-HBASE-Build/847/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-22939 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12978950/HBASE-22939-v2.patch |
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux a43a91d471dc 4.15.0-55-generic #60-Ubuntu SMP Tue Jul 2
18:22:20 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 090c55f3ff |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.11 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/847/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/847/testReport/ |
| Max. process+thread count | 4898 (vs. ulimit of 1) |
| modules | C: hbase-server U:
[jira] [Commented] (HBASE-22933) Do not need to kick reassign for rs group change any more
[
https://issues.apache.org/jira/browse/HBASE-22933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919533#comment-16919533
]
Hudson commented on HBASE-22933:
Results for branch master
[build #1377 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1377/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1377//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1377//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1377//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Do not need to kick reassign for rs group change any more
> -
>
> Key: HBASE-22933
> URL: https://issues.apache.org/jira/browse/HBASE-22933
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.2.2
>
>
> The old implementation is a bit strange, the isStuck method is like this:
> {code}
> public boolean isStuck() {
> return isInState(State.FAILED_OPEN) && getProcedure() != null;
> }
> {code}
> It can only return true when there are ongoing procedures. But if we have a
> procedure, then the procedure will try to reassign region. Scheduling a new
> procedure does not make sense here, at least for branch-2.2+.
> I suggest we just remove the related code, since the default retry number for
> assigning a region is Integer.MAX_VALUE. And even if user set this to small
> value and finally the region is left in FAILED_OPEN state without a
> procedure, HBCK2 is used to deal with this, it is not necessary to deal it
> automatically.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [hbase] Apache-HBase commented on issue #547: HBASE-22899 logging improvements for snapshot operations w/large manifests
Apache-HBase commented on issue #547: HBASE-22899 logging improvements for snapshot operations w/large manifests URL: https://github.com/apache/hbase/pull/547#issuecomment-526618662 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 44 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 | mvninstall | 421 | master passed | | +1 | compile | 71 | master passed | | +1 | checkstyle | 107 | master passed | | +1 | shadedjars | 351 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 46 | master passed | | 0 | spotbugs | 311 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 309 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 352 | the patch passed | | +1 | compile | 78 | the patch passed | | +1 | javac | 78 | the patch passed | | +1 | checkstyle | 110 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 357 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1116 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 40 | the patch passed | | +1 | findbugs | 302 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 9655 | hbase-server in the patch passed. | | +1 | asflicense | 25 | The patch does not generate ASF License warnings. | | | | 13796 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-547/7/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/547 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux af18517388e1 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-547/out/precommit/personality/provided.sh | | git revision | master / 090c55f3ff | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-547/7/testReport/ | | Max. process+thread count | 4295 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-547/7/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] ravowlga123 commented on issue #552: HBASE-22893
ravowlga123 commented on issue #552: HBASE-22893 URL: https://github.com/apache/hbase/pull/552#issuecomment-526618615 Hi @jatsakthi I have made the changes. Please do review it if they are okay and thank you for your patience. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #528: HBASE-22890 Verify the files when RegionServer is starting and BucketCache is in file mode
Apache-HBase commented on issue #528: HBASE-22890 Verify the files when RegionServer is starting and BucketCache is in file mode URL: https://github.com/apache/hbase/pull/528#issuecomment-526620860 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 54 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 1 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ branch-1 Compile Tests _ | | +1 | mvninstall | 490 | branch-1 passed | | +1 | compile | 44 | branch-1 passed | | +1 | checkstyle | 114 | branch-1 passed | | -1 | shadedjars | 15 | branch has 10 errors when building our shaded downstream artifacts. | | +1 | javadoc | 52 | branch-1 passed | | 0 | spotbugs | 159 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 157 | branch-1 passed | | -0 | patch | 167 | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 116 | the patch passed | | +1 | compile | 42 | the patch passed | | +1 | javac | 42 | the patch passed | | -1 | checkstyle | 114 | hbase-server: The patch generated 10 new + 51 unchanged - 0 fixed = 61 total (was 51) | | +1 | whitespace | 0 | The patch has no whitespace issues. | | -1 | shadedjars | 13 | patch has 10 errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 303 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 | javadoc | 29 | the patch passed | | +1 | findbugs | 143 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 6629 | hbase-server in the patch passed. | | +1 | asflicense | 31 | The patch does not generate ASF License warnings. | | | | 8544 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-528/19/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/528 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 69a7f853106a 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-528/out/precommit/personality/provided.sh | | git revision | branch-1 / 73e0304 | | Default Java | 1.8.0_222 | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-528/19/artifact/out/branch-shadedjars.txt | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-528/19/artifact/out/diff-checkstyle-hbase-server.txt | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-528/19/artifact/out/patch-shadedjars.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-528/19/testReport/ | | Max. process+thread count | 4322 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-528/19/console | | versions | git=1.9.1 maven=3.0.5 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] ravowlga123 commented on a change in pull request #564: HBASE-22900
ravowlga123 commented on a change in pull request #564: HBASE-22900 URL: https://github.com/apache/hbase/pull/564#discussion_r319532216 ## File path: pom.xml ## @@ -1276,6 +1276,7 @@ 3.2.2 3.1 4.4.4 +4.5.2 Review comment: There were three different versions of org.apache.httpcomponents:httpclient in branch-1.4 namely: 4.1.2, 4.2.5 and 4.5.2. Should I use one of the other two versions? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22933) Do not need to kick reassign for rs group change any more
[
https://issues.apache.org/jira/browse/HBASE-22933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919587#comment-16919587
]
Hudson commented on HBASE-22933:
Results for branch branch-2.2
[build #555 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/555/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/555//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/555//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/555//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 3. [see log for
details|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/555//artifact/output-integration/hadoop-3.log].
(note that this means we didn't check the Hadoop 3 shaded client)
> Do not need to kick reassign for rs group change any more
> -
>
> Key: HBASE-22933
> URL: https://issues.apache.org/jira/browse/HBASE-22933
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.2.2
>
>
> The old implementation is a bit strange, the isStuck method is like this:
> {code}
> public boolean isStuck() {
> return isInState(State.FAILED_OPEN) && getProcedure() != null;
> }
> {code}
> It can only return true when there are ongoing procedures. But if we have a
> procedure, then the procedure will try to reassign region. Scheduling a new
> procedure does not make sense here, at least for branch-2.2+.
> I suggest we just remove the related code, since the default retry number for
> assigning a region is Integer.MAX_VALUE. And even if user set this to small
> value and finally the region is left in FAILED_OPEN state without a
> procedure, HBCK2 is used to deal with this, it is not necessary to deal it
> automatically.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [hbase] Apache-HBase commented on issue #361: HBase-22027: Split non-MR related parts of TokenUtil off into a Clien…
Apache-HBase commented on issue #361: HBase-22027: Split non-MR related parts of TokenUtil off into a Clien… URL: https://github.com/apache/hbase/pull/361#issuecomment-526632239 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 52 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ master Compile Tests _ | | 0 | mvndep | 68 | Maven dependency ordering for branch | | +1 | mvninstall | 335 | master passed | | +1 | compile | 86 | master passed | | +1 | checkstyle | 120 | master passed | | +1 | shadedjars | 289 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 59 | master passed | | 0 | spotbugs | 240 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 309 | master passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 18 | Maven dependency ordering for patch | | +1 | mvninstall | 306 | the patch passed | | +1 | compile | 84 | the patch passed | | +1 | javac | 84 | the patch passed | | +1 | checkstyle | 119 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 301 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 931 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 58 | the patch passed | | +1 | findbugs | 320 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 114 | hbase-client in the patch passed. | | +1 | unit | 9319 | hbase-server in the patch passed. | | +1 | asflicense | 60 | The patch does not generate ASF License warnings. | | | | 13343 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-361/146/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/361 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux f2d700eaa3ed 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-361/out/precommit/personality/provided.sh | | git revision | master / 090c55f3ff | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-361/146/testReport/ | | Max. process+thread count | 4403 (vs. ulimit of 1) | | modules | C: hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-361/146/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22933) Do not need to kick reassign for rs group change any more
[
https://issues.apache.org/jira/browse/HBASE-22933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919643#comment-16919643
]
Hudson commented on HBASE-22933:
Results for branch branch-2
[build #2207 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2207/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2207//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2207//console].
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2207//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Do not need to kick reassign for rs group change any more
> -
>
> Key: HBASE-22933
> URL: https://issues.apache.org/jira/browse/HBASE-22933
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.2.2
>
>
> The old implementation is a bit strange, the isStuck method is like this:
> {code}
> public boolean isStuck() {
> return isInState(State.FAILED_OPEN) && getProcedure() != null;
> }
> {code}
> It can only return true when there are ongoing procedures. But if we have a
> procedure, then the procedure will try to reassign region. Scheduling a new
> procedure does not make sense here, at least for branch-2.2+.
> I suggest we just remove the related code, since the default retry number for
> assigning a region is Integer.MAX_VALUE. And even if user set this to small
> value and finally the region is left in FAILED_OPEN state without a
> procedure, HBCK2 is used to deal with this, it is not necessary to deal it
> automatically.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [hbase] busbey commented on a change in pull request #565: [HBASE-22874] Canary should not be IA.Public
busbey commented on a change in pull request #565: [HBASE-22874] Canary should
not be IA.Public
URL: https://github.com/apache/hbase/pull/565#discussion_r319562324
##
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/tool/Canary.java
##
@@ -1629,7 +1684,7 @@ public static void main(String[] args) throws Exception {
final Configuration conf = HBaseConfiguration.create();
// Loading the generic options to conf
-new GenericOptionsParser(conf, args);
+//new GenericOptionsParser(conf, args);
Review comment:
why aren't we parsing generic options anymore?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] busbey commented on a change in pull request #565: [HBASE-22874] Canary should not be IA.Public
busbey commented on a change in pull request #565: [HBASE-22874] Canary should
not be IA.Public
URL: https://github.com/apache/hbase/pull/565#discussion_r319562097
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/tool/CanaryInterface.java
##
@@ -0,0 +1,40 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.tool;
+
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.yetus.audience.InterfaceAudience;
+
+import java.util.concurrent.ExecutorService;
+
+@InterfaceAudience.Public
+public interface CanaryInterface {
Review comment:
What do folks think about having the interface be `Canary` and the tool be
`CanaryTool` or `CanaryImpl`?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] busbey commented on a change in pull request #565: [HBASE-22874] Canary should not be IA.Public
busbey commented on a change in pull request #565: [HBASE-22874] Canary should
not be IA.Public
URL: https://github.com/apache/hbase/pull/565#discussion_r319561895
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/tool/CanaryInterface.java
##
@@ -0,0 +1,40 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.tool;
+
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.yetus.audience.InterfaceAudience;
+
+import java.util.concurrent.ExecutorService;
+
+@InterfaceAudience.Public
+public interface CanaryInterface {
+
+ static CanaryInterface create(Configuration conf, ExecutorService executor) {
+return new Canary(conf);
Review comment:
we don't use the executor?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] busbey commented on a change in pull request #565: [HBASE-22874] Canary should not be IA.Public
busbey commented on a change in pull request #565: [HBASE-22874] Canary should
not be IA.Public
URL: https://github.com/apache/hbase/pull/565#discussion_r319562878
##
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/tool/Canary.java
##
@@ -843,9 +876,9 @@ public int run(String[] args) throws Exception {
}
}
currentTimeLength = System.currentTimeMillis() - startTime;
-if (currentTimeLength > this.timeout) {
+if (currentTimeLength > timeout) {
Review comment:
nit: avoid unrelated style changes like removing the `this` variables
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Created] (HBASE-22952) HBCK2 replication command is incompatible with 2.0.x
Peter Somogyi created HBASE-22952:
-
Summary: HBCK2 replication command is incompatible with 2.0.x
Key: HBASE-22952
URL: https://issues.apache.org/jira/browse/HBASE-22952
Project: HBase
Issue Type: Bug
Components: hbase-operator-tools
Affects Versions: operator-1.0.0
Reporter: Peter Somogyi
Assignee: Peter Somogyi
Fix For: operator-1.0.0
HBCK2 does not have a valid version check for `replication` command on 2.0
releases.
This command relies on ReplicationQueueStorage and ReplicationStorageFactory
classes which were added in 2.1.0 so running HBCK2 replication on a 2.0 cluster
causes runtime error.
{noformat}
$ bin/hbase hbck -j hbase-hbck2-1.0.0-SNAPSHOT.jar replication
17:36:16.362 [main] WARN org.apache.hadoop.util.NativeCodeLoader - Unable to
load native-hadoop library for your platform... using builtin-java classes
where applicable
17:36:16.634 [main] INFO org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient -
Connect 0x201a4587 to localhost:2181 with session timeout=9ms, retries 30,
retry interval 1000ms, keepAlive=6ms
17:36:17.246 [main] INFO
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper - Process
identifier=HBCK2 connecting to ZooKeeper ensemble=localhost:2181
17:36:17.253 [main] INFO
org.apache.hadoop.hbase.client.ConnectionImplementation - Closing master
protocol: MasterService
17:36:17.253 [main] INFO org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient -
Close zookeeper connection 0x201a4587 to localhost:2181
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/hadoop/hbase/replication/ReplicationStorageFactory
at
org.apache.hbase.hbck1.ReplicationChecker.(ReplicationChecker.java:59)
at
org.apache.hbase.hbck1.HBaseFsck.checkAndFixReplication(HBaseFsck.java:3772)
at org.apache.hbase.ReplicationFsck.fsck(ReplicationFsck.java:65)
at org.apache.hbase.HBCK2.doCommandLine(HBCK2.java:571)
at org.apache.hbase.HBCK2.run(HBCK2.java:470)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.hbase.HBCK2.main(HBCK2.java:620)
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.hbase.replication.ReplicationStorageFactory
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 8 more
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HBASE-22724) Add a emoji on the vote table for pre commit result on github
[ https://issues.apache.org/jira/browse/HBASE-22724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919589#comment-16919589 ] Sean Busbey commented on HBASE-22724: - [yetus 0.11.0 is out now with the emoji votes in place|https://s.apache.org/yetus-0.11.0-announce] > Add a emoji on the vote table for pre commit result on github > - > > Key: HBASE-22724 > URL: https://issues.apache.org/jira/browse/HBASE-22724 > Project: HBase > Issue Type: Improvement >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > [~zghaobac] said that the current vote table on github is not good enough, as > the colors are almost the same, it is not easy to find out which line is > broken. > Since github can not change the color of the text, he suggested that we add a > column at the left most with some emojis to better notify the developpers. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [hbase] jatsakthi commented on issue #552: HBASE-22893
jatsakthi commented on issue #552: HBASE-22893 URL: https://github.com/apache/hbase/pull/552#issuecomment-526606570 Yes, please. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] wchevreuil opened a new pull request #566: HBASE-22380 break circle replication when doing bulkload
wchevreuil opened a new pull request #566: HBASE-22380 break circle replication when doing bulkload URL: https://github.com/apache/hbase/pull/566 PR for master branch This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22380) break circle replication when doing bulkload
[ https://issues.apache.org/jira/browse/HBASE-22380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919624#comment-16919624 ] Wellington Chevreuil commented on HBASE-22380: -- Thanks [~anoop.hbase]. Had create another PR for master branch. Please let me know if this looks good to go. > break circle replication when doing bulkload > > > Key: HBASE-22380 > URL: https://issues.apache.org/jira/browse/HBASE-22380 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 3.0.0, 1.5.0, 2.2.0, 1.4.10, 2.0.5, 2.3.0, 2.1.5, 1.3.5 >Reporter: chenxu >Assignee: Wellington Chevreuil >Priority: Critical > Labels: bulkload > Fix For: 3.0.0, 1.5.0, 2.3.0, 1.4.11, 2.1.7, 2.2.2 > > > when enabled master-master bulkload replication, HFiles will be replicated > circularly between two clusters -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [hbase] Apache-HBase commented on issue #479: HBASE-22802 Avoid temp ByteBuffer allocation in FileIOEngine#read
Apache-HBase commented on issue #479: HBASE-22802 Avoid temp ByteBuffer allocation in FileIOEngine#read URL: https://github.com/apache/hbase/pull/479#issuecomment-526640027 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 73 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | 0 | mvndep | 32 | Maven dependency ordering for branch | | +1 | mvninstall | 341 | master passed | | +1 | compile | 81 | master passed | | +1 | checkstyle | 116 | master passed | | +1 | shadedjars | 300 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 57 | master passed | | 0 | spotbugs | 262 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 309 | master passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 17 | Maven dependency ordering for patch | | +1 | mvninstall | 328 | the patch passed | | +1 | compile | 79 | the patch passed | | +1 | javac | 79 | the patch passed | | +1 | checkstyle | 27 | hbase-common: The patch generated 0 new + 48 unchanged - 2 fixed = 48 total (was 50) | | +1 | checkstyle | 88 | The patch passed checkstyle in hbase-server | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 298 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1034 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 55 | the patch passed | | +1 | findbugs | 320 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 176 | hbase-common in the patch passed. | | -1 | unit | 15563 | hbase-server in the patch failed. | | +1 | asflicense | 50 | The patch does not generate ASF License warnings. | | | | 19765 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.replication.regionserver.TestWALEntryStream | | | hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaSameHosts | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-479/63/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/479 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux e3792c069231 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-479/out/precommit/personality/provided.sh | | git revision | master / 090c55f3ff | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-479/63/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-479/63/testReport/ | | Max. process+thread count | 4345 (vs. ulimit of 1) | | modules | C: hbase-common hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-479/63/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22893) Change the comment in HBaseClassTestRule to reflect change in default test timeouts
[
https://issues.apache.org/jira/browse/HBASE-22893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919667#comment-16919667
]
HBase QA commented on HBASE-22893:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
41s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
41s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 5m
49s{color} | {color:blue} branch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 5m
59s{color} | {color:blue} patch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 32s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.1 Server=19.03.1 base:
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-552/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hbase/pull/552 |
| JIRA Issue | HBASE-22893 |
| Optional Tests | dupname asflicense refguide |
| uname | Linux ffed4e19f444 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-552/out/precommit/personality/provided.sh
|
| git revision | master / 090c55f3ff |
| refguide |
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-552/5/artifact/out/branch-site/book.html
|
| refguide |
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-552/5/artifact/out/patch-site/book.html
|
| Max. process+thread count | 96 (vs. ulimit of 1) |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-552/5/console |
| versions | git=2.11.0 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.10.0 http://yetus.apache.org |
This message was automatically generated.
> Change the comment in HBaseClassTestRule to reflect change in default test
> timeouts
> ---
>
> Key: HBASE-22893
> URL: https://issues.apache.org/jira/browse/HBASE-22893
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Rabi Kumar K C
>Priority: Trivial
> Labels: beginner, noob
> Fix For: 3.0.0, 2.3.0, 2.2.1, 2.1.6
>
> Attachments: HBASE-22893.patch
>
>
> The default timeout is 13 minutes. Change the comment accordingly.
> {code}
> // All tests have a 10minute timeout.
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HBASE-22514) Move rsgroup feature into core of HBase
[
https://issues.apache.org/jira/browse/HBASE-22514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919510#comment-16919510
]
Hudson commented on HBASE-22514:
Results for branch HBASE-22514
[build #82 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/82/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/82//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/82//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/82//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Move rsgroup feature into core of HBase
> ---
>
> Key: HBASE-22514
> URL: https://issues.apache.org/jira/browse/HBASE-22514
> Project: HBase
> Issue Type: Umbrella
> Components: Admin, Client, rsgroup
>Reporter: Yechao Chen
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HBASE-22514.master.001.patch,
> image-2019-05-31-18-25-38-217.png
>
>
> The class RSGroupAdminClient is not public
> we need to use java api RSGroupAdminClient to manager RSG
> so RSGroupAdminClient should be public
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [hbase] Apache-HBase commented on issue #551: HBASE-22896 TestHRegion.testFlushMarkersWALFail is flaky
Apache-HBase commented on issue #551: HBASE-22896 TestHRegion.testFlushMarkersWALFail is flaky URL: https://github.com/apache/hbase/pull/551#issuecomment-526606023 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 50 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | +1 | mvninstall | 404 | master passed | | +1 | compile | 64 | master passed | | +1 | checkstyle | 92 | master passed | | +1 | shadedjars | 337 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 42 | master passed | | 0 | spotbugs | 270 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 267 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 375 | the patch passed | | +1 | compile | 64 | the patch passed | | +1 | javac | 64 | the patch passed | | +1 | checkstyle | 90 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 337 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1172 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 42 | the patch passed | | +1 | findbugs | 284 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 9412 | hbase-server in the patch passed. | | +1 | asflicense | 34 | The patch does not generate ASF License warnings. | | | | 13528 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-551/6/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/551 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 5f3b8bacf823 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-551/out/precommit/personality/provided.sh | | git revision | master / 090c55f3ff | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-551/6/testReport/ | | Max. process+thread count | 4887 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-551/6/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] ravowlga123 commented on issue #552: HBASE-22893
ravowlga123 commented on issue #552: HBASE-22893 URL: https://github.com/apache/hbase/pull/552#issuecomment-526605979 Hi @jatsakthi Do you want me to remove the commit with the change in HBaseClassTestRule.java and just keep the commit with documentation change?? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #435: HBASE-22767 System table RIT STUCK if their RSGroup has no highest ve…
Apache-HBase commented on issue #435: HBASE-22767 System table RIT STUCK if their RSGroup has no highest ve… URL: https://github.com/apache/hbase/pull/435#issuecomment-526645239 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 33 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | 0 | shelldocs | 0 | Shelldocs was not available. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | 0 | mvndep | 33 | Maven dependency ordering for branch | | +1 | mvninstall | 369 | master passed | | +1 | compile | 104 | master passed | | +1 | checkstyle | 126 | master passed | | +1 | shadedjars | 297 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 75 | master passed | | 0 | spotbugs | 259 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 350 | master passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 16 | Maven dependency ordering for patch | | +1 | mvninstall | 324 | the patch passed | | +1 | compile | 103 | the patch passed | | +1 | javac | 103 | the patch passed | | -1 | checkstyle | 88 | hbase-server: The patch generated 1 new + 9 unchanged - 0 fixed = 10 total (was 9) | | +1 | shellcheck | 0 | There were no new shellcheck issues. | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 298 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1018 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 75 | the patch passed | | -1 | findbugs | 57 | hbase-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | +1 | unit | 175 | hbase-common in the patch passed. | | +1 | unit | 9096 | hbase-server in the patch passed. | | +1 | unit | 401 | hbase-rsgroup in the patch passed. | | +1 | asflicense | 65 | The patch does not generate ASF License warnings. | | | | 13884 | | | Reason | Tests | |---:|:--| | FindBugs | module:hbase-common | | | org.apache.hadoop.hbase.Version.() invokes inefficient new String(String) constructor At Version.java:inefficient new String(String) constructor At Version.java:[line 10] | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-435/91/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/435 | | Optional Tests | dupname asflicense shellcheck shelldocs javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 192dbf808173 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-435/out/precommit/personality/provided.sh | | git revision | master / 090c55f3ff | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-435/91/artifact/out/diff-checkstyle-hbase-server.txt | | findbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-435/91/artifact/out/new-findbugs-hbase-common.html | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-435/91/testReport/ | | Max. process+thread count | 4909 (vs. ulimit of 1) | | modules | C: hbase-common hbase-server hbase-rsgroup U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-435/91/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) shellcheck=0.4.4 findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] shahrs87 commented on issue #565: [HBASE-22874] Canary should not be IA.Public
shahrs87 commented on issue #565: [HBASE-22874] Canary should not be IA.Public URL: https://github.com/apache/hbase/pull/565#issuecomment-526677985 Thank you @busbey for your reviews. This is still a WIP patch. I wanted to run all the test once and see if I introduced any regression. Will add unit tests, cleanup code, address your comments in next revision. :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22724) Add a emoji on the vote table for pre commit result on github
[
https://issues.apache.org/jira/browse/HBASE-22724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919845#comment-16919845
]
Hudson commented on HBASE-22724:
Results for branch HBASE-22724
[build #9 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22724/9/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22724/9//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22724/9//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22724/9//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add a emoji on the vote table for pre commit result on github
> -
>
> Key: HBASE-22724
> URL: https://issues.apache.org/jira/browse/HBASE-22724
> Project: HBase
> Issue Type: Improvement
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
>
> [~zghaobac] said that the current vote table on github is not good enough, as
> the colors are almost the same, it is not easy to find out which line is
> broken.
> Since github can not change the color of the text, he suggested that we add a
> column at the left most with some emojis to better notify the developpers.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [hbase] Apache-HBase commented on issue #566: HBASE-22380 break circle replication when doing bulkload
Apache-HBase commented on issue #566: HBASE-22380 break circle replication when doing bulkload URL: https://github.com/apache/hbase/pull/566#issuecomment-526724584 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 38 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ master Compile Tests _ | | 0 | mvndep | 36 | Maven dependency ordering for branch | | +1 | mvninstall | 307 | master passed | | +1 | compile | 119 | master passed | | +1 | checkstyle | 134 | master passed | | +1 | shadedjars | 274 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 76 | master passed | | 0 | spotbugs | 236 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 445 | master passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 17 | Maven dependency ordering for patch | | +1 | mvninstall | 293 | the patch passed | | +1 | compile | 121 | the patch passed | | +1 | cc | 121 | the patch passed | | +1 | javac | 121 | the patch passed | | -1 | checkstyle | 83 | hbase-server: The patch generated 1 new + 283 unchanged - 0 fixed = 284 total (was 283) | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 270 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 924 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | hbaseprotoc | 115 | the patch passed | | -1 | javadoc | 36 | hbase-server generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | -1 | findbugs | 248 | hbase-server generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | +1 | unit | 42 | hbase-protocol-shaded in the patch passed. | | +1 | unit | 108 | hbase-client in the patch passed. | | -1 | unit | 9315 | hbase-server in the patch failed. | | +1 | asflicense | 89 | The patch does not generate ASF License warnings. | | | | 13835 | | | Reason | Tests | |---:|:--| | FindBugs | module:hbase-server | | | org.apache.hadoop.hbase.replication.regionserver.ReplicationSink.replicateEntries(List, CellScanner, String, String, String) makes inefficient use of keySet iterator instead of entrySet iterator At ReplicationSink.java:makes inefficient use of keySet iterator instead of entrySet iterator At ReplicationSink.java:[line 258] | | Failed junit tests | hadoop.hbase.tool.TestBulkLoadHFilesSplitRecovery | | | hadoop.hbase.tool.TestSecureBulkLoadHFilesSplitRecovery | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-566/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/566 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc | | uname | Linux e999bac446c9 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-566/out/precommit/personality/provided.sh | | git revision | master / 090c55f3ff | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-566/1/artifact/out/diff-checkstyle-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-566/1/artifact/out/diff-javadoc-javadoc-hbase-server.txt | | findbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-566/1/artifact/out/new-findbugs-hbase-server.html | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-566/1/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-566/1/testReport/ | | Max. process+thread count | 4545 (vs. ulimit of 1) | | modules | C: hbase-protocol-shaded hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-566/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and
[jira] [Commented] (HBASE-22760) Stop/Resume Snapshot Auto-Cleanup activity with shell command
[ https://issues.apache.org/jira/browse/HBASE-22760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919805#comment-16919805 ] Viraj Jasani commented on HBASE-22760: -- Uploaded backport patch for branch-1 and branch-2 > Stop/Resume Snapshot Auto-Cleanup activity with shell command > - > > Key: HBASE-22760 > URL: https://issues.apache.org/jira/browse/HBASE-22760 > Project: HBase > Issue Type: Improvement > Components: Admin, shell, snapshots >Affects Versions: 3.0.0, 1.5.0, 2.3.0 >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 1.4.11, 2.1.7 > > Attachments: HBASE-22760.branch-1.000.patch, > HBASE-22760.branch-2.000.patch, HBASE-22760.master.003.patch, > HBASE-22760.master.004.patch, HBASE-22760.master.005.patch, > HBASE-22760.master.008.patch, HBASE-22760.master.009.patch > > > For any scheduled snapshot backup activity, we would like to disable > auto-cleaner for snapshot based on TTL. However, as per HBASE-22648 we have a > config to disable snapshot auto-cleaner: > hbase.master.cleaner.snapshot.disable, which would take effect only upon > HMaster restart just similar to any other hbase-site configs. > For any running cluster, we should be able to stop/resume auto-cleanup > activity for snapshot based on shell command. Something similar to below > command should be able to stop/start cleanup chore: > hbase(main):001:0> snapshot_auto_cleanup_switch false (disable > auto-cleaner) > hbase(main):001:0> snapshot_auto_cleanup_switch true (enable auto-cleaner) -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22103) HDFS-13209 in Hadoop 3.3.0 breaks asyncwal
[
https://issues.apache.org/jira/browse/HBASE-22103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919852#comment-16919852
]
HBase QA commented on HBASE-22103:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 6s{color}
| {color:red} HBASE-22103 does not apply to master. Rebase required? Wrong
Branch? See
https://yetus.apache.org/documentation/in-progress/precommit-patchnames for
help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-22103 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12963619/HBASE-22103.master.001.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/849/console |
| Powered by | Apache Yetus 0.9.0 http://yetus.apache.org |
This message was automatically generated.
> HDFS-13209 in Hadoop 3.3.0 breaks asyncwal
> --
>
> Key: HBASE-22103
> URL: https://issues.apache.org/jira/browse/HBASE-22103
> Project: HBase
> Issue Type: Bug
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Attachments: HBASE-22103.master.001.patch
>
>
> HDFS-13209 added an additional parameter to {{DistributedFileSystem.create}}
> and broke asyncwal.
> {noformat}
> 2019-03-25 12:19:21,061 ERROR [Listener at localhost/54758]
> asyncfs.FanOutOneBlockAsyncDFSOutputHelper(562): Couldn't properly initialize
> access to HDFS internals. Please update your WAL Provider to not make use of
> the 'asyncfs' provider. See HBASE-16110 for more information.
> java.lang.NoSuchMethodException:
> org.apache.hadoop.hdfs.protocol.ClientProtocol.create(java.lang.String,
> org.apache.hadoop.fs.permission.FsPermission, java.lang.String,
> org.apache.hadoop.io.EnumSetWritable, boolean, short, long,
> [Lorg.apache.hadoop.crypto.CryptoProtocolVersion;)
> at java.lang.Class.getMethod(Class.java:1786)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createFileCreator2(FanOutOneBlockAsyncDFSOutputHelper.java:513)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createFileCreator(FanOutOneBlockAsyncDFSOutputHelper.java:530)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:557)
> at
> org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:51)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:169)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:166)
> at
> org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:105)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createAsyncWriter(AsyncFSWAL.java:663)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:669)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:126)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:813)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:519)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:460)
> at
> org.apache.hadoop.hbase.regionserver.wal.TestAsyncFSWAL.newWAL(TestAsyncFSWAL.java:72)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractTestFSWAL.testWALComparator(AbstractTestFSWAL.java:194)
> {noformat}
> Credit: this bug was found by [~gabor.bota]
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [hbase] Apache-HBase commented on issue #560: HBASE-22945 Show quota infos in master UI
Apache-HBase commented on issue #560: HBASE-22945 Show quota infos in master UI URL: https://github.com/apache/hbase/pull/560#issuecomment-526668248 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 117 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | 0 | mvndep | 43 | Maven dependency ordering for branch | | +1 | mvninstall | 476 | master passed | | +1 | compile | 111 | master passed | | +1 | checkstyle | 134 | master passed | | +1 | shadedjars | 371 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 75 | master passed | | 0 | spotbugs | 314 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 397 | master passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 18 | Maven dependency ordering for patch | | +1 | mvninstall | 454 | the patch passed | | +1 | compile | 111 | the patch passed | | +1 | javac | 111 | the patch passed | | +1 | checkstyle | 141 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 365 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1365 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 72 | the patch passed | | +1 | findbugs | 384 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 118 | hbase-client in the patch passed. | | -1 | unit | 17001 | hbase-server in the patch failed. | | +1 | asflicense | 48 | The patch does not generate ASF License warnings. | | | | 22352 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.client.TestSnapshotTemporaryDirectoryWithRegionReplicas | | | hadoop.hbase.client.TestFromClientSideWithCoprocessor | | | hadoop.hbase.client.TestFromClientSide3 | | | hadoop.hbase.master.TestSplitWALManager | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-560/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/560 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 2cc3fe041973 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-560/out/precommit/personality/provided.sh | | git revision | master / 090c55f3ff | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-560/4/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-560/4/testReport/ | | Max. process+thread count | 4634 (vs. ulimit of 1) | | modules | C: hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-560/4/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22951) [HBCK2] hbase hbck throws IOE "No FileSystem for scheme: hdfs"
[
https://issues.apache.org/jira/browse/HBASE-22951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919796#comment-16919796
]
stack commented on HBASE-22951:
---
Fails. hbck1 always wanted to talk to hdfs. hbck2 does now since we backfilled
checks of hdfs. Let me see if a 'shaded' or cut-down hdfs client to add to CP
by default. Thanks for taking a look.
> [HBCK2] hbase hbck throws IOE "No FileSystem for scheme: hdfs"
> --
>
> Key: HBASE-22951
> URL: https://issues.apache.org/jira/browse/HBASE-22951
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Priority: Major
>
> Input appreciated on this one.
> If I do the below, passing a config that is pointing at a HDFS, I get the
> below (If I run w/o, hbck just picks up the wrong fs -- the local fs).
> {code}
> $ /vagrant/hbase/bin/hbase --config hbase-conf hbck
> 2019-08-30 05:04:54,467 WARN [main] util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes where
> applicable
> Exception in thread "main" java.io.IOException: No FileSystem for scheme: hdfs
> at
> org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2799)
> at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2810)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:100)
> at
> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2849)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2831)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:389)
> at org.apache.hadoop.fs.Path.getFileSystem(Path.java:356)
> at
> org.apache.hadoop.hbase.util.CommonFSUtils.getRootDir(CommonFSUtils.java:361)
> at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3605)
> {code}
> Its because the CLASSPATH is carefully curated so as to use shaded client
> only; there are no hdfs classes on the CLASSPATH intentionally.
> So, how to fix? Happens whether hbck1 or hbck2 (you have to do a hdfs
> operation for hbck2 to trigger same issue).
> Could be careful in hbck2 and note that if fs operation, you need to add hdfs
> jars to CLASSPATH so hbck2 can go against hdfs.
> If add the ' --internal-classpath' flag, then all classes are put on the
> CLASSPATH for hbck(2) (including the hdfs client jar which got the hdfs
> implementation after 2.7.2 was released) and stuff 'works'.
> Could edit the bin/hbase script and make it so hdfs classes are added to the
> hbck CLASSPATH? Could see if could do hdfs client-only?
> Anyways, putting this up for now. Others may have opinions. Thanks.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Comment Edited] (HBASE-22760) Stop/Resume Snapshot Auto-Cleanup activity with shell command
[ https://issues.apache.org/jira/browse/HBASE-22760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919805#comment-16919805 ] Viraj Jasani edited comment on HBASE-22760 at 8/30/19 6:21 PM: --- Uploaded backport patch for branch-1 and branch-2 Build for branch-1 patch in progress: [https://builds.apache.org/job/PreCommit-HBASE-Build/848/] Build for branch-2 patch timed out: [https://builds.apache.org/job/PreCommit-HBASE-Build/846/] was (Author: vjasani): Uploaded backport patch for branch-1 and branch-2 > Stop/Resume Snapshot Auto-Cleanup activity with shell command > - > > Key: HBASE-22760 > URL: https://issues.apache.org/jira/browse/HBASE-22760 > Project: HBase > Issue Type: Improvement > Components: Admin, shell, snapshots >Affects Versions: 3.0.0, 1.5.0, 2.3.0 >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 1.4.11, 2.1.7 > > Attachments: HBASE-22760.branch-1.000.patch, > HBASE-22760.branch-2.000.patch, HBASE-22760.master.003.patch, > HBASE-22760.master.004.patch, HBASE-22760.master.005.patch, > HBASE-22760.master.008.patch, HBASE-22760.master.009.patch > > > For any scheduled snapshot backup activity, we would like to disable > auto-cleaner for snapshot based on TTL. However, as per HBASE-22648 we have a > config to disable snapshot auto-cleaner: > hbase.master.cleaner.snapshot.disable, which would take effect only upon > HMaster restart just similar to any other hbase-site configs. > For any running cluster, we should be able to stop/resume auto-cleanup > activity for snapshot based on shell command. Something similar to below > command should be able to stop/start cleanup chore: > hbase(main):001:0> snapshot_auto_cleanup_switch false (disable > auto-cleaner) > hbase(main):001:0> snapshot_auto_cleanup_switch true (enable auto-cleaner) -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HBASE-22953) Supporting Hadoop 3.3.0
[ https://issues.apache.org/jira/browse/HBASE-22953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HBASE-22953: Issue Type: Umbrella (was: Task) > Supporting Hadoop 3.3.0 > --- > > Key: HBASE-22953 > URL: https://issues.apache.org/jira/browse/HBASE-22953 > Project: HBase > Issue Type: Umbrella >Reporter: Wei-Chiu Chuang >Priority: Major > > The Hadoop community has started to discuss a 3.3.0 release. > [http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201908.mbox/%3CCAD%2B%2BeCneLtC%2BkfxRRKferufnNxhaXXGa0YPaVp%3DEBbc-R5JfqA%40mail.gmail.com%3E] > While still early, it wouldn't hurt to start exploring what's coming in > Hadoop 3.3.0. In particular, there are a bunch of new features that brings in > all sorts of new dependencies. > > I will use this Jira to list things that are related to Hadoop 3.3.0. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (HBASE-22954) Whitelist net.java.dev.jna which got pulled in through Hadoop 3.3.0
Wei-Chiu Chuang created HBASE-22954:
---
Summary: Whitelist net.java.dev.jna which got pulled in through
Hadoop 3.3.0
Key: HBASE-22954
URL: https://issues.apache.org/jira/browse/HBASE-22954
Project: HBase
Issue Type: Bug
Reporter: Wei-Chiu Chuang
Assignee: Wei-Chiu Chuang
YARN-9477 added a new dependency net.java.dev.jna, which resulted in license
check failure in HBase because the checker thinks it's LGPL 2.1 licensed. But
in fact, it is dual licensed.
{code}
Java Native Access
Java Native Access
https://github.com/java-native-access/jna
LGPL, version 2.1
http://www.gnu.org/licenses/licenses.html
repo
Apache License v2.0
http://www.apache.org/licenses/LICENSE-2.0.txt
repo
{code}
We can
(1) white list this dependency
(2) or update the license checker to search for if any of the licenses is
permitted
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HBASE-21879) Read HFile's block to ByteBuffer directly instead of to byte for reducing young gc purpose
[
https://issues.apache.org/jira/browse/HBASE-21879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919774#comment-16919774
]
Hudson commented on HBASE-21879:
Results for branch HBASE-21879
[build #225 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/225/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/225//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/225//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/225//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Read HFile's block to ByteBuffer directly instead of to byte for reducing
> young gc purpose
> --
>
> Key: HBASE-21879
> URL: https://issues.apache.org/jira/browse/HBASE-21879
> Project: HBase
> Issue Type: Improvement
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: HBASE-21879.v1.patch, HBASE-21879.v1.patch,
> QPS-latencies-before-HBASE-21879.png, gc-data-before-HBASE-21879.png
>
>
> In HFileBlock#readBlockDataInternal, we have the following:
> {code}
> @VisibleForTesting
> protected HFileBlock readBlockDataInternal(FSDataInputStream is, long offset,
> long onDiskSizeWithHeaderL, boolean pread, boolean verifyChecksum,
> boolean updateMetrics)
> throws IOException {
> // .
> // TODO: Make this ByteBuffer-based. Will make it easier to go to HDFS with
> BBPool (offheap).
> byte [] onDiskBlock = new byte[onDiskSizeWithHeader + hdrSize];
> int nextBlockOnDiskSize = readAtOffset(is, onDiskBlock, preReadHeaderSize,
> onDiskSizeWithHeader - preReadHeaderSize, true, offset +
> preReadHeaderSize, pread);
> if (headerBuf != null) {
> // ...
> }
> // ...
> }
> {code}
> In the read path, we still read the block from hfile to on-heap byte[], then
> copy the on-heap byte[] to offheap bucket cache asynchronously, and in my
> 100% get performance test, I also observed some frequent young gc, The
> largest memory footprint in the young gen should be the on-heap block byte[].
> In fact, we can read HFile's block to ByteBuffer directly instead of to
> byte[] for reducing young gc purpose. we did not implement this before,
> because no ByteBuffer reading interface in the older HDFS client, but 2.7+
> has supported this now, so we can fix this now. I think.
> Will provide an patch and some perf-comparison for this.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [hbase] Apache-HBase commented on issue #476: HBASE-11062 hbtop
Apache-HBase commented on issue #476: HBASE-11062 hbtop URL: https://github.com/apache/hbase/pull/476#issuecomment-526695348 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 46 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | 0 | shelldocs | 1 | Shelldocs was not available. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 2 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 22 new or modified test files. | ||| _ master Compile Tests _ | | 0 | mvndep | 36 | Maven dependency ordering for branch | | +1 | mvninstall | 308 | master passed | | +1 | compile | 191 | master passed | | +1 | checkstyle | 159 | master passed | | +1 | shadedjars | 286 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 188 | master passed | | 0 | spotbugs | 32 | Used deprecated FindBugs config; considering switching to SpotBugs. | | 0 | findbugs | 32 | branch/hbase-assembly no findbugs output file (findbugsXml.xml) | ||| _ Patch Compile Tests _ | | 0 | mvndep | 16 | Maven dependency ordering for patch | | +1 | mvninstall | 322 | the patch passed | | +1 | compile | 191 | the patch passed | | +1 | javac | 191 | the patch passed | | +1 | checkstyle | 148 | the patch passed | | +1 | shellcheck | 2 | There were no new shellcheck issues. | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | xml | 4 | The patch has no ill-formed XML file. | | +1 | shadedjars | 294 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 968 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 221 | the patch passed | | 0 | findbugs | 32 | hbase-assembly has no data from findbugs | ||| _ Other Tests _ | | -1 | unit | 10648 | root in the patch failed. | | +1 | asflicense | 85 | The patch does not generate ASF License warnings. | | | | 16405 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-476/63/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/476 | | Optional Tests | dupname asflicense shellcheck shelldocs javac javadoc unit shadedjars hadoopcheck xml compile spotbugs findbugs hbaseanti checkstyle | | uname | Linux 72889c52b1e2 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-476/out/precommit/personality/provided.sh | | git revision | master / 090c55f3ff | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-476/63/artifact/out/patch-unit-root.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-476/63/testReport/ | | Max. process+thread count | 5183 (vs. ulimit of 1) | | modules | C: hbase-hbtop . hbase-assembly U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-476/63/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) shellcheck=0.4.4 findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22951) [HBCK2] hbase hbck throws IOE "No FileSystem for scheme: hdfs"
[
https://issues.apache.org/jira/browse/HBASE-22951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919804#comment-16919804
]
Sean Busbey commented on HBASE-22951:
-
There's a blurb in the {{hbase}} script that gets hadoop's classpath from the
hadoop executable if it's on the path. I'd say just do that. eventually can
move to use hadoop client facing artifacts maybe, but ATM we don't ship them in
our omnibus tarball at all. Alternatively, we could finally isolate our own
hadoop libs in the omnibus tarball so we can add *just* them and not everything
in the internal classpath.
(or just take the easy path and move hbck to be in the set of commands that
always needs an internal classpath)
> [HBCK2] hbase hbck throws IOE "No FileSystem for scheme: hdfs"
> --
>
> Key: HBASE-22951
> URL: https://issues.apache.org/jira/browse/HBASE-22951
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Priority: Major
>
> Input appreciated on this one.
> If I do the below, passing a config that is pointing at a HDFS, I get the
> below (If I run w/o, hbck just picks up the wrong fs -- the local fs).
> {code}
> $ /vagrant/hbase/bin/hbase --config hbase-conf hbck
> 2019-08-30 05:04:54,467 WARN [main] util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes where
> applicable
> Exception in thread "main" java.io.IOException: No FileSystem for scheme: hdfs
> at
> org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2799)
> at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2810)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:100)
> at
> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2849)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2831)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:389)
> at org.apache.hadoop.fs.Path.getFileSystem(Path.java:356)
> at
> org.apache.hadoop.hbase.util.CommonFSUtils.getRootDir(CommonFSUtils.java:361)
> at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3605)
> {code}
> Its because the CLASSPATH is carefully curated so as to use shaded client
> only; there are no hdfs classes on the CLASSPATH intentionally.
> So, how to fix? Happens whether hbck1 or hbck2 (you have to do a hdfs
> operation for hbck2 to trigger same issue).
> Could be careful in hbck2 and note that if fs operation, you need to add hdfs
> jars to CLASSPATH so hbck2 can go against hdfs.
> If add the ' --internal-classpath' flag, then all classes are put on the
> CLASSPATH for hbck(2) (including the hdfs client jar which got the hdfs
> implementation after 2.7.2 was released) and stuff 'works'.
> Could edit the bin/hbase script and make it so hdfs classes are added to the
> hbck CLASSPATH? Could see if could do hdfs client-only?
> Anyways, putting this up for now. Others may have opinions. Thanks.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HBASE-22760) Stop/Resume Snapshot Auto-Cleanup activity with shell command
[ https://issues.apache.org/jira/browse/HBASE-22760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated HBASE-22760: - Attachment: HBASE-22760.branch-1.000.patch > Stop/Resume Snapshot Auto-Cleanup activity with shell command > - > > Key: HBASE-22760 > URL: https://issues.apache.org/jira/browse/HBASE-22760 > Project: HBase > Issue Type: Improvement > Components: Admin, shell, snapshots >Affects Versions: 3.0.0, 1.5.0, 2.3.0 >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 1.4.11, 2.1.7 > > Attachments: HBASE-22760.branch-1.000.patch, > HBASE-22760.branch-2.000.patch, HBASE-22760.master.003.patch, > HBASE-22760.master.004.patch, HBASE-22760.master.005.patch, > HBASE-22760.master.008.patch, HBASE-22760.master.009.patch > > > For any scheduled snapshot backup activity, we would like to disable > auto-cleaner for snapshot based on TTL. However, as per HBASE-22648 we have a > config to disable snapshot auto-cleaner: > hbase.master.cleaner.snapshot.disable, which would take effect only upon > HMaster restart just similar to any other hbase-site configs. > For any running cluster, we should be able to stop/resume auto-cleanup > activity for snapshot based on shell command. Something similar to below > command should be able to stop/start cleanup chore: > hbase(main):001:0> snapshot_auto_cleanup_switch false (disable > auto-cleaner) > hbase(main):001:0> snapshot_auto_cleanup_switch true (enable auto-cleaner) -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (HBASE-22953) Supporting Hadoop 3.3.0
Wei-Chiu Chuang created HBASE-22953: --- Summary: Supporting Hadoop 3.3.0 Key: HBASE-22953 URL: https://issues.apache.org/jira/browse/HBASE-22953 Project: HBase Issue Type: Task Reporter: Wei-Chiu Chuang The Hadoop community has started to discuss a 3.3.0 release. [http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201908.mbox/%3CCAD%2B%2BeCneLtC%2BkfxRRKferufnNxhaXXGa0YPaVp%3DEBbc-R5JfqA%40mail.gmail.com%3E] While still early, it wouldn't hurt to start exploring what's coming in Hadoop 3.3.0. In particular, there are a bunch of new features that brings in all sorts of new dependencies. I will use this Jira to list things that are related to Hadoop 3.3.0. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [hbase-operator-tools] asf-ci commented on issue #20: HBASE-22952 HBCK2 replication command is incompatible with 2.0.x
asf-ci commented on issue #20: HBASE-22952 HBCK2 replication command is incompatible with 2.0.x URL: https://github.com/apache/hbase-operator-tools/pull/20#issuecomment-526664972 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/PreCommit-HBASE-OPERATOR-TOOLS-Build/70/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #553: HBASE-22939 SpaceQuotas- Bulkload from different hdfs failed when space quotas are turned on.
Apache-HBase commented on issue #553: HBASE-22939 SpaceQuotas- Bulkload from different hdfs failed when space quotas are turned on. URL: https://github.com/apache/hbase/pull/553#issuecomment-526671973 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 41 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | +1 | mvninstall | 393 | master passed | | +1 | compile | 66 | master passed | | +1 | checkstyle | 108 | master passed | | +1 | shadedjars | 348 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 46 | master passed | | 0 | spotbugs | 320 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 317 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 381 | the patch passed | | +1 | compile | 74 | the patch passed | | +1 | javac | 74 | the patch passed | | +1 | checkstyle | 116 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 350 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1257 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 39 | the patch passed | | +1 | findbugs | 323 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 10250 | hbase-server in the patch passed. | | +1 | asflicense | 28 | The patch does not generate ASF License warnings. | | | | 14614 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-553/5/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/553 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux dffbab1873b7 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-553/out/precommit/personality/provided.sh | | git revision | master / 090c55f3ff | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-553/5/testReport/ | | Max. process+thread count | 4448 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-553/5/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] shahrs87 edited a comment on issue #565: [HBASE-22874] Canary should not be IA.Public
shahrs87 edited a comment on issue #565: [HBASE-22874] Canary should not be IA.Public URL: https://github.com/apache/hbase/pull/565#issuecomment-526677985 Thank you @busbey for your reviews. This is still a WIP patch. I wanted to run all the test once and see if I introduced any regression. Will add unit tests, cleanup code, address your comments in next revision. Hopefully by EOD I should be able to get it to review state. :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #509: HBASE-22877 WebHDFS based export snapshot will fail if hfile is in archive directory
Apache-HBase commented on issue #509: HBASE-22877 WebHDFS based export snapshot will fail if hfile is in archive directory URL: https://github.com/apache/hbase/pull/509#issuecomment-526604402 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 73 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 | mvninstall | 343 | master passed | | +1 | compile | 61 | master passed | | +1 | checkstyle | 81 | master passed | | +1 | shadedjars | 286 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 40 | master passed | | 0 | spotbugs | 268 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 264 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 314 | the patch passed | | +1 | compile | 61 | the patch passed | | +1 | javac | 61 | the patch passed | | +1 | checkstyle | 82 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 289 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 976 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 39 | the patch passed | | +1 | findbugs | 275 | the patch passed | ||| _ Other Tests _ | | -1 | unit | 16008 | hbase-server in the patch failed. | | +1 | asflicense | 29 | The patch does not generate ASF License warnings. | | | | 19608 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.master.TestSplitWALManager | | | hadoop.hbase.client.TestAsyncRegionAdminApi | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-509/36/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/509 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 2cdee7bb8752 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-509/out/precommit/personality/provided.sh | | git revision | master / 090c55f3ff | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-509/36/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-509/36/testReport/ | | Max. process+thread count | 4842 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-509/36/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22927) Upgrade mockito version for Java 11 compatibility
[ https://issues.apache.org/jira/browse/HBASE-22927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919673#comment-16919673 ] Rabi Kumar K C commented on HBASE-22927: Hi [~jatsakthi], I updated the mockito-core version to 2.23.0 ran the tests for java 11.0.1 and tests failed for following classes: * org.apache.hadoop.hbase.util.TestFutureUtils * org.apache.hadoop.hbase.io.hfile.TestHFile * org.apache.hadoop.hbase.http.log.TestLogLevel * org.apache.hadoop.hbase.io.hfile.TestHFileScannerImplReferenceCount * org.apache.hadoop.hbase.coprocessor.TestCoprocessorEndpoint * org.apache.hadoop.hbase.coprocessor.TestSecureExport But none of the errors were due to updation of mockito version. Do you have any suggestions on how to proceed or should i run another test just to be sure?? > Upgrade mockito version for Java 11 compatibility > - > > Key: HBASE-22927 > URL: https://issues.apache.org/jira/browse/HBASE-22927 > Project: HBase > Issue Type: Task >Reporter: Sakthi >Assignee: Rabi Kumar K C >Priority: Major > > Pasting the discussion from HBASE-22534 here: > "Currently mockito-core version is at 2.1.0. According to > [https://github.com/mockito/mockito/blob/release/2.x/doc/release-notes/official.md], > looks like Java 11 compatibility was introduced in 2.19+. And 2.23.2 claims > to have full java 11 support after byte-buddy fix etc." -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [hbase] shahrs87 commented on a change in pull request #565: [HBASE-22874] Canary should not be IA.Public
shahrs87 commented on a change in pull request #565: [HBASE-22874] Canary
should not be IA.Public
URL: https://github.com/apache/hbase/pull/565#discussion_r319691459
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/tool/CanaryInterface.java
##
@@ -0,0 +1,40 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.tool;
+
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.yetus.audience.InterfaceAudience;
+
+import java.util.concurrent.ExecutorService;
+
+@InterfaceAudience.Public
+public interface CanaryInterface {
Review comment:
I also thought about this. If we backport this to working branches. In my
case it is 1.3. We already have other tools which calls Canary via toolRunner
and if I backport this patch to 1.3 then my current code will fail since Canary
will be an interface now instead of ToolRunner implementation.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Updated] (HBASE-22954) Whitelist net.java.dev.jna which got pulled in through Hadoop 3.3.0
[
https://issues.apache.org/jira/browse/HBASE-22954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang updated HBASE-22954:
Attachment: HBASE-22954.master.001.patch
> Whitelist net.java.dev.jna which got pulled in through Hadoop 3.3.0
> ---
>
> Key: HBASE-22954
> URL: https://issues.apache.org/jira/browse/HBASE-22954
> Project: HBase
> Issue Type: Bug
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Attachments: HBASE-22954.001.patch, HBASE-22954.master.001.patch
>
>
> YARN-9477 added a new dependency net.java.dev.jna, which resulted in license
> check failure in HBase because the checker thinks it's LGPL 2.1 licensed. But
> in fact, it is dual licensed.
> {code}
> Java Native Access
> Java Native Access
> https://github.com/java-native-access/jna
>
>
> LGPL, version 2.1
> http://www.gnu.org/licenses/licenses.html
> repo
>
>
> Apache License v2.0
> http://www.apache.org/licenses/LICENSE-2.0.txt
> repo
>
>
> {code}
> We can
> (1) white list this dependency
> (2) or update the license checker to search for if any of the licenses is
> permitted
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HBASE-22954) Whitelist net.java.dev.jna which got pulled in through Hadoop 3.3.0
[
https://issues.apache.org/jira/browse/HBASE-22954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919977#comment-16919977
]
Wei-Chiu Chuang commented on HBASE-22954:
-
Posted an initial patch to update license for JNA.
[^HBASE-22954.master.001.patch]
[~busbey] what do you say? Should the license checker figure out which license
is applicable automatically, or should this be manual.
> Whitelist net.java.dev.jna which got pulled in through Hadoop 3.3.0
> ---
>
> Key: HBASE-22954
> URL: https://issues.apache.org/jira/browse/HBASE-22954
> Project: HBase
> Issue Type: Bug
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Attachments: HBASE-22954.001.patch, HBASE-22954.master.001.patch
>
>
> YARN-9477 added a new dependency net.java.dev.jna, which resulted in license
> check failure in HBase because the checker thinks it's LGPL 2.1 licensed. But
> in fact, it is dual licensed.
> {code}
> Java Native Access
> Java Native Access
> https://github.com/java-native-access/jna
>
>
> LGPL, version 2.1
> http://www.gnu.org/licenses/licenses.html
> repo
>
>
> Apache License v2.0
> http://www.apache.org/licenses/LICENSE-2.0.txt
> repo
>
>
> {code}
> We can
> (1) white list this dependency
> (2) or update the license checker to search for if any of the licenses is
> permitted
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HBASE-22954) Whitelist net.java.dev.jna which got pulled in through Hadoop 3.3.0
[
https://issues.apache.org/jira/browse/HBASE-22954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919978#comment-16919978
]
HBase QA commented on HBASE-22954:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
36s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
11s{color} | {color:green} hbase-resource-bundle in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 18m 6s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.1 Server=19.03.1 base:
https://builds.apache.org/job/PreCommit-HBASE-Build/850/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-22954 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12979028/HBASE-22954.master.001.patch
|
| Optional Tests | dupname asflicense javac javadoc unit xml |
| uname | Linux c55162a04218 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 090c55f3ff |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/850/testReport/ |
| Max. process+thread count | 85 (vs. ulimit of 1) |
| modules | C: hbase-resource-bundle U: hbase-resource-bundle |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/850/console |
| Powered by | Apache Yetus 0.9.0 http://yetus.apache.org |
This message was automatically generated.
> Whitelist net.java.dev.jna which got pulled in through Hadoop 3.3.0
> ---
>
> Key: HBASE-22954
> URL: https://issues.apache.org/jira/browse/HBASE-22954
> Project: HBase
> Issue Type: Bug
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Attachments: HBASE-22954.001.patch, HBASE-22954.master.001.patch
>
>
> YARN-9477 added a new dependency net.java.dev.jna, which resulted in license
> check failure in HBase because the checker thinks it's LGPL 2.1 licensed. But
> in fact, it is dual licensed.
> {code}
> Java Native Access
> Java Native Access
> https://github.com/java-native-access/jna
>
>
> LGPL, version 2.1
> http://www.gnu.org/licenses/licenses.html
> repo
>
>
> Apache License v2.0
> http://www.apache.org/licenses/LICENSE-2.0.txt
> repo
>
>
> {code}
> We can
> (1) white list this dependency
> (2) or update the license checker to search for if any of the licenses is
> permitted
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HBASE-22954) Whitelist net.java.dev.jna which got pulled in through Hadoop 3.3.0
[
https://issues.apache.org/jira/browse/HBASE-22954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang updated HBASE-22954:
Status: Patch Available (was: Open)
> Whitelist net.java.dev.jna which got pulled in through Hadoop 3.3.0
> ---
>
> Key: HBASE-22954
> URL: https://issues.apache.org/jira/browse/HBASE-22954
> Project: HBase
> Issue Type: Bug
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Attachments: HBASE-22954.001.patch, HBASE-22954.master.001.patch
>
>
> YARN-9477 added a new dependency net.java.dev.jna, which resulted in license
> check failure in HBase because the checker thinks it's LGPL 2.1 licensed. But
> in fact, it is dual licensed.
> {code}
> Java Native Access
> Java Native Access
> https://github.com/java-native-access/jna
>
>
> LGPL, version 2.1
> http://www.gnu.org/licenses/licenses.html
> repo
>
>
> Apache License v2.0
> http://www.apache.org/licenses/LICENSE-2.0.txt
> repo
>
>
> {code}
> We can
> (1) white list this dependency
> (2) or update the license checker to search for if any of the licenses is
> permitted
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Created] (HBASE-22955) Branches-1 nightly yetus jobs are using jdk8 for jdk7 jobs
Sean Busbey created HBASE-22955:
---
Summary: Branches-1 nightly yetus jobs are using jdk8 for jdk7 jobs
Key: HBASE-22955
URL: https://issues.apache.org/jira/browse/HBASE-22955
Project: HBase
Issue Type: Bug
Affects Versions: 1.5.0, 1.3.6, 1.4.11
Reporter: Sean Busbey
branch-1 nightly job is failing while trying to do our yetus shaded jar test
because the enforcer plugin complains about a jdk8 jdk-tools jar being present.
looking at the yetus footer, it's because jdk8 is being used instead of jdk7:
{code:java}
[2019-08-30T13:30:34.353Z] | maven | version: Apache Maven 3.0.5 |
[2019-08-30T13:30:34.353Z] | Default Java | 1.8.0_222 |
{code}
Same thing in the footer of the jdk7 build (which passes, even though it
shouldn't)
{code}
[2019-08-30T15:43:29.103Z] | maven | version: Apache Maven 3.0.5 |
[2019-08-30T15:43:29.103Z] | Default Java | 1.8.0_222 |
{code}
Both builds appear to be properly running in Docker mode.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HBASE-22760) Stop/Resume Snapshot Auto-Cleanup activity with shell command
[
https://issues.apache.org/jira/browse/HBASE-22760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919904#comment-16919904
]
HBase QA commented on HBASE-22760:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 17m
4s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 5m
50s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
24s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
50s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 11m
41s{color} | {color:green} branch-1 passed {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 3m
19s{color} | {color:blue} branch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red} 0m
12s{color} | {color:red} branch has 10 errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
10s{color} | {color:green} branch-1 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
34s{color} | {color:red} hbase-server: The patch generated 2 new + 287
unchanged - 0 fixed = 289 total (was 287) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 5m
41s{color} | {color:red} root: The patch generated 2 new + 788 unchanged - 0
fixed = 790 total (was 788) {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m
17s{color} | {color:red} The patch generated 8 new + 1045 unchanged - 4 fixed =
1053 total (was 1049) {color} |
| {color:green}+1{color} | {color:green} ruby-lint {color} | {color:green} 0m
3s{color} | {color:green} There were no new ruby-lint issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 2m
57s{color} | {color:blue} patch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red} 0m
9s{color} | {color:red} patch has 10 errors when building our shaded downstream
artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
4m 44s{color} | {color:green} Patch does not cause any errors with Hadoop 2.8.5
2.9.2. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
3m 16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit
[GitHub] [hbase] shahrs87 commented on a change in pull request #565: [HBASE-22874] Canary should not be IA.Public
shahrs87 commented on a change in pull request #565: [HBASE-22874] Canary
should not be IA.Public
URL: https://github.com/apache/hbase/pull/565#discussion_r319691668
##
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/tool/Canary.java
##
@@ -1629,7 +1684,7 @@ public static void main(String[] args) throws Exception {
final Configuration conf = HBaseConfiguration.create();
// Loading the generic options to conf
-new GenericOptionsParser(conf, args);
+//new GenericOptionsParser(conf, args);
Review comment:
This was redundant. We already do parsing in
https://github.com/apache/hadoop/blob/branch-2.8/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/ToolRunner.java#L70
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] shahrs87 commented on a change in pull request #565: [HBASE-22874] Canary should not be IA.Public
shahrs87 commented on a change in pull request #565: [HBASE-22874] Canary
should not be IA.Public
URL: https://github.com/apache/hbase/pull/565#discussion_r319691798
##
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/tool/Canary.java
##
@@ -843,9 +876,9 @@ public int run(String[] args) throws Exception {
}
}
currentTimeLength = System.currentTimeMillis() - startTime;
-if (currentTimeLength > this.timeout) {
+if (currentTimeLength > timeout) {
Review comment:
I removed bunch of class variables and store them in conf object. I
replaced the class variable with method local variable. So I can't use "this"
anymore.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Updated] (HBASE-22954) Whitelist net.java.dev.jna which got pulled in through Hadoop 3.3.0
[
https://issues.apache.org/jira/browse/HBASE-22954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang updated HBASE-22954:
Attachment: HBASE-22954.001.patch
> Whitelist net.java.dev.jna which got pulled in through Hadoop 3.3.0
> ---
>
> Key: HBASE-22954
> URL: https://issues.apache.org/jira/browse/HBASE-22954
> Project: HBase
> Issue Type: Bug
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Attachments: HBASE-22954.001.patch
>
>
> YARN-9477 added a new dependency net.java.dev.jna, which resulted in license
> check failure in HBase because the checker thinks it's LGPL 2.1 licensed. But
> in fact, it is dual licensed.
> {code}
> Java Native Access
> Java Native Access
> https://github.com/java-native-access/jna
>
>
> LGPL, version 2.1
> http://www.gnu.org/licenses/licenses.html
> repo
>
>
> Apache License v2.0
> http://www.apache.org/licenses/LICENSE-2.0.txt
> repo
>
>
> {code}
> We can
> (1) white list this dependency
> (2) or update the license checker to search for if any of the licenses is
> permitted
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Created] (HBASE-22957) [HBCK2] reference file check fails if compiled with old version but check against new
stack created HBASE-22957:
-
Summary: [HBCK2] reference file check fails if compiled with old
version but check against new
Key: HBASE-22957
URL: https://issues.apache.org/jira/browse/HBASE-22957
Project: HBase
Issue Type: Sub-task
Reporter: stack
The innocuous looking change HBASE-22721 Refactor HBaseFsck: move the inner
class out made it so some hbck2 filesystem checks fail if hbck2 was compiled
with a version from before this change but run against a newer version. In name
of making hbck2 able to run against broader swath of hbases, let me make some
changes. Here is the exception you'd see:
{code}
05:22:09.920 [main] INFO
org.apache.hadoop.hbase.client.ConnectionImplementation - Closing master
protocol: MasterService
Exception in thread "main" java.lang.NoSuchMethodError:
org.apache.hadoop.hbase.util.FSUtils.getTableStoreFilePathMap(Lorg/apache/hadoop/fs/FileSystem;Lorg/apache/hadoop/fs/Path;Lorg/apache/hadoop/fs/PathFilter;Ljava/util/concurrent/ExecutorService;Lorg/apache/hadoop/hbase/util/HBaseFsck$ErrorReporter;)Ljava/util/Map;
at
org.apache.hbase.hbck1.HBaseFsck.offlineReferenceFileRepair(HBaseFsck.java:1191)
at org.apache.hbase.hbck1.HBaseFsck.offlineHbck(HBaseFsck.java:846)
at org.apache.hbase.FileSystemFsck.fsck(FileSystemFsck.java:103)
at org.apache.hbase.HBCK2.doCommandLine(HBCK2.java:560)
at org.apache.hbase.HBCK2.run(HBCK2.java:470)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
at org.apache.hbase.HBCK2.main(HBCK2.java:620)
{code}
While the 'fix' is for hbase-operator-tools, this area up in hbase needs
commentary and deprecation. It was wonky from the get-go and we need to be
explicit that this code is not to be enhanced and is going away. Will file a
companion issue for hbase changes.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [hbase-operator-tools] busbey commented on a change in pull request #18: HBASE-22567 - HBCK2 addMissingRegionsToMeta
busbey commented on a change in pull request #18: HBASE-22567 - HBCK2 addMissingRegionsToMeta URL: https://github.com/apache/hbase-operator-tools/pull/18#discussion_r319710510 ## File path: hbase-hbck2/README.md ## @@ -345,3 +345,47 @@ HBASE_CLASSPATH_PREFIX=./hbase-hbck2-1.0.0-SNAPSHOT.jar hbase org.apache.hbase.H The same may happen to the _hbase:namespace_ system table. Look for the encoded Region name of the _hbase:namespace_ Region and do similar to what we did for _hbase:meta_. + +### Missing Regions in META + +There's been some extra-ordinary cases where table regions are removed from META table. Review comment: this paragraph needs some reworking now that there's an _OfflineMetaRepair_ shipped in `hbase-hbck2` ref: https://github.com/apache/hbase-operator-tools/tree/master/hbase-hbck2#hbasemeta-regiontable-restorerebuild This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase-operator-tools] busbey commented on a change in pull request #18: HBASE-22567 - HBCK2 addMissingRegionsToMeta
busbey commented on a change in pull request #18: HBASE-22567 - HBCK2
addMissingRegionsToMeta
URL:
https://github.com/apache/hbase-operator-tools/pull/18#discussion_r319711436
##
File path: hbase-hbck2/src/main/java/org/apache/hbase/HBCK2.java
##
@@ -164,6 +178,102 @@ int setRegionState(String region, RegionState.State
newState)
return EXIT_FAILURE;
}
+ Map> reportTablesWithMissingRegionsInMeta(String...
nameSpaceOrTable)
+ throws Exception {
+Map> report;
+try(final MetaFixer metaFixer = new MetaFixer(this.conf)){
+ List names = nameSpaceOrTable != null ?
Arrays.asList(nameSpaceOrTable) : null;
+ report = metaFixer.reportTablesMissingRegions(names);
+} catch (Exception e) {
+ LOG.error("Error reporting missing regions: ", e);
+ throw e;
+}
+if(LOG.isDebugEnabled()) {
+ LOG.debug(formatMissingRegionsInMetaReport(report));
+}
+return report;
+ }
+
+ List addMissingRegionsInMeta(List regionsPath) throws
IOException {
+List reAddedRegionsEncodedNames = new ArrayList<>();
+try(final MetaFixer metaFixer = new MetaFixer(this.conf)){
+ for(Path regionPath : regionsPath){
+metaFixer.putRegionInfoFromHdfsInMeta(regionPath);
+reAddedRegionsEncodedNames.add(regionPath.getName());
+ }
+}
+return reAddedRegionsEncodedNames;
+ }
+
+ Pair, List>
addMissingRegionsInMetaForTables(String...
+ nameSpaceOrTable) {
+ExecutorService executorService = Executors.newFixedThreadPool(
+ (nameSpaceOrTable == null ||
+nameSpaceOrTable.length > Runtime.getRuntime().availableProcessors()) ?
+ Runtime.getRuntime().availableProcessors() :
+ nameSpaceOrTable.length);
+List>> futures = new ArrayList<>( nameSpaceOrTable ==
null ? 1 :
+ nameSpaceOrTable.length);
+final List readdedRegionNames = new ArrayList<>();
+List executionErrors = new ArrayList<>();
+try {
+ //reducing number of retries in case disable fails due to namespace
table region also missing
+ this.conf.setInt(HConstants.HBASE_CLIENT_RETRIES_NUMBER, 1);
+ try(Connection conn = ConnectionFactory.createConnection(this.conf);
+final Admin admin = conn.getAdmin()) {
+Map> report =
reportTablesWithMissingRegionsInMeta(nameSpaceOrTable);
+for (TableName tableName : report.keySet()) {
+ if(admin.tableExists(tableName)) {
+futures.add(executorService.submit(new Callable>() {
+ @Override
+ public List call() throws Exception {
+LOG.debug("running thread for {}",
tableName.getNameWithNamespaceInclAsString());
+try {
+ admin.disableTable(tableName);
+} catch (IOException e) {
+ LOG.debug("Failed to disable table {}, "
+ + "is namespace table also missing regions? Continue
anyway...",
+tableName.getNameWithNamespaceInclAsString(), e);
+}
+List reAddedRegions =
addMissingRegionsInMeta(report.get(tableName));
+try {
+ admin.enableTable(tableName);
+} catch (IOException e) {
+ LOG.debug("Failed enabling table {}. It might be that
namespace table "
+ + "region is also missing.\n"
+ + "After this command finishes, please make sure on this
table state.",
+tableName.getNameWithNamespaceInclAsString(), e);
+}
+return reAddedRegions;
+ }
+}));
+ } else {
+LOG.warn("Table {} does not exist! Skipping...",
+ tableName.getNameWithNamespaceInclAsString());
+ }
+}
+for(Future> f : futures){
+ try {
+readdedRegionNames.addAll(f.get());
+ } catch (ExecutionException e){
+//we want to allow potential running threads to finish, so we
collect execution
+//errors and show those later
+LOG.debug("Caught execution error: ", e);
+executionErrors.add(e);
+ }
+}
+ }
+} catch (Exception ie){
+ LOG.error("ERROR executing thread: ", ie);
+} finally {
+ executorService.shutdown();
Review comment:
this method doesn't wait for any outstanding tasks to finish. do we need to
worry about that? the javadocs for `ExecutorService` have an example for
waiting for outstanding jobs.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
