[jira] [Comment Edited] (HBASE-21620) Problem in scan query when using more than one column prefix filter in some cases.
[
https://issues.apache.org/jira/browse/HBASE-21620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744767#comment-16744767
]
Karthick edited comment on HBASE-21620 at 1/17/19 7:54 AM:
---
We tried the same scan using HBase-2.1.2. The scan is even more slower than
1.4.8 (It takes around 3-4 seconds now). Because of this issue we are unable to
update our production clusters from 1.2.5 to a higher version. [~openinx] Can
you please provide a fix for this?
was (Author: karthickram):
We tried the same scan using HBase-2.1.2. The scan is even more slower than
1.4.8 (It takes around 3-4 seconds now). Because of this issue we are not able
to update our production clusters from 1.2.5 to a higher version. [~openinx]
Can you please provide a fix for this?
> Problem in scan query when using more than one column prefix filter in some
> cases.
> --
>
> Key: HBASE-21620
> URL: https://issues.apache.org/jira/browse/HBASE-21620
> Project: HBase
> Issue Type: Bug
> Components: scan
>Affects Versions: 3.0.0, 1.5.0, 2.2.0, 1.4.8, 2.1.2, 2.0.4
> Environment: hbase-1.4.8, hbase-1.4.9
> hadoop-2.7.3
>Reporter: Mohamed Mohideen Meeran
>Assignee: Zheng Hu
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.2, 2.0.4
>
> Attachments: HBASE-21620.branch-1.patch, HBASE-21620.v1.patch,
> HBASE-21620.v2.patch, HBASE-21620.v3.patch, HBASE-21620.v3.patch,
> HBaseImportData.java, columnkey.txt, file.txt, test.patch
>
>
> In some cases, unable to get the scan results when using more than one column
> prefix filter.
> Attached a java file to import the data which we used and a text file
> containing the values..
> While executing the following query (hbase shell as well as java program) it
> is waiting indefinitely and after RPC timeout we got the following error..
> Also we noticed high cpu, high load average and very frequent young gc in
> the region server containing this row...
> scan 'namespace:tablename',\{STARTROW => 'test',ENDROW => 'test', FILTER =>
> "ColumnPrefixFilter('1544770422942010001_') OR
> ColumnPrefixFilter('1544769883529010001_')"}
> ROW COLUMN+CELL
> ERROR: Call id=18,
> waitTime=60005, rpcTimetout=6
>
> Note: Table scan operation and scan with a single column prefix filter works
> fine in this case.
> When we check the same query in hbase-1.2.5 it is working fine.
> Can you please help me on this..
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21620) Problem in scan query when using more than one column prefix filter in some cases.
[
https://issues.apache.org/jira/browse/HBASE-21620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744767#comment-16744767
]
Karthick commented on HBASE-21620:
--
We tried the same scan using HBase-2.1.2. The scan is even more slower than
1.4.8 (It takes around 3-4 seconds now). Because of this issue we are not able
to update our production clusters from 1.2.5 to a higher version. [~openinx]
Can you please provide a fix for this?
> Problem in scan query when using more than one column prefix filter in some
> cases.
> --
>
> Key: HBASE-21620
> URL: https://issues.apache.org/jira/browse/HBASE-21620
> Project: HBase
> Issue Type: Bug
> Components: scan
>Affects Versions: 3.0.0, 1.5.0, 2.2.0, 1.4.8, 2.1.2, 2.0.4
> Environment: hbase-1.4.8, hbase-1.4.9
> hadoop-2.7.3
>Reporter: Mohamed Mohideen Meeran
>Assignee: Zheng Hu
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.2, 2.0.4
>
> Attachments: HBASE-21620.branch-1.patch, HBASE-21620.v1.patch,
> HBASE-21620.v2.patch, HBASE-21620.v3.patch, HBASE-21620.v3.patch,
> HBaseImportData.java, columnkey.txt, file.txt, test.patch
>
>
> In some cases, unable to get the scan results when using more than one column
> prefix filter.
> Attached a java file to import the data which we used and a text file
> containing the values..
> While executing the following query (hbase shell as well as java program) it
> is waiting indefinitely and after RPC timeout we got the following error..
> Also we noticed high cpu, high load average and very frequent young gc in
> the region server containing this row...
> scan 'namespace:tablename',\{STARTROW => 'test',ENDROW => 'test', FILTER =>
> "ColumnPrefixFilter('1544770422942010001_') OR
> ColumnPrefixFilter('1544769883529010001_')"}
> ROW COLUMN+CELL
> ERROR: Call id=18,
> waitTime=60005, rpcTimetout=6
>
> Note: Table scan operation and scan with a single column prefix filter works
> fine in this case.
> When we check the same query in hbase-1.2.5 it is working fine.
> Can you please help me on this..
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20209) Do Not Use Both Map containsKey and get Methods in Replication Sink
[
https://issues.apache.org/jira/browse/HBASE-20209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744734#comment-16744734
]
Hudson commented on HBASE-20209:
Results for branch branch-2
[build #1617 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1617/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1617//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1617//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1617//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Do Not Use Both Map containsKey and get Methods in Replication Sink
> ---
>
> Key: HBASE-20209
> URL: https://issues.apache.org/jira/browse/HBASE-20209
> Project: HBase
> Issue Type: Improvement
> Components: Replication
>Affects Versions: 2.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Trivial
> Fix For: 3.0.0, 1.5.0, 2.2.0
>
> Attachments: HBASE-20209.1.patch, HBASE-20209.2.patch,
> HBASE-20209.3.patch, HBASE-20209.4.patch
>
>
> {code:title=ReplicationSink.java}
> String tableName = table.getNameWithNamespaceInclAsString();
> if (bulkLoadHFileMap.containsKey(tableName)) {
> List>> familyHFilePathsList =
> bulkLoadHFileMap.get(tableName);
> boolean foundFamily = false;
> for (int i = 0; i < familyHFilePathsList.size(); i++) {
> Pair> familyHFilePathsPair =
> familyHFilePathsList.get(i);
> if (Bytes.equals(familyHFilePathsPair.getFirst(), family)) {
> // Found family already present, just add the path to the
> existing list
> familyHFilePathsPair.getSecond().add(pathToHfileFromNS);
> foundFamily = true;
> break;
> }
> }
> {code}
> I propose that this code does not use the Map methods _containsKey_ *and*
> _get_. Simply use the _get_ method once and check a _null_ return value to
> check for existence. Saves a trip to the Map data structure for each call.
> Also, use enhanced for loop.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20209) Do Not Use Both Map containsKey and get Methods in Replication Sink
[
https://issues.apache.org/jira/browse/HBASE-20209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744726#comment-16744726
]
Hudson commented on HBASE-20209:
Results for branch master
[build #725 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/725/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/725//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/725//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/725//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Do Not Use Both Map containsKey and get Methods in Replication Sink
> ---
>
> Key: HBASE-20209
> URL: https://issues.apache.org/jira/browse/HBASE-20209
> Project: HBase
> Issue Type: Improvement
> Components: Replication
>Affects Versions: 2.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Trivial
> Fix For: 3.0.0, 1.5.0, 2.2.0
>
> Attachments: HBASE-20209.1.patch, HBASE-20209.2.patch,
> HBASE-20209.3.patch, HBASE-20209.4.patch
>
>
> {code:title=ReplicationSink.java}
> String tableName = table.getNameWithNamespaceInclAsString();
> if (bulkLoadHFileMap.containsKey(tableName)) {
> List>> familyHFilePathsList =
> bulkLoadHFileMap.get(tableName);
> boolean foundFamily = false;
> for (int i = 0; i < familyHFilePathsList.size(); i++) {
> Pair> familyHFilePathsPair =
> familyHFilePathsList.get(i);
> if (Bytes.equals(familyHFilePathsPair.getFirst(), family)) {
> // Found family already present, just add the path to the
> existing list
> familyHFilePathsPair.getSecond().add(pathToHfileFromNS);
> foundFamily = true;
> break;
> }
> }
> {code}
> I propose that this code does not use the Map methods _containsKey_ *and*
> _get_. Simply use the _get_ method once and check a _null_ return value to
> check for existence. Saves a trip to the Map data structure for each call.
> Also, use enhanced for loop.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21726) Add getAllRegionLocations method to AsyncTableRegionLocator
[
https://issues.apache.org/jira/browse/HBASE-21726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744723#comment-16744723
]
Hadoop QA commented on HBASE-21726:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
0s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
58s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 37s{color}
| {color:red} hbase-client generated 1 new + 99 unchanged - 1 fixed = 100 total
(was 100) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
5s{color} | {color:red} hbase-server: The patch generated 1 new + 0 unchanged -
0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
8s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 38s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
44s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
15s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}127m
44s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
10s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}181m 30s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21726 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12955174/HBASE-21726.patch |
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 8041d15c37bd 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/
[jira] [Updated] (HBASE-17165) Add retry to LoadIncrementalHFiles tool
[ https://issues.apache.org/jira/browse/HBASE-17165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-17165: --- Labels: s3 (was: ) > Add retry to LoadIncrementalHFiles tool > --- > > Key: HBASE-17165 > URL: https://issues.apache.org/jira/browse/HBASE-17165 > Project: HBase > Issue Type: Improvement > Components: HFile, tooling >Affects Versions: 1.2.3, 2.0.0 >Reporter: Mike Grimes >Assignee: Mike Grimes >Priority: Critical > Labels: s3 > Fix For: 2.0.0 > > Attachments: HBASE-17165.branch-1.001.patch, > HBASE-17165.branch-1.001.patch, HBASE-17165.branch-1.002.patch, > HBASE-17165.branch-1.002.patch, HBASE-17165.branch-1.2.001.patch, > HBASE-17165.branch-1.2.002.patch, HBASE-17165.branch-1.2.003.patch, > HBASE-17165.branch-1.2.004.patch, HBASE-17165.master.001.patch, > HBASE-17165.master.002.patch, HBASE-17165.master.003.patch, > HBASE-17165.master.004.patch, HBASE-17165.master.004.patch > > Original Estimate: 0h > Remaining Estimate: 0h > > As using the LoadIncrementalHFiles tool with S3 as the filesystem is prone to > failing due to FileNotFoundExceptions due to inconsistency, simple, > configurable retry logic was added. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20430) Improve store file management for non-HDFS filesystems
[ https://issues.apache.org/jira/browse/HBASE-20430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-20430: --- Labels: s3 (was: ) > Improve store file management for non-HDFS filesystems > -- > > Key: HBASE-20430 > URL: https://issues.apache.org/jira/browse/HBASE-20430 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Mingliang Liu >Priority: Major > Labels: s3 > > HBase keeps a file open for every active store file so no additional round > trips to the NameNode are needed after the initial open. HDFS internally > multiplexes open files, but the Hadoop S3 filesystem implementations do not, > or, at least, not as well. As the bulk of data under management increases we > observe the required number of concurrently open connections will rise, and > expect it will eventually exhaust a limit somewhere (the client, the OS file > descriptor table or open file limits, or the S3 service). > Initially we can simply introduce an option to close every store file after > the reader has finished, and determine the performance impact. Use cases > backed by non-HDFS filesystems will already have to cope with a different > read performance profile. Based on experiments with the S3 backed Hadoop > filesystems, notably S3A, even with aggressively tuned options simple reads > can be very slow when there are blockcache misses, 15-20 seconds observed for > Get of a single small row, for example. We expect extensive use of the > BucketCache to mitigate in this application already. Could be backed by > offheap storage, but more likely a large number of cache files managed by the > file engine on local SSD storage. If misses are already going to be super > expensive, then the motivation to do more than simply open store files on > demand is largely absent. > Still, we could employ a predictive cache. Where frequent access to a given > store file (or, at least, its store) is predicted, keep a reference to the > store file open. Can keep statistics about read frequency, write it out to > HFiles during compaction, and note these stats when opening the region, > perhaps by reading all meta blocks of region HFiles when opening. Otherwise, > close the file after reading and open again on demand. Need to be careful not > to use ARC or equivalent as cache replacement strategy as it is encumbered. > The size of the cache can be determined at startup after detecting the > underlying filesystem. Eg. setCacheSize(VERY_LARGE_CONSTANT) if (fs > instanceof DistributedFileSystem), so we don't lose much when on HDFS still. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20431) Store commit transaction for filesystems that do not support an atomic rename
[ https://issues.apache.org/jira/browse/HBASE-20431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-20431: --- Labels: s3 (was: ) > Store commit transaction for filesystems that do not support an atomic rename > - > > Key: HBASE-20431 > URL: https://issues.apache.org/jira/browse/HBASE-20431 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Priority: Major > Labels: s3 > > HBase expects the Hadoop filesystem implementation to support an atomic > rename() operation. HDFS does. The S3 backed filesystems do not. The > fundamental issue is the non-atomic and eventually consistent nature of the > S3 service. A S3 bucket is not a filesystem. S3 is not always immediately > read-your-writes. Object metadata can be temporarily inconsistent just after > new objects are stored. There can be a settling period to ride over. > Renaming/moving objects from one path to another are copy operations with > O(file) complexity and O(data) time followed by a series of deletes with > O(file) complexity. Failures at any point prior to completion will leave the > operation in an inconsistent state. The missing atomic rename semantic opens > opportunities for corruption and data loss, which may or may not be > repairable with HBCK. > Handling this at the HBase level could be done with a new multi-step > filesystem transaction framework. Call it StoreCommitTransaction. > SplitTransaction and MergeTransaction are well established cases where even > on HDFS we have non-atomic filesystem changes and are our implementation > template for the new work. In this new StoreCommitTransaction we'd be moving > flush and compaction temporaries out of the temporary directory into the > region store directory. On HDFS the implementation would be easy. We can rely > on the filesystem's atomic rename semantics. On S3 it would be work: First we > would build the list of objects to move, then copy each object into the > destination, and then finally delete all objects at the original path. We > must handle transient errors with retry strategies appropriate for the action > at hand. We must handle serious or permanent errors where the RS doesn't need > to be aborted with a rollback that cleans it all up. Finally, we must handle > permanent errors where the RS must be aborted with a rollback during region > open/recovery. Note that after all objects have been copied and we are > deleting obsolete source objects we must roll forward, not back. To support > recovery after an abort we must utilize the WAL to track transaction > progress. Put markers in for StoreCommitTransaction start and completion > state, with details of the store file(s) involved, so it can be rolled back > during region recovery at open. This will be significant work in HFile, > HStore, flusher, compactor, and HRegion. Wherever we use HDFS's rename now we > would substitute the running of this new multi-step filesystem transaction. > We need to determine this for certain, but I believe on S3 the PUT or > multipart upload of an object must complete before the object is visible, so > we don't have to worry about the case where an object is visible before fully > uploaded as part of normal operations. So an individual object copy will > either happen entirely and the target will then become visible, or it won't > and the target won't exist. > S3 has an optimization, PUT COPY > (https://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectCOPY.html), which > the AmazonClient embedded in S3A utilizes for moves. When designing the > StoreCommitTransaction be sure to allow for filesystem implementations that > leverage a server side copy operation. Doing a get-then-put should be > optional. (Not sure Hadoop has an interface that advertises this capability > yet; we can add one if not.) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20723) Custom hbase.wal.dir results in data loss because we write recovered edits into a different place than where the recovering region server looks for them
[
https://issues.apache.org/jira/browse/HBASE-20723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nihal Jain updated HBASE-20723:
---
Labels: s3 (was: )
> Custom hbase.wal.dir results in data loss because we write recovered edits
> into a different place than where the recovering region server looks for them
>
>
> Key: HBASE-20723
> URL: https://issues.apache.org/jira/browse/HBASE-20723
> Project: HBase
> Issue Type: Bug
> Components: Recovery, wal
>Affects Versions: 1.4.0, 1.4.1, 1.4.2, 1.4.3, 1.4.4, 2.0.0
>Reporter: Rohan Pednekar
>Assignee: Ted Yu
>Priority: Critical
> Labels: s3
> Fix For: 2.1.0, 1.5.0, 1.3.3, 1.4.6
>
> Attachments: 20723.branch-1.txt, 20723.branch-2.txt, 20723.v1.txt,
> 20723.v10.txt, 20723.v2.txt, 20723.v3.txt, 20723.v4.txt, 20723.v5.txt,
> 20723.v5.txt, 20723.v6.txt, 20723.v7.txt, 20723.v8.txt, 20723.v9.txt, logs.zip
>
>
> Description:
> When custom hbase.wal.dir is configured the recovery system uses it in place
> of the HBase root dir and thus constructs an incorrect path for recovered
> edits when splitting WALs. This causes the recovery code in Region Servers to
> believe there are no recovered edits to replay, which causes a loss of writes
> that had not flushed prior to loss of a server.
>
> Reproduction:
> This is an Azure HDInsight HBase cluster with HDP 2.6. and HBase
> 1.1.2.2.6.3.2-14
> By default the underlying data is going to wasb://x@y/hbase
> I tried to move WAL folders to HDFS, which is the SSD mounted on each VM at
> /mnt.
> hbase.wal.dir= hdfs://mycluster/walontest
> hbase.wal.dir.perms=700
> hbase.rootdir.perms=700
> hbase.rootdir=
> wasb://XYZ[@hbaseperf.core.net|mailto:duohbase5ds...@duohbaseperf.blob.core.windows.net]/hbase
> Procedure to reproduce this issue:
> 1. create a table in hbase shell
> 2. insert a row in hbase shell
> 3. reboot the VM which hosts that region
> 4. scan the table in hbase shell and it is empty
> Looking at the region server logs:
> {code:java}
> 2018-06-12 22:08:40,455 INFO [RS_LOG_REPLAY_OPS-wn2-duohba:16020-0-Writer-1]
> wal.WALSplitter: This region's directory doesn't exist:
> hdfs://mycluster/walontest/data/default/tb1/b7fd7db5694eb71190955292b3ff7648.
> It is very likely that it was already split so it's safe to discard those
> edits.
> {code}
> The log split/replay ignored actual WAL due to WALSplitter is looking for the
> region directory in the hbase.wal.dir we specified rather than the
> hbase.rootdir.
> Looking at the source code,
>
> [https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java]
> it uses the rootDir, which is walDir, as the tableDir root path.
> So if we use HBASE-17437, waldir and hbase rootdir are in different path or
> even in different filesystem, then the #5 uses walDir as tableDir is
> apparently wrong.
> CC: [~zyork], [~yuzhih...@gmail.com] Attached the logs for quick review.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20218) Proposed Performance Enhancements For TableSnapshotInputFomat
[
https://issues.apache.org/jira/browse/HBASE-20218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744707#comment-16744707
]
Hadoop QA commented on HBASE-20218:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 6s{color}
| {color:red} HBASE-20218 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.8.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-20218 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12915171/HBASE-20218.01.patch |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/15611/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Proposed Performance Enhancements For TableSnapshotInputFomat
> -
>
> Key: HBASE-20218
> URL: https://issues.apache.org/jira/browse/HBASE-20218
> Project: HBase
> Issue Type: Improvement
> Components: mapreduce
>Affects Versions: 1.4.0
> Environment: HBase 1.4.0 running in AWS EMR 5.12.0 with the HBase
> rootdir set to a folder in S3
>
>Reporter: Saad Mufti
>Priority: Minor
> Labels: s3
> Attachments: HBASE-20218.01.patch, HBASE-20218.patch
>
>
> I have been testing a few Spark jobs we have at my company which work off of
> TableSnapshotInputFormat to read directly from the filesystem snapshots
> created on another EMR/Hbase cluster and stored in S3. During performance
> testing I found various small changes which would greatly enhance peformance.
> Right now we are running our jobs linked with a patched version of HBase
> 1.4.0 in which I made these changes, and I am hoping to submit my patch for
> review and eventual acceptance into the main codebase.
>
> The list of changes are :
>
> 1. a flag to control whether the snapshot restore uses a UUID based random
> temp dir in the specified restore directory. We use the flag to turn this off
> so that we can benefit from a AWS S3 specific bucket partitioning scheme we
> have provisioned. The way S3 partitioning works, you have to give a fixed
> path prefix and a pattern of files after that, and AWS can then partition on
> the paths after the fixed prefix into different resources to get more
> parallelization. We were advised by AWS that we could only get this good
> partitioning behavior if we didn't have that rancom directory in there.
>
> 2. a flag to turn off the code that tries to compute locality information
> for the splits. This is useless when dealing with S3 since the files are not
> on the cluster so there is no use in computing locality; and worse yet, it
> uses a single thread in the driver to iterate over all the files in the
> restored snapshot. For a very large table this was taking hours and hours
> iterating through S3 objects just to list them (about 360,000 of them for the
> our specific table).
>
> 3. a flag to override the column family schema setting to prefetch regions on
> open. This was causing the main executor thread on which a Spark task was
> running, which was trying to read through HFile's for its scan, compete for a
> lock on the underlying EMRFS stream object with prefetch threads trying to
> read the same file, so most tasks in the Spark stage would finish but the
> last few would linger half an hour or more competing with the prefetch
> threads alternately for a lock on an EMRFS stream object. This is the only
> change that had to be outside the mapreduce package as it directly affects
> the prefetch behavior in CacheConfig.java
>
> 4. a flag to turn off maintenance of Scan metrics. this was also causing a
> major slowdown, getting rid of this sped things up 4-5 times. What I observed
> in the thread dumps was that every call to update scan metrics was trying to
> get some HBase counter object and deep underneath was trying to access some
> Java resource bundle, and throwing an exception that it wasn't found. The
> exception was never visible at the application level and was swallowed
> underneath but whatever it was doing was causing a major slowdown. So we use
> this flag to avoid collecting those metrics because we never used them
>
> I am polishing my patch a bit more and hopefully will attach it tomorrow. One
> caveat, I tried but struggled with how to write any useful unit/component
> tests for these as these are invisible behaviors that do not affect the final
> result at all. And I am not that familiar with the HBa
[jira] [Updated] (HBASE-15646) Add some docs about exporting and importing snapshots using S3
[ https://issues.apache.org/jira/browse/HBASE-15646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-15646: --- Labels: s3 (was: ) > Add some docs about exporting and importing snapshots using S3 > -- > > Key: HBASE-15646 > URL: https://issues.apache.org/jira/browse/HBASE-15646 > Project: HBase > Issue Type: Task > Components: documentation, snapshots >Affects Versions: 2.0.0 >Reporter: Misty Linville >Assignee: Misty Linville >Priority: Major > Labels: s3 > Fix For: 2.0.0 > > Attachments: HBASE-15646-v1.patch, HBASE-15646-v2.patch, > HBASE-15646-v3.patch, HBASE-15646-v4.patch, HBASE-15646.patch > > > Got a request to add this and wrote something up using the info I could > scrounge together. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-17437) Support specifying a WAL directory outside of the root directory
[ https://issues.apache.org/jira/browse/HBASE-17437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-17437: --- Labels: patch s3 (was: patch) > Support specifying a WAL directory outside of the root directory > > > Key: HBASE-17437 > URL: https://issues.apache.org/jira/browse/HBASE-17437 > Project: HBase > Issue Type: Improvement > Components: Filesystem Integration, wal >Affects Versions: 1.2.4 >Reporter: Yishan Yang >Assignee: Zach York >Priority: Major > Labels: patch, s3 > Fix For: 1.4.0, 1.3.3, 2.0.0 > > Attachments: HBASE-17437.branch-1.001.patch, > HBASE-17437.branch-1.002.patch, HBASE-17437.branch-1.003.patch, > HBASE-17437.branch-1.004.patch, HBASE-17437.master.001.patch, > HBASE-17437.master.002.patch, HBASE-17437.master.003.patch, > HBASE-17437.master.004.patch, HBASE-17437.master.005.patch, > HBASE-17437.master.006.patch, HBASE-17437.master.007.patch, > HBASE-17437.master.008.patch, HBASE-17437.master.009.patch, > HBASE-17437.master.010.patch, HBASE-17437.master.011.patch, > HBASE-17437.master.012.patch, hbase-17437-branch-1.2.patch, > hbase-17437-master.patch > > > Currently, the WAL and the StoreFiles need to be on the same FileSystem. Some > FileSystems (such as Amazon S3) don’t support append or consistent writes. > These two properties are imperative for the WAL in order to avoid loss of > writes. However, StoreFiles don’t necessarily need the same consistency > guarantees (since writes are cached locally and if writes fail, they can > always be replayed from the WAL). > > This JIRA aims to allow users to configure a log directory (for WALs) that is > outside of the root directory or even in a different FileSystem. The default > value will still put the log directory under the root directory. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-17878) java.lang.NoSuchMethodError: org.joda.time.format.DateTimeFormatter.withZoneUTC()Lorg/joda/time/format/DateTimeFormatter when starting HBase with hbase.rootdir on S3
[
https://issues.apache.org/jira/browse/HBASE-17878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nihal Jain updated HBASE-17878:
---
Labels: s3 (was: )
> java.lang.NoSuchMethodError:
> org.joda.time.format.DateTimeFormatter.withZoneUTC()Lorg/joda/time/format/DateTimeFormatter
> when starting HBase with hbase.rootdir on S3
> -
>
> Key: HBASE-17878
> URL: https://issues.apache.org/jira/browse/HBASE-17878
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Labels: s3
> Fix For: 2.0.0
>
> Attachments: HBASE-17878.master.000.patch, jruby-core-dep-tree.txt
>
>
> When setting up HBASE-17437 (Support specifying a WAL directory outside of
> the root directory), we specify
> (1) hbase.rootdir on s3a
> (2) hbase.wal.dir on HDFS
> When starting HBase, the following exception is thrown:
> {code}
> Caused by: java.lang.NoSuchMethodError:
> org.joda.time.format.DateTimeFormatter.withZoneUTC()Lorg/joda/time/format/DateTimeFormatter;
> at
> com.amazonaws.auth.internal.AWS4SignerUtils.(AWS4SignerUtils.java:26)
> at
> com.amazonaws.auth.internal.AWS4SignerRequestParams.(AWS4SignerRequestParams.java:85)
> at com.amazonaws.auth.AWS4Signer.sign(AWS4Signer.java:184)
> at
> com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:709)
> at
> com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:489)
> at
> com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:310)
> at
> com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3785)
> at
> com.amazonaws.services.s3.AmazonS3Client.headBucket(AmazonS3Client.java:1107)
> at
> com.amazonaws.services.s3.AmazonS3Client.doesBucketExist(AmazonS3Client.java:1070)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:232)
> at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2669)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:94)
> at
> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2703)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2685)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:373)
> at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
> at org.apache.hadoop.hbase.util.FSUtils.getRootDir(FSUtils.java:1007)
> at
> org.apache.hadoop.hbase.util.FSUtils.isValidWALRootDir(FSUtils.java:1050)
> at
> org.apache.hadoop.hbase.util.FSUtils.getWALRootDir(FSUtils.java:1032)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.initializeFileSystem(HRegionServer.java:627)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:570)
> at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:393)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2456)
> ... 5 more
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-17997) In dev environment, add jruby-complete jar to classpath only when jruby is needed
[
https://issues.apache.org/jira/browse/HBASE-17997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nihal Jain updated HBASE-17997:
---
Labels: s3 (was: )
> In dev environment, add jruby-complete jar to classpath only when jruby is
> needed
> -
>
> Key: HBASE-17997
> URL: https://issues.apache.org/jira/browse/HBASE-17997
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Xiang Li
>Priority: Major
> Labels: s3
> Fix For: 2.0.0
>
> Attachments: HBASE-17997.master.000.patch,
> HBASE-17997.master.001.patch, HBASE-17997.master.002.patch,
> HBASE-17997.master.003.debug.diff, HBASE-17997.master.003.patch
>
>
> HBASE-15199 moves jruby-complete-1.6.8.jar to lib/ruby directory.
> However, jruby-complete-1.6.8.jar still appears in cached_classpath.txt
> This means that user would see exception similar to the following when
> starting hbase in standalone mode with s3a as rootdir :
> {code}
> 2017-05-04 16:41:32,854 WARN
> [RpcServer.FifoWFPBQ.priority.handler=18,queue=0,port=38659]
> internal.S3MetadataResponseHandler: Unable to parse last modified date: Thu,
> 04 May 2017 16:27:09 GMT
> java.lang.IllegalStateException: Joda-time 2.2 or later version is required,
> but found version: null
> at com.amazonaws.util.DateUtils.handleException(DateUtils.java:149)
> at com.amazonaws.util.DateUtils.parseRFC822Date(DateUtils.java:195)
> at
> com.amazonaws.services.s3.internal.ServiceUtils.parseRfc822Date(ServiceUtils.java:78)
> at
> com.amazonaws.services.s3.internal.AbstractS3ResponseHandler.populateObjectMetadata(AbstractS3ResponseHandler.java:115)
> at
> com.amazonaws.services.s3.internal.S3ObjectResponseHandler.handle(S3ObjectResponseHandler.java:52)
> at
> com.amazonaws.services.s3.internal.S3ObjectResponseHandler.handle(S3ObjectResponseHandler.java:30)
> at
> com.amazonaws.http.AmazonHttpClient.handleResponse(AmazonHttpClient.java:1072)
> at
> com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:746)
> at
> com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:489)
> at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:310)
> at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3785)
> at
> com.amazonaws.services.s3.AmazonS3Client.getObject(AmazonS3Client.java:1191)
> at org.apache.hadoop.fs.s3a.S3AInputStream.reopen(S3AInputStream.java:148)
> at org.apache.hadoop.fs.s3a.S3AInputStream.lazySeek(S3AInputStream.java:281)
> at org.apache.hadoop.fs.s3a.S3AInputStream.read(S3AInputStream.java:364)
> at org.apache.hadoop.fs.FSInputStream.read(FSInputStream.java:75)
> at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:92)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlock.positionalReadWithExtra(HFileBlock.java:722)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1420)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockDataInternal(HFileBlock.java:1677)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockData(HFileBlock.java:1504)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:439)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:904)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:267)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:169)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.seekScanners(StoreScanner.java:363)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:217)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2132)
> at org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2122)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.(HRegion.java:5687)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2679)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2665)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2647)
> at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6906)
> at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6885)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2007)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21098) Improve Snapshot Performance with Temporary Snapshot Directory when rootDir on S3
[ https://issues.apache.org/jira/browse/HBASE-21098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-21098: --- Labels: s3 (was: ) > Improve Snapshot Performance with Temporary Snapshot Directory when rootDir > on S3 > - > > Key: HBASE-21098 > URL: https://issues.apache.org/jira/browse/HBASE-21098 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0, 1.4.8, 2.1.1 >Reporter: Tyler Mi >Assignee: Tyler Mi >Priority: Major > Labels: s3 > Fix For: 3.0.0, 1.5.0, 2.2.0, 1.4.9 > > Attachments: HBASE-21098.branch-1.001.patch, > HBASE-21098.branch-1.002.patch, HBASE-21098.master.001.patch, > HBASE-21098.master.002.patch, HBASE-21098.master.003.patch, > HBASE-21098.master.004.patch, HBASE-21098.master.005.patch, > HBASE-21098.master.006.patch, HBASE-21098.master.007.patch, > HBASE-21098.master.008.patch, HBASE-21098.master.009.patch, > HBASE-21098.master.010.patch, HBASE-21098.master.011.patch, > HBASE-21098.master.012.patch, HBASE-21098.master.013.patch > > > When using Apache HBase, the snapshot feature can be used to make a point in > time recovery. To do this, HBase creates a manifest of all the files in all > of the Regions so that those files can be referenced again when a user > restores a snapshot. With HBase's S3 storage mode, developers can store their > data off-cluster on Amazon S3. However, utilizing S3 as a file system is > inefficient in some operations, namely renames. Most Hadoop ecosystem > applications use an atomic rename as a method of committing data. However, > with S3, a rename is a separate copy and then a delete of every file which is > no longer atomic and, in fact, quite costly. In addition, puts and deletes on > S3 have latency issues that traditional filesystems do not encounter when > manipulating the region snapshots to consolidate into a single manifest. When > HBase on S3 users have a significant amount of regions, puts, deletes, and > renames (the final commit stage of the snapshot) become the bottleneck > causing snapshots to take many minutes or even hours to complete. > The purpose of this patch is to increase the overall performance of snapshots > while utilizing HBase on S3 through the use of a temporary directory for the > snapshots that exists on a traditional filesystem like HDFS to circumvent the > bottlenecks. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18209) Include httpclient / httpcore jars in build artifacts
[ https://issues.apache.org/jira/browse/HBASE-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-18209: --- Labels: s3 (was: ) > Include httpclient / httpcore jars in build artifacts > - > > Key: HBASE-18209 > URL: https://issues.apache.org/jira/browse/HBASE-18209 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Major > Labels: s3 > Fix For: 2.0.0-alpha-2, 2.0.0 > > Attachments: 18209.v1.txt, 18209.v2.txt > > > We need httpclient & httpcore jars to be present when rootdir is placed on > s3(a). > Attempts to move to the fully shaded amazon-SDK JAR caused problems of its > own. (according to [~steve_l]) > Here are the versions we should use: > 4.5.2 > 4.4.4 > Currently they are declared test dependency. > This JIRA is to move to compile time dependency so that the corresponding > jars are bundled in lib directory. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18784) Use of filesystem that requires hflush / hsync / append / etc should query outputstream capabilities
[ https://issues.apache.org/jira/browse/HBASE-18784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-18784: --- Labels: s3 (was: ) > Use of filesystem that requires hflush / hsync / append / etc should query > outputstream capabilities > > > Key: HBASE-18784 > URL: https://issues.apache.org/jira/browse/HBASE-18784 > Project: HBase > Issue Type: Improvement > Components: Filesystem Integration >Affects Versions: 1.4.0, 2.0.0-alpha-2 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Labels: s3 > Fix For: 2.0.0 > > Attachments: HBASE-18784-branch-1.v2.patch, HBASE-18784.0.patch, > HBASE-18784.1.patch, HBASE-18784.2.patch > > > In places where we rely on the underlying filesystem holding up the promises > of hflush/hsync (most importantly the WAL), we should use the new interfaces > provided by HDFS-11644 to fail loudly when they are not present (e.g. on S3, > on EC mounts, etc). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20140) HRegion FileSystem should be instantiated from hbase rootDir not default
[
https://issues.apache.org/jira/browse/HBASE-20140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nihal Jain updated HBASE-20140:
---
Labels: s3 (was: )
> HRegion FileSystem should be instantiated from hbase rootDir not default
>
>
> Key: HBASE-20140
> URL: https://issues.apache.org/jira/browse/HBASE-20140
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Affects Versions: 3.0.0, 2.0.2, 1.4.7
>Reporter: Adrian Muraru
>Assignee: Adrian Muraru
>Priority: Minor
> Labels: s3
> Attachments: HBASE-20140.branch-1.v01.patch,
> HBASE-20140.branch-2.v01.patch, HBASE-20140.v01.patch
>
>
> HRegion fs initialization is done based on HDFS default {{fs.defaultFs}}
> instead of {{hbase.rootdir}}
> This is breaking in cases where the {{fs.defaultFs}} is set to a different
> filesystem than the one set in fully qualified {{hbase.rootdir}}
>
> The usecase is the following: running an yarn cluster with `fs.defaultFs` set
> to local HDFS and running a MR job over HBase Snapshots setting the
> {{hbase.rootDir}} to external S3 Filesystem (fully qualified). In this case
> when the regions are actually cold loaded in map tasks the defaultFs is used
> instead of actual hbase.rootDir
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18478) Allow users to remove RegionFinder from LoadBalancer calculations if no locality possible
[ https://issues.apache.org/jira/browse/HBASE-18478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-18478: --- Labels: s3 (was: ) > Allow users to remove RegionFinder from LoadBalancer calculations if no > locality possible > - > > Key: HBASE-18478 > URL: https://issues.apache.org/jira/browse/HBASE-18478 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: Zach York >Assignee: Zach York >Priority: Major > Labels: s3 > Fix For: 1.4.0, 2.0.0 > > Attachments: HBASE-18478.branch-1.001.patch, > HBASE-18478.master.001.patch > > > BaseLoadBalancer should have the option to remove RegionFinder from load > balancing. This provides significant cluster start time reduction for > FileSystems that do not surface locality such as Amazon S3. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18885) HFileOutputFormat2 hardcodes default FileOutputCommitter
[ https://issues.apache.org/jira/browse/HBASE-18885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-18885: --- Labels: s3 (was: ) > HFileOutputFormat2 hardcodes default FileOutputCommitter > > > Key: HBASE-18885 > URL: https://issues.apache.org/jira/browse/HBASE-18885 > Project: HBase > Issue Type: Bug > Components: mapreduce >Reporter: Shaofeng SHI >Assignee: Shaofeng SHI >Priority: Major > Labels: s3 > Fix For: 1.4.0, 1.3.2, 2.0.0-alpha-4, 2.0.0, 1.2.7 > > Attachments: HBASE-18885.branch-1.001.patch, > HBASE-18885.master.001.patch > > > Apache Kylin uses HBase's HFileOutputFormat2.java to configure the MR job. > The original reporting is in KYLIN-2788[1]. After some investigation, we > found this class always uses the default "FileOutputCommitter", see [2], > regardless of the job's configuration; so it always writing to "_temporary" > folder. Since AWS EMR configured to use DirectOutputCommitter for S3, then > this problem occurs: Hadoop expects to see the file directly under output > path, while the RecordWriter generates them in "_temporary" folder. This > caused no data be loaded to HTable. > Seems this problem exists in all versions so far. > [1] https://issues.apache.org/jira/browse/KYLIN-2788 > [2] > https://github.com/apache/hbase/blob/master/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat2.java#L193 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20429) Support for mixed or write-heavy workloads on non-HDFS filesystems
[ https://issues.apache.org/jira/browse/HBASE-20429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-20429: --- Labels: s3 (was: ) > Support for mixed or write-heavy workloads on non-HDFS filesystems > -- > > Key: HBASE-20429 > URL: https://issues.apache.org/jira/browse/HBASE-20429 > Project: HBase > Issue Type: Umbrella >Reporter: Andrew Purtell >Priority: Major > Labels: s3 > > We can support reasonably well use cases on non-HDFS filesystems, like S3, > where an external writer has loaded (and continues to load) HFiles via the > bulk load mechanism, and then we serve out a read only workload at the HBase > API. > Mixed workloads or write-heavy workloads won't fare as well. In fact, data > loss seems certain. It will depend in the specific filesystem, but all of the > S3 backed Hadoop filesystems suffer from a couple of obvious problems, > notably a lack of atomic rename. > This umbrella will serve to collect some related ideas for consideration. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20218) Proposed Performance Enhancements For TableSnapshotInputFomat
[ https://issues.apache.org/jira/browse/HBASE-20218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-20218: --- Labels: s3 (was: ) > Proposed Performance Enhancements For TableSnapshotInputFomat > - > > Key: HBASE-20218 > URL: https://issues.apache.org/jira/browse/HBASE-20218 > Project: HBase > Issue Type: Improvement > Components: mapreduce >Affects Versions: 1.4.0 > Environment: HBase 1.4.0 running in AWS EMR 5.12.0 with the HBase > rootdir set to a folder in S3 > >Reporter: Saad Mufti >Priority: Minor > Labels: s3 > Attachments: HBASE-20218.01.patch, HBASE-20218.patch > > > I have been testing a few Spark jobs we have at my company which work off of > TableSnapshotInputFormat to read directly from the filesystem snapshots > created on another EMR/Hbase cluster and stored in S3. During performance > testing I found various small changes which would greatly enhance peformance. > Right now we are running our jobs linked with a patched version of HBase > 1.4.0 in which I made these changes, and I am hoping to submit my patch for > review and eventual acceptance into the main codebase. > > The list of changes are : > > 1. a flag to control whether the snapshot restore uses a UUID based random > temp dir in the specified restore directory. We use the flag to turn this off > so that we can benefit from a AWS S3 specific bucket partitioning scheme we > have provisioned. The way S3 partitioning works, you have to give a fixed > path prefix and a pattern of files after that, and AWS can then partition on > the paths after the fixed prefix into different resources to get more > parallelization. We were advised by AWS that we could only get this good > partitioning behavior if we didn't have that rancom directory in there. > > 2. a flag to turn off the code that tries to compute locality information > for the splits. This is useless when dealing with S3 since the files are not > on the cluster so there is no use in computing locality; and worse yet, it > uses a single thread in the driver to iterate over all the files in the > restored snapshot. For a very large table this was taking hours and hours > iterating through S3 objects just to list them (about 360,000 of them for the > our specific table). > > 3. a flag to override the column family schema setting to prefetch regions on > open. This was causing the main executor thread on which a Spark task was > running, which was trying to read through HFile's for its scan, compete for a > lock on the underlying EMRFS stream object with prefetch threads trying to > read the same file, so most tasks in the Spark stage would finish but the > last few would linger half an hour or more competing with the prefetch > threads alternately for a lock on an EMRFS stream object. This is the only > change that had to be outside the mapreduce package as it directly affects > the prefetch behavior in CacheConfig.java > > 4. a flag to turn off maintenance of Scan metrics. this was also causing a > major slowdown, getting rid of this sped things up 4-5 times. What I observed > in the thread dumps was that every call to update scan metrics was trying to > get some HBase counter object and deep underneath was trying to access some > Java resource bundle, and throwing an exception that it wasn't found. The > exception was never visible at the application level and was swallowed > underneath but whatever it was doing was causing a major slowdown. So we use > this flag to avoid collecting those metrics because we never used them > > I am polishing my patch a bit more and hopefully will attach it tomorrow. One > caveat, I tried but struggled with how to write any useful unit/component > tests for these as these are invisible behaviors that do not affect the final > result at all. And I am not that familiar with the HBase testing standards, > so for now I am looking for guidance on what to tests. > > Would appreciate any feedback plus guidance on writing tests, provided of > course there is interest in incorporating my patch into the main codebase. > > Cheers. > > Saad > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21070) SnapshotFileCache won't update for snapshots stored in S3
[ https://issues.apache.org/jira/browse/HBASE-21070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-21070: --- Labels: FSRedo s3 (was: FSRedo) > SnapshotFileCache won't update for snapshots stored in S3 > - > > Key: HBASE-21070 > URL: https://issues.apache.org/jira/browse/HBASE-21070 > Project: HBase > Issue Type: Bug > Components: snapshots >Affects Versions: 3.0.0, 2.1.1, 1.4.7 >Reporter: Zach York >Assignee: Zach York >Priority: Critical > Labels: FSRedo, s3 > Fix For: 3.0.0, 1.5.0, 2.2.0 > > Attachments: HBASE-21070.master.001.patch, > HBASE-21070.master.002.patch, HBASE-21070.master.003.patch > > > The SnapshotFileCache depends on last modified time to determine whether to > update the Snapshot HFile cache. However, in S3, real 'folders' don't exist. > S3 filesystems create a dummy file in place of a folder, but the dummy file > last modified time is not updated when files are changed 'under' it. This > means that the SnapshotFileCache doesn't pick up new snapshot HFiles and > these files aren't removed from the HFileCleaner and can be eligible for > deletion. > > My patch removes the lastmodified assumption. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-17370) Fix or provide shell scripts to drain and decommission region server
[ https://issues.apache.org/jira/browse/HBASE-17370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744701#comment-16744701 ] Nihal Jain commented on HBASE-17370: Thanks for the review [~zghaobac]. I have tested the new patch on a 3 node-cluster, works as expected. > Fix or provide shell scripts to drain and decommission region server > > > Key: HBASE-17370 > URL: https://issues.apache.org/jira/browse/HBASE-17370 > Project: HBase > Issue Type: Sub-task >Reporter: Jerry He >Assignee: Nihal Jain >Priority: Major > Labels: operability > Attachments: HBASE-17370.master.001.patch, > HBASE-17370.master.002.patch > > > 1. Update the existing shell scripts to use the new drain related API. > 2 Or provide new shell scripts. > 3. Provide a 'decommission' shell tool that puts the server in drain mode and > offload the server. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20209) Do Not Use Both Map containsKey and get Methods in Replication Sink
[
https://issues.apache.org/jira/browse/HBASE-20209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744695#comment-16744695
]
Hudson commented on HBASE-20209:
Results for branch branch-1
[build #639 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/639/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/639//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/639//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/639//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> Do Not Use Both Map containsKey and get Methods in Replication Sink
> ---
>
> Key: HBASE-20209
> URL: https://issues.apache.org/jira/browse/HBASE-20209
> Project: HBase
> Issue Type: Improvement
> Components: Replication
>Affects Versions: 2.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Trivial
> Fix For: 3.0.0, 1.5.0, 2.2.0
>
> Attachments: HBASE-20209.1.patch, HBASE-20209.2.patch,
> HBASE-20209.3.patch, HBASE-20209.4.patch
>
>
> {code:title=ReplicationSink.java}
> String tableName = table.getNameWithNamespaceInclAsString();
> if (bulkLoadHFileMap.containsKey(tableName)) {
> List>> familyHFilePathsList =
> bulkLoadHFileMap.get(tableName);
> boolean foundFamily = false;
> for (int i = 0; i < familyHFilePathsList.size(); i++) {
> Pair> familyHFilePathsPair =
> familyHFilePathsList.get(i);
> if (Bytes.equals(familyHFilePathsPair.getFirst(), family)) {
> // Found family already present, just add the path to the
> existing list
> familyHFilePathsPair.getSecond().add(pathToHfileFromNS);
> foundFamily = true;
> break;
> }
> }
> {code}
> I propose that this code does not use the Map methods _containsKey_ *and*
> _get_. Simply use the _get_ method once and check a _null_ return value to
> check for existence. Saves a trip to the Map data structure for each call.
> Also, use enhanced for loop.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21616) Port HBASE-21034 (Add new throttle type: read/write capacity unit) to branch-1
[
https://issues.apache.org/jira/browse/HBASE-21616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744696#comment-16744696
]
Hudson commented on HBASE-21616:
Results for branch branch-1
[build #639 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/639/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/639//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/639//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/639//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> Port HBASE-21034 (Add new throttle type: read/write capacity unit) to branch-1
> --
>
> Key: HBASE-21616
> URL: https://issues.apache.org/jira/browse/HBASE-21616
> Project: HBase
> Issue Type: Sub-task
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Major
> Fix For: 1.5.0
>
> Attachments: HBASE-21616-branch-1.patch, HBASE-21616-branch-1.patch,
> HBASE-21616-branch-1.patch
>
>
> Port HBASE-21034 (Add new throttle type: read/write capacity unit) to branch-1
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21642) CopyTable by reading snapshot and bulkloading will save a lot of time.
[
https://issues.apache.org/jira/browse/HBASE-21642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744694#comment-16744694
]
Hudson commented on HBASE-21642:
Results for branch branch-1
[build #639 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/639/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/639//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/639//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/639//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> CopyTable by reading snapshot and bulkloading will save a lot of time.
> --
>
> Key: HBASE-21642
> URL: https://issues.apache.org/jira/browse/HBASE-21642
> Project: HBase
> Issue Type: Improvement
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21642.v1.patch
>
>
> In our HBase clusters, some users has the need to merge two diff table's
> data into one. Currently , the CopyTable will scan the source table , and
> put mutations into destination table.
> Although CopyTable with bulkload can speed a lot (compared to CopyTable with
> scan and put), it still take lots of time to scan the source table. and the
> worst thing is: CopyTable with scan table will impact the cluster's
> availablity, it cost lots of resource in RS to scanning, the cpu, memory,
> gc stw, rs handlers, disk io, network io ... etc. All those things will
> affect the availablity.
> So in our clusters, we tried to do all scanning job by using scan snapshot
> instead of scan table. it at least isolate the cpu & memory & gc resource
> between the online RS and scanning job. What's more, the snapshot scanning
> is much faster than scaning RS, and it's more stable.
> So, here, I'll make the copy table tool support snapshot scanning.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21675) Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a lot of time) to branch-1
[
https://issues.apache.org/jira/browse/HBASE-21675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744693#comment-16744693
]
Hudson commented on HBASE-21675:
Results for branch branch-1
[build #639 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/639/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/639//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/639//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/639//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a
> lot of time) to branch-1
>
>
> Key: HBASE-21675
> URL: https://issues.apache.org/jira/browse/HBASE-21675
> Project: HBase
> Issue Type: Sub-task
>Reporter: Andrew Purtell
>Assignee: Zheng Hu
>Priority: Major
> Fix For: 1.5.0
>
> Attachments: HBASE-21675-branch-1-WIP.patch,
> HBASE-21675.branch-1.v2.patch, HBASE-21675.branch-1.v3.patch,
> HBASE-21675.branch-1.v4.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21034) Add new throttle type: read/write capacity unit
[
https://issues.apache.org/jira/browse/HBASE-21034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744697#comment-16744697
]
Hudson commented on HBASE-21034:
Results for branch branch-1
[build #639 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/639/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/639//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/639//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/639//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> Add new throttle type: read/write capacity unit
> ---
>
> Key: HBASE-21034
> URL: https://issues.apache.org/jira/browse/HBASE-21034
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 3.0.0, 2.2.0
>Reporter: Yi Mei
>Assignee: Yi Mei
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21034.branch-2.0.001.patch,
> HBASE-21034.branch-2.0.001.patch, HBASE-21034.branch-2.1.001.patch,
> HBASE-21034.branch-2.1.001.patch, HBASE-21034.master.001.patch,
> HBASE-21034.master.002.patch, HBASE-21034.master.003.patch,
> HBASE-21034.master.004.patch, HBASE-21034.master.005.patch,
> HBASE-21034.master.006.patch, HBASE-21034.master.006.patch,
> HBASE-21034.master.007.patch, HBASE-21034.master.007.patch
>
>
> Add new throttle type: read/write capacity unit like DynamoDB.
> One read capacity unit represents that read up to 1K data per time unit. If
> data size is more than 1K, then consume additional read capacity units.
> One write capacity unit represents that one write for an item up to 1 KB in
> size per time unit. If data size is more than 1K, then consume additional
> write capacity units.
> For example, 100 read capacity units per second means that, HBase user can
> read 100 times for 1K data in every second, or 50 times for 2K data in every
> second and so on.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21686) Release 1.2.10
[
https://issues.apache.org/jira/browse/HBASE-21686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744688#comment-16744688
]
Hudson commented on HBASE-21686:
Results for branch branch-1.2
[build #629 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/629/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/629//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/629//JDK7_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/629//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Release 1.2.10
> --
>
> Key: HBASE-21686
> URL: https://issues.apache.org/jira/browse/HBASE-21686
> Project: HBase
> Issue Type: Task
> Components: community
>Affects Versions: 1.2.10
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 1.2.10
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21686) Release 1.2.10
[ https://issues.apache.org/jira/browse/HBASE-21686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744661#comment-16744661 ] Hudson commented on HBASE-21686: SUCCESS: Integrated in Jenkins build HBase-1.2-IT #1201 (See [https://builds.apache.org/job/HBase-1.2-IT/1201/]) HBASE-21686 mark release date of 1.2.10 in CHANGES; set version to (busbey: [https://github.com/apache/hbase/commit/dab4bf76d29d09ad3d14697a4626f15a6406e670]) * (edit) hbase-shell/pom.xml * (edit) hbase-thrift/pom.xml * (edit) hbase-server/pom.xml * (edit) hbase-examples/pom.xml * (edit) hbase-resource-bundle/pom.xml * (edit) hbase-checkstyle/pom.xml * (edit) hbase-external-blockcache/pom.xml * (edit) hbase-protocol/pom.xml * (edit) CHANGES.txt * (edit) hbase-it/pom.xml * (edit) hbase-prefix-tree/pom.xml * (edit) hbase-common/pom.xml * (edit) hbase-shaded/hbase-shaded-server/pom.xml * (edit) hbase-annotations/pom.xml * (edit) hbase-hadoop-compat/pom.xml * (edit) hbase-hadoop2-compat/pom.xml * (edit) pom.xml * (edit) hbase-rest/pom.xml * (edit) hbase-shaded/hbase-shaded-check-invariants/pom.xml * (edit) hbase-testing-util/pom.xml * (edit) hbase-client/pom.xml * (edit) hbase-shaded/pom.xml * (edit) hbase-procedure/pom.xml * (edit) hbase-assembly/pom.xml * (edit) hbase-shaded/hbase-shaded-client/pom.xml > Release 1.2.10 > -- > > Key: HBASE-21686 > URL: https://issues.apache.org/jira/browse/HBASE-21686 > Project: HBase > Issue Type: Task > Components: community >Affects Versions: 1.2.10 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Fix For: 1.2.10 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21680) Port HBASE-20194 (Basic Replication WebUI - Master) and HBASE-20193 (Basic Replication Web UI - Regionserver) to branch-1
[
https://issues.apache.org/jira/browse/HBASE-21680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744654#comment-16744654
]
Hadoop QA commented on HBASE-21680:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
51s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
45s{color} | {color:green} branch-1 passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} branch-1 passed with JDK v1.7.0_201 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
27s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
52s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} branch-1 passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | {color:green} branch-1 passed with JDK v1.7.0_201 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} the patch passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
42s{color} | {color:green} the patch passed with JDK v1.7.0_201 {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 42s{color}
| {color:red} hbase-server-jdk1.7.0_201 with JDK v1.7.0_201 generated 2 new + 4
unchanged - 2 fixed = 6 total (was 6) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
28s{color} | {color:red} hbase-server: The patch generated 7 new + 322
unchanged - 0 fixed = 329 total (was 322) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
46s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
1m 38s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} the patch passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} the patch passed with JDK v1.7.0_201 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}119m 30s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}139m 22s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.balancer.TestStochasticLoadBalancer2
|
| | hadoop.hbase.replication.TestReplicationMetricsforUI |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:61288f8 |
| JIRA Issue | HBASE-21680 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12955166/HBASE-21680-
[jira] [Updated] (HBASE-21726) Add getAllRegionLocations method to AsyncTableRegionLocator
[ https://issues.apache.org/jira/browse/HBASE-21726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21726: -- Assignee: Duo Zhang Fix Version/s: 2.0.5 2.1.3 2.2.0 3.0.0 Status: Patch Available (was: Open) > Add getAllRegionLocations method to AsyncTableRegionLocator > --- > > Key: HBASE-21726 > URL: https://issues.apache.org/jira/browse/HBASE-21726 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5 > > Attachments: HBASE-21726.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19695) Handle disabled table for async client
[ https://issues.apache.org/jira/browse/HBASE-19695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744649#comment-16744649 ] Jingyun Tian commented on HBASE-19695: -- +1 for this. > Handle disabled table for async client > -- > > Key: HBASE-19695 > URL: https://issues.apache.org/jira/browse/HBASE-19695 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5 > > Attachments: HBASE-19695-v1.patch, HBASE-19695-v2.patch, > HBASE-19695.master.001.patch, HBASE-19695.master.002.patch, HBASE-19695.patch > > > Now for async client we will not check if a table is disabled when retrying, > so we will retry until the time or count limit is reached, and will not throw > a TableNotEnabledException. > We should have the same behavior as sync client, but the implementation > should be carefully designed. For sync client, we will also check if a table > is disabled if it is a retry, no matter what the exception is. This will > double the pressure on meta table. We should try our best to eliminate > unnecessary access to meta table. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21726) Add getAllRegionLocations method to AsyncTableRegionLocator
[ https://issues.apache.org/jira/browse/HBASE-21726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21726: -- Attachment: HBASE-21726.patch > Add getAllRegionLocations method to AsyncTableRegionLocator > --- > > Key: HBASE-21726 > URL: https://issues.apache.org/jira/browse/HBASE-21726 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Priority: Major > Attachments: HBASE-21726.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21680) Port HBASE-20194 (Basic Replication WebUI - Master) and HBASE-20193 (Basic Replication Web UI - Regionserver) to branch-1
[
https://issues.apache.org/jira/browse/HBASE-21680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744651#comment-16744651
]
Hadoop QA commented on HBASE-21680:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
42s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
37s{color} | {color:green} branch-1 passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} branch-1 passed with JDK v1.7.0_201 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
30s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
43s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} branch-1 passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} branch-1 passed with JDK v1.7.0_201 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} the patch passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} the patch passed with JDK v1.7.0_201 {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 38s{color}
| {color:red} hbase-server-jdk1.7.0_201 with JDK v1.7.0_201 generated 2 new + 4
unchanged - 2 fixed = 6 total (was 6) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
24s{color} | {color:red} hbase-server: The patch generated 7 new + 322
unchanged - 0 fixed = 329 total (was 322) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
46s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
1m 39s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} the patch passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} the patch passed with JDK v1.7.0_201 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}105m
19s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}124m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:61288f8 |
| JIRA Issue | HBASE-21680 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12955167/HBASE-21680-branch-1.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Lin
[jira] [Commented] (HBASE-21564) race condition in WAL rolling resulting in size-based rolling getting stuck
[ https://issues.apache.org/jira/browse/HBASE-21564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744599#comment-16744599 ] Duo Zhang commented on HBASE-21564: --- I've commented on RB. The problem here is that, the implementation of the FSWAL also highly depends on the implementation here, especially how we roll a wal on error. We need to find a way to prevent rolling lots of wal files when error occurs... > race condition in WAL rolling resulting in size-based rolling getting stuck > --- > > Key: HBASE-21564 > URL: https://issues.apache.org/jira/browse/HBASE-21564 > Project: HBase > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HBASE-21564.master.001.patch, > HBASE-21564.master.002.patch, HBASE-21564.master.003.patch, > HBASE-21564.master.004.patch, HBASE-21564.master.005.patch > > > Manifests at least with AsyncFsWriter. > There's a window after LogRoller replaces the writer in the WAL, but before > it sets the rollLog boolean to false in the finally, where the WAL class can > request another log roll (it can happen in particular when the logs are > getting archived in the LogRoller thread, and there's high write volume > causing the logs to roll quickly). > LogRoller will blindly reset the rollLog flag in finally and "forget" about > this request. > AsyncWAL in turn never requests it again because its own rollRequested field > is set and it expects a callback. Logs don't get rolled until a periodic roll > is triggered after that. > The acknowledgment of roll requests by LogRoller should be atomic. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21728) Backport to branch-2 (and maybe branch-1) HBASE-21684 Throw DNRIOE when connection or rpc client is closed
[ https://issues.apache.org/jira/browse/HBASE-21728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744593#comment-16744593 ] Duo Zhang commented on HBASE-21728: --- OK, seems fine. +1 on committing to branch-2. > Backport to branch-2 (and maybe branch-1) HBASE-21684 Throw DNRIOE when > connection or rpc client is closed > -- > > Key: HBASE-21728 > URL: https://issues.apache.org/jira/browse/HBASE-21728 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Priority: Major > Fix For: 2.2.0, 2.1.3, 2.0.5 > > Attachments: HBASE-21728-branch-1.001.patch > > > Backport the parent. May need a few changes. See suggestions in parent. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21564) race condition in WAL rolling resulting in size-based rolling getting stuck
[ https://issues.apache.org/jira/browse/HBASE-21564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744595#comment-16744595 ] Duo Zhang commented on HBASE-21564: --- Sorry, let me take a quick look. > race condition in WAL rolling resulting in size-based rolling getting stuck > --- > > Key: HBASE-21564 > URL: https://issues.apache.org/jira/browse/HBASE-21564 > Project: HBase > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HBASE-21564.master.001.patch, > HBASE-21564.master.002.patch, HBASE-21564.master.003.patch, > HBASE-21564.master.004.patch, HBASE-21564.master.005.patch > > > Manifests at least with AsyncFsWriter. > There's a window after LogRoller replaces the writer in the WAL, but before > it sets the rollLog boolean to false in the finally, where the WAL class can > request another log roll (it can happen in particular when the logs are > getting archived in the LogRoller thread, and there's high write volume > causing the logs to roll quickly). > LogRoller will blindly reset the rollLog flag in finally and "forget" about > this request. > AsyncWAL in turn never requests it again because its own rollRequested field > is set and it expects a callback. Logs don't get rolled until a periodic roll > is triggered after that. > The acknowledgment of roll requests by LogRoller should be atomic. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21728) Backport to branch-2 (and maybe branch-1) HBASE-21684 Throw DNRIOE when connection or rpc client is closed
[ https://issues.apache.org/jira/browse/HBASE-21728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16743527#comment-16743527 ] Xu Cang edited comment on HBASE-21728 at 1/17/19 1:38 AM: -- the original patch can be applied to branch-2 without any change. Tested locally. for branch-1, since there is no ReadOnlyZKClient class, only need to change the StoppedRpcClientException class to throw DNRIOE. was (Author: xucang): the original patch can be applied to branch-1 without any change. Tested locally. for branch-1, since there is no ReadOnlyZKClient class, only need to change the StoppedRpcClientException class to throw DNRIOE. > Backport to branch-2 (and maybe branch-1) HBASE-21684 Throw DNRIOE when > connection or rpc client is closed > -- > > Key: HBASE-21728 > URL: https://issues.apache.org/jira/browse/HBASE-21728 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Priority: Major > Fix For: 2.2.0, 2.1.3, 2.0.5 > > Attachments: HBASE-21728-branch-1.001.patch > > > Backport the parent. May need a few changes. See suggestions in parent. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21685) Change repository urls to Gitbox
[ https://issues.apache.org/jira/browse/HBASE-21685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744589#comment-16744589 ] Duo Zhang commented on HBASE-21685: --- I mean we could just copy their workflow then? As hadoop and hbase had almost the same work flow before. Do they still make use of yetus? In the mailing list I saw that yetus is about to release a new version which will support the workflow for github PR? > Change repository urls to Gitbox > > > Key: HBASE-21685 > URL: https://issues.apache.org/jira/browse/HBASE-21685 > Project: HBase > Issue Type: Task >Reporter: Peter Somogyi >Assignee: Peter Somogyi >Priority: Critical > Fix For: 3.0.0, 1.5.0, 2.2.0, 1.4.10, 2.1.3, 2.0.5, 1.3.4, 1.2.11 > > Attachments: HBASE-21685.master.001.patch, > HBASE-21685.master.001.patch > > > Moving to Gitbox is approaching and references to git-wip-us need to be > changed to gitbox. > Some of the Jenkins jobs are referring to git-wip-us which if going to be > locked after the migration. We could move them to github so the build flow > will remain intact. > Previous discussion on dev@: > [https://lists.apache.org/thread.html/3496568d6cc002f74f5c3bcce46ed44b7ee9e90d7d53af2c65b6f785@%3Cdev.hbase.apache.org%3E] > After this notify INFRA to make the change -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21680) Port HBASE-20194 (Basic Replication WebUI - Master) and HBASE-20193 (Basic Replication Web UI - Regionserver) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21680: --- Attachment: HBASE-21680-branch-1.patch > Port HBASE-20194 (Basic Replication WebUI - Master) and HBASE-20193 (Basic > Replication Web UI - Regionserver) to branch-1 > - > > Key: HBASE-21680 > URL: https://issues.apache.org/jira/browse/HBASE-21680 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-21680-branch-1.patch, HBASE-21680-branch-1.patch, > Screen Shot 2019-01-16 at 3.20.00 PM.png, Screen Shot 2019-01-16 at 3.20.50 > PM.png, Screen Shot 2019-01-16 at 3.21.17 PM.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21680) Port HBASE-20194 (Basic Replication WebUI - Master) and HBASE-20193 (Basic Replication Web UI - Regionserver) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744581#comment-16744581 ] Andrew Purtell commented on HBASE-21680: Rebase patch > Port HBASE-20194 (Basic Replication WebUI - Master) and HBASE-20193 (Basic > Replication Web UI - Regionserver) to branch-1 > - > > Key: HBASE-21680 > URL: https://issues.apache.org/jira/browse/HBASE-21680 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-21680-branch-1.patch, HBASE-21680-branch-1.patch, > Screen Shot 2019-01-16 at 3.20.00 PM.png, Screen Shot 2019-01-16 at 3.20.50 > PM.png, Screen Shot 2019-01-16 at 3.21.17 PM.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21680) Port HBASE-20194 (Basic Replication WebUI - Master) and HBASE-20193 (Basic Replication Web UI - Regionserver) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21680: --- Attachment: (was: HBASE-21680-branch-1.patch) > Port HBASE-20194 (Basic Replication WebUI - Master) and HBASE-20193 (Basic > Replication Web UI - Regionserver) to branch-1 > - > > Key: HBASE-21680 > URL: https://issues.apache.org/jira/browse/HBASE-21680 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-21680-branch-1.patch, HBASE-21680-branch-1.patch, > Screen Shot 2019-01-16 at 3.20.00 PM.png, Screen Shot 2019-01-16 at 3.20.50 > PM.png, Screen Shot 2019-01-16 at 3.21.17 PM.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21616) Port HBASE-21034 (Add new throttle type: read/write capacity unit) to branch-1
[
https://issues.apache.org/jira/browse/HBASE-21616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744577#comment-16744577
]
Hadoop QA commented on HBASE-21616:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 21m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 5m
41s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
4s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
25s{color} | {color:green} branch-1 passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
34s{color} | {color:green} branch-1 passed with JDK v1.7.0_201 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
21s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
29s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} branch-1 passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
9s{color} | {color:green} branch-1 passed with JDK v1.7.0_201 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
17s{color} | {color:green} the patch passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
28s{color} | {color:green} the patch passed with JDK v1.7.0_201 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
32s{color} | {color:red} hbase-client: The patch generated 13 new + 267
unchanged - 12 fixed = 280 total (was 279) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
15s{color} | {color:red} hbase-server: The patch generated 21 new + 74
unchanged - 11 fixed = 95 total (was 85) {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m
6s{color} | {color:red} The patch generated 5 new + 142 unchanged - 6 fixed =
147 total (was 148) {color} |
| {color:green}+1{color} | {color:green} ruby-lint {color} | {color:green} 0m
2s{color} | {color:green} There were no new ruby-lint issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
30s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
1m 32s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
1m 45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color
[jira] [Updated] (HBASE-21680) Port HBASE-20194 (Basic Replication WebUI - Master) and HBASE-20193 (Basic Replication Web UI - Regionserver) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21680: --- Attachment: HBASE-21680-branch-1.patch > Port HBASE-20194 (Basic Replication WebUI - Master) and HBASE-20193 (Basic > Replication Web UI - Regionserver) to branch-1 > - > > Key: HBASE-21680 > URL: https://issues.apache.org/jira/browse/HBASE-21680 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-21680-branch-1.patch, HBASE-21680-branch-1.patch, > Screen Shot 2019-01-16 at 3.20.00 PM.png, Screen Shot 2019-01-16 at 3.20.50 > PM.png, Screen Shot 2019-01-16 at 3.21.17 PM.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21686) Release 1.2.10
[ https://issues.apache.org/jira/browse/HBASE-21686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744567#comment-16744567 ] Sean Busbey commented on HBASE-21686: - * update release date for 1.2.10 in CHANGES, set version on branch-1.2 to 1.2.11-SNAPSHOT > Release 1.2.10 > -- > > Key: HBASE-21686 > URL: https://issues.apache.org/jira/browse/HBASE-21686 > Project: HBase > Issue Type: Task > Components: community >Affects Versions: 1.2.10 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Fix For: 1.2.10 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21720) metric to measure how actions are distributed to servers within a MultiAction
[
https://issues.apache.org/jira/browse/HBASE-21720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744563#comment-16744563
]
Hadoop QA commented on HBASE-21720:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
10s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
27s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
29s{color} | {color:red} hbase-client: The patch generated 2 new + 47 unchanged
- 0 fixed = 49 total (was 47) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
4s{color} | {color:red} hbase-server: The patch generated 1 new + 0 unchanged -
0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
7s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 44s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
11s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}128m
32s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
48s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}175m 52s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21720 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12955146/HBASE-21720.master.003.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux b42db4fa9c03 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenk
[jira] [Commented] (HBASE-21686) Release 1.2.10
[ https://issues.apache.org/jira/browse/HBASE-21686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744553#comment-16744553 ] Sean Busbey commented on HBASE-21686: - * made short links to vote and results * svn moved the RC to the release area * released and dropped the staging repo in repository.a.o * added 1.2.10 to reporter.a.o (had to change my local TZ to GMT to do that :/) * removed 1.2.9 from dist.a.o * closed out version 1.2.10 in jira * set start date for 1.2.11 in jira to RC creation point > Release 1.2.10 > -- > > Key: HBASE-21686 > URL: https://issues.apache.org/jira/browse/HBASE-21686 > Project: HBase > Issue Type: Task > Components: community >Affects Versions: 1.2.10 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Fix For: 1.2.10 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21686) Release 1.2.10
[ https://issues.apache.org/jira/browse/HBASE-21686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744552#comment-16744552 ] Sean Busbey commented on HBASE-21686: - RC0 Passed: vote: https://s.apache.org/hbase-1.2.10-rc0-vote results: https://s.apache.org/hbase-1.2.10-vote-results > Release 1.2.10 > -- > > Key: HBASE-21686 > URL: https://issues.apache.org/jira/browse/HBASE-21686 > Project: HBase > Issue Type: Task > Components: community >Affects Versions: 1.2.10 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Fix For: 1.2.10 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21728) Backport to branch-2 (and maybe branch-1) HBASE-21684 Throw DNRIOE when connection or rpc client is closed
[ https://issues.apache.org/jira/browse/HBASE-21728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744546#comment-16744546 ] stack commented on HBASE-21728: --- Thanks [~xucang] [~Apache9] suggests: bq. The exception is marked as IA.Public so theoretically it should only be pushed to master? ... which is strictly true only IMO this a bug fix where we change an intermediary Interface so client does the right thing. I think we should mark this incompatible change and put it in branch-2 at least but would be in favor of pulling it back to branch-2... > Backport to branch-2 (and maybe branch-1) HBASE-21684 Throw DNRIOE when > connection or rpc client is closed > -- > > Key: HBASE-21728 > URL: https://issues.apache.org/jira/browse/HBASE-21728 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Priority: Major > Fix For: 2.2.0, 2.1.3, 2.0.5 > > Attachments: HBASE-21728-branch-1.001.patch > > > Backport the parent. May need a few changes. See suggestions in parent. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21616) Port HBASE-21034 (Add new throttle type: read/write capacity unit) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744540#comment-16744540 ] Andrew Purtell commented on HBASE-21616: Attached what I committed, including checkstyle warning fixes > Port HBASE-21034 (Add new throttle type: read/write capacity unit) to branch-1 > -- > > Key: HBASE-21616 > URL: https://issues.apache.org/jira/browse/HBASE-21616 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-21616-branch-1.patch, HBASE-21616-branch-1.patch, > HBASE-21616-branch-1.patch > > > Port HBASE-21034 (Add new throttle type: read/write capacity unit) to branch-1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21616) Port HBASE-21034 (Add new throttle type: read/write capacity unit) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21616: --- Attachment: HBASE-21616-branch-1.patch > Port HBASE-21034 (Add new throttle type: read/write capacity unit) to branch-1 > -- > > Key: HBASE-21616 > URL: https://issues.apache.org/jira/browse/HBASE-21616 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-21616-branch-1.patch, HBASE-21616-branch-1.patch, > HBASE-21616-branch-1.patch > > > Port HBASE-21034 (Add new throttle type: read/write capacity unit) to branch-1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21616) Port HBASE-21034 (Add new throttle type: read/write capacity unit) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21616: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) > Port HBASE-21034 (Add new throttle type: read/write capacity unit) to branch-1 > -- > > Key: HBASE-21616 > URL: https://issues.apache.org/jira/browse/HBASE-21616 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-21616-branch-1.patch, HBASE-21616-branch-1.patch, > HBASE-21616-branch-1.patch > > > Port HBASE-21034 (Add new throttle type: read/write capacity unit) to branch-1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21680) Port HBASE-20194 (Basic Replication WebUI - Master) and HBASE-20193 (Basic Replication Web UI - Regionserver) to branch-1
[
https://issues.apache.org/jira/browse/HBASE-21680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744535#comment-16744535
]
Hadoop QA commented on HBASE-21680:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s{color}
| {color:red} HBASE-21680 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.8.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-21680 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/15607/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Port HBASE-20194 (Basic Replication WebUI - Master) and HBASE-20193 (Basic
> Replication Web UI - Regionserver) to branch-1
> -
>
> Key: HBASE-21680
> URL: https://issues.apache.org/jira/browse/HBASE-21680
> Project: HBase
> Issue Type: Sub-task
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Major
> Fix For: 1.5.0
>
> Attachments: HBASE-21680-branch-1.patch, Screen Shot 2019-01-16 at
> 3.20.00 PM.png, Screen Shot 2019-01-16 at 3.20.50 PM.png, Screen Shot
> 2019-01-16 at 3.21.17 PM.png
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21616) Port HBASE-21034 (Add new throttle type: read/write capacity unit) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744531#comment-16744531 ] Andrew Purtell commented on HBASE-21616: Thanks! Will fix checkstyle nits for commit > Port HBASE-21034 (Add new throttle type: read/write capacity unit) to branch-1 > -- > > Key: HBASE-21616 > URL: https://issues.apache.org/jira/browse/HBASE-21616 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-21616-branch-1.patch, HBASE-21616-branch-1.patch > > > Port HBASE-21034 (Add new throttle type: read/write capacity unit) to branch-1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21680) Port HBASE-20194 (Basic Replication WebUI - Master) and HBASE-20193 (Basic Replication Web UI - Regionserver) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21680: --- Status: Patch Available (was: Open) > Port HBASE-20194 (Basic Replication WebUI - Master) and HBASE-20193 (Basic > Replication Web UI - Regionserver) to branch-1 > - > > Key: HBASE-21680 > URL: https://issues.apache.org/jira/browse/HBASE-21680 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-21680-branch-1.patch, Screen Shot 2019-01-16 at > 3.20.00 PM.png, Screen Shot 2019-01-16 at 3.20.50 PM.png, Screen Shot > 2019-01-16 at 3.21.17 PM.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21680) Port HBASE-20194 (Basic Replication WebUI - Master) and HBASE-20193 (Basic Replication Web UI - Regionserver) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21680: --- Attachment: Screen Shot 2019-01-16 at 3.21.17 PM.png Screen Shot 2019-01-16 at 3.20.50 PM.png Screen Shot 2019-01-16 at 3.20.00 PM.png > Port HBASE-20194 (Basic Replication WebUI - Master) and HBASE-20193 (Basic > Replication Web UI - Regionserver) to branch-1 > - > > Key: HBASE-21680 > URL: https://issues.apache.org/jira/browse/HBASE-21680 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-21680-branch-1.patch, Screen Shot 2019-01-16 at > 3.20.00 PM.png, Screen Shot 2019-01-16 at 3.20.50 PM.png, Screen Shot > 2019-01-16 at 3.21.17 PM.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21680) Port HBASE-20194 (Basic Replication WebUI - Master) and HBASE-20193 (Basic Replication Web UI - Regionserver) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744529#comment-16744529 ] Andrew Purtell commented on HBASE-21680: Attached backport patch. New unit tests pass. Existing replication unit tests pass. Manual verification of UI changes looks good, screenshots attached. > Port HBASE-20194 (Basic Replication WebUI - Master) and HBASE-20193 (Basic > Replication Web UI - Regionserver) to branch-1 > - > > Key: HBASE-21680 > URL: https://issues.apache.org/jira/browse/HBASE-21680 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-21680-branch-1.patch, Screen Shot 2019-01-16 at > 3.20.00 PM.png, Screen Shot 2019-01-16 at 3.20.50 PM.png, Screen Shot > 2019-01-16 at 3.21.17 PM.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21680) Port HBASE-20194 (Basic Replication WebUI - Master) and HBASE-20193 (Basic Replication Web UI - Regionserver) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21680: --- Attachment: HBASE-21680-branch-1.patch > Port HBASE-20194 (Basic Replication WebUI - Master) and HBASE-20193 (Basic > Replication Web UI - Regionserver) to branch-1 > - > > Key: HBASE-21680 > URL: https://issues.apache.org/jira/browse/HBASE-21680 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-21680-branch-1.patch, Screen Shot 2019-01-16 at > 3.20.00 PM.png, Screen Shot 2019-01-16 at 3.20.50 PM.png, Screen Shot > 2019-01-16 at 3.21.17 PM.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21616) Port HBASE-21034 (Add new throttle type: read/write capacity unit) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744502#comment-16744502 ] Abhishek Singh Chouhan commented on HBASE-21616: LGTM +1. Checkstyle warnings of the previous QA look related. Might be good to have a look. > Port HBASE-21034 (Add new throttle type: read/write capacity unit) to branch-1 > -- > > Key: HBASE-21616 > URL: https://issues.apache.org/jira/browse/HBASE-21616 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-21616-branch-1.patch, HBASE-21616-branch-1.patch > > > Port HBASE-21034 (Add new throttle type: read/write capacity unit) to branch-1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20209) Do Not Use Both Map containsKey and get Methods in Replication Sink
[
https://issues.apache.org/jira/browse/HBASE-20209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-20209:
Resolution: Fixed
Fix Version/s: 2.2.0
1.5.0
3.0.0
Status: Resolved (was: Patch Available)
pushed to next minors. thanks for the patch [~belugabehr]!
> Do Not Use Both Map containsKey and get Methods in Replication Sink
> ---
>
> Key: HBASE-20209
> URL: https://issues.apache.org/jira/browse/HBASE-20209
> Project: HBase
> Issue Type: Improvement
> Components: Replication
>Affects Versions: 2.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Trivial
> Fix For: 3.0.0, 1.5.0, 2.2.0
>
> Attachments: HBASE-20209.1.patch, HBASE-20209.2.patch,
> HBASE-20209.3.patch, HBASE-20209.4.patch
>
>
> {code:title=ReplicationSink.java}
> String tableName = table.getNameWithNamespaceInclAsString();
> if (bulkLoadHFileMap.containsKey(tableName)) {
> List>> familyHFilePathsList =
> bulkLoadHFileMap.get(tableName);
> boolean foundFamily = false;
> for (int i = 0; i < familyHFilePathsList.size(); i++) {
> Pair> familyHFilePathsPair =
> familyHFilePathsList.get(i);
> if (Bytes.equals(familyHFilePathsPair.getFirst(), family)) {
> // Found family already present, just add the path to the
> existing list
> familyHFilePathsPair.getSecond().add(pathToHfileFromNS);
> foundFamily = true;
> break;
> }
> }
> {code}
> I propose that this code does not use the Map methods _containsKey_ *and*
> _get_. Simply use the _get_ method once and check a _null_ return value to
> check for existence. Saves a trip to the Map data structure for each call.
> Also, use enhanced for loop.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21402) Backport parent "HBASE-21325 Force to terminate regionserver when abort hang in somewhere"
[
https://issues.apache.org/jira/browse/HBASE-21402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744499#comment-16744499
]
Hadoop QA commented on HBASE-21402:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-2.0 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
35s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
4s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
18s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
42s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
7s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} branch-2.0 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
57s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 16s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}154m
15s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
53s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}200m 34s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:6f01af0 |
| JIRA Issue | HBASE-21402 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12955135/HBASE-21402.branch-2.0.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux f8030080edaf 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct
31 10:55:11 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | branch-2.0 / a37ca83d27 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/15604/testReport/ |
| Max. process+thread count | 4051 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/15604/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This mes
[jira] [Commented] (HBASE-21616) Port HBASE-21034 (Add new throttle type: read/write capacity unit) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744478#comment-16744478 ] Andrew Purtell commented on HBASE-21616: Update patch for cut-and-paste issue found during review > Port HBASE-21034 (Add new throttle type: read/write capacity unit) to branch-1 > -- > > Key: HBASE-21616 > URL: https://issues.apache.org/jira/browse/HBASE-21616 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-21616-branch-1.patch, HBASE-21616-branch-1.patch > > > Port HBASE-21034 (Add new throttle type: read/write capacity unit) to branch-1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21616) Port HBASE-21034 (Add new throttle type: read/write capacity unit) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21616: --- Attachment: HBASE-21616-branch-1.patch > Port HBASE-21034 (Add new throttle type: read/write capacity unit) to branch-1 > -- > > Key: HBASE-21616 > URL: https://issues.apache.org/jira/browse/HBASE-21616 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-21616-branch-1.patch, HBASE-21616-branch-1.patch > > > Port HBASE-21034 (Add new throttle type: read/write capacity unit) to branch-1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21720) metric to measure how actions are distributed to servers within a MultiAction
[ https://issues.apache.org/jira/browse/HBASE-21720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tommy Li updated HBASE-21720: - Attachment: HBASE-21720.master.003.patch > metric to measure how actions are distributed to servers within a MultiAction > - > > Key: HBASE-21720 > URL: https://issues.apache.org/jira/browse/HBASE-21720 > Project: HBase > Issue Type: Improvement > Components: Client, metrics, monitoring >Reporter: Tommy Li >Assignee: Tommy Li >Priority: Minor > Fix For: 3.0.0 > > Attachments: HBASE-21720.master.001.patch, > HBASE-21720.master.002.patch, HBASE-21720.master.003.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21402) Backport parent "HBASE-21325 Force to terminate regionserver when abort hang in somewhere"
[
https://issues.apache.org/jira/browse/HBASE-21402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744465#comment-16744465
]
Hadoop QA commented on HBASE-21402:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-2.1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
39s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
56s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
16s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
9s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
5s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} branch-2.1 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
7s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 13s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}138m
23s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}177m 54s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:42ca976 |
| JIRA Issue | HBASE-21402 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12955130/HBASE-21402.branch-2.1.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 5dbf5f137216 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct
31 10:55:11 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | branch-2.1 / 0363f7890a |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/15603/testReport/ |
| Max. process+thread count | 4687 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/15603/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was
[jira] [Commented] (HBASE-21070) SnapshotFileCache won't update for snapshots stored in S3
[ https://issues.apache.org/jira/browse/HBASE-21070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1671#comment-1671 ] Zach York commented on HBASE-21070: --- I fixed the checkstyle issue locally. > SnapshotFileCache won't update for snapshots stored in S3 > - > > Key: HBASE-21070 > URL: https://issues.apache.org/jira/browse/HBASE-21070 > Project: HBase > Issue Type: Bug > Components: snapshots >Affects Versions: 3.0.0, 2.1.1, 1.4.7 >Reporter: Zach York >Assignee: Zach York >Priority: Critical > Labels: FSRedo > Fix For: 3.0.0, 1.5.0, 2.2.0 > > Attachments: HBASE-21070.master.001.patch, > HBASE-21070.master.002.patch, HBASE-21070.master.003.patch > > > The SnapshotFileCache depends on last modified time to determine whether to > update the Snapshot HFile cache. However, in S3, real 'folders' don't exist. > S3 filesystems create a dummy file in place of a folder, but the dummy file > last modified time is not updated when files are changed 'under' it. This > means that the SnapshotFileCache doesn't pick up new snapshot HFiles and > these files aren't removed from the HFileCleaner and can be eligible for > deletion. > > My patch removes the lastmodified assumption. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21070) SnapshotFileCache won't update for snapshots stored in S3
[
https://issues.apache.org/jira/browse/HBASE-21070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744417#comment-16744417
]
Hadoop QA commented on HBASE-21070:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
19s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
25s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
23s{color} | {color:red} hbase-server: The patch generated 1 new + 1 unchanged
- 1 fixed = 2 total (was 2) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
12s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
11m 2s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}128m
19s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}175m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21070 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12955123/HBASE-21070.master.003.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 4dcefddaef48 4.4.0-138-generic #164~14.04.1-Ubuntu SMP Fri Oct
5 08:56:16 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / 594341d6fe |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/15602/artifact/patchprocess/diff-checkstyle-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/15602/testReport/ |
| Max. process+thread count | 5153 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Conso
[jira] [Commented] (HBASE-21731) Do not need to use ClusterConnection in IntegrationTestBigLinkedListWithVisibility
[
https://issues.apache.org/jira/browse/HBASE-21731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744413#comment-16744413
]
Hudson commented on HBASE-21731:
Results for branch master
[build #724 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/724/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/724//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/724//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/724//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Do not need to use ClusterConnection in
> IntegrationTestBigLinkedListWithVisibility
> --
>
> Key: HBASE-21731
> URL: https://issues.apache.org/jira/browse/HBASE-21731
> Project: HBase
> Issue Type: Task
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5
>
> Attachments: HBASE-21731-v1.patch, HBASE-21731.patch
>
>
> We could just use the RegionLocator instead of
> ClusterConnection.relocateRegion
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21626) log the regions blocking WAL from being archived
[ https://issues.apache.org/jira/browse/HBASE-21626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744400#comment-16744400 ] stack commented on HBASE-21626: --- Go for it. Looks like you are trying it [~sershe] That'll figure whether addendum is good or not better than my eyeballing. +1. > log the regions blocking WAL from being archived > > > Key: HBASE-21626 > URL: https://issues.apache.org/jira/browse/HBASE-21626 > Project: HBase > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-21626.01.patch, HBASE-21626.02.patch, > HBASE-21626.ADDENDUM.patch, HBASE-21626.patch > > > The WALs not being archived for a long time can result in a long recovery > later. It's useful to know what regions are blocking the WALs from being > archived, to be able to debug flush logic and tune configuration. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21711) Remove references to git.apache.org/hbase.git
[
https://issues.apache.org/jira/browse/HBASE-21711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744410#comment-16744410
]
Hudson commented on HBASE-21711:
Results for branch master
[build #724 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/724/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/724//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/724//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/724//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Remove references to git.apache.org/hbase.git
> -
>
> Key: HBASE-21711
> URL: https://issues.apache.org/jira/browse/HBASE-21711
> Project: HBase
> Issue Type: Sub-task
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Critical
> Fix For: 3.0.0, 1.5.0, 2.2.0, 1.4.10, 2.1.3, 2.0.5, 1.3.4, 1.2.11
>
> Attachments: HBASE-21711.branch-1.001.patch,
> HBASE-21711.master.001.patch
>
>
> With the GitBox migration not only git-wip-us was removed but also
> git.apache.org/hbase.git is not available anymore. (INFRA-17640)
> We need to remove all references to this url.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21727) Simplify documentation around client timeout
[
https://issues.apache.org/jira/browse/HBASE-21727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744411#comment-16744411
]
Hudson commented on HBASE-21727:
Results for branch master
[build #724 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/724/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/724//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/724//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/724//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Simplify documentation around client timeout
>
>
> Key: HBASE-21727
> URL: https://issues.apache.org/jira/browse/HBASE-21727
> Project: HBase
> Issue Type: Sub-task
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Minor
> Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5
>
> Attachments: HBASE-21727.master.001.patch
>
>
> Client rpc timeouts are not easy to understand from the documentation.
> [~stack] also had an idea to point to doc when exception is thrown.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21732) Should call toUpperCase before using Enum.valueOf in some methods for ColumnFamilyDescriptor
[
https://issues.apache.org/jira/browse/HBASE-21732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744412#comment-16744412
]
Hudson commented on HBASE-21732:
Results for branch master
[build #724 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/724/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/724//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/724//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/724//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Should call toUpperCase before using Enum.valueOf in some methods for
> ColumnFamilyDescriptor
>
>
> Key: HBASE-21732
> URL: https://issues.apache.org/jira/browse/HBASE-21732
> Project: HBase
> Issue Type: Bug
> Components: Client
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5
>
> Attachments: HBASE-21732.patch
>
>
> When upgrading we faced a problem that the some regions can not be opened due
> to the region server can not recognize the lower case 'snappy' config. In
> code for branch-1, we have done this
> {code}
> public Compression.Algorithm getCompression() {
> String n = getValue(COMPRESSION);
> if (n == null) {
> return Compression.Algorithm.NONE;
> }
> return Compression.Algorithm.valueOf(n.toUpperCase(Locale.ROOT));
> }
> {code}
> But in the code of 2.0+, we just call valueOf.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21710) Add quota related methods to the Admin interface
[
https://issues.apache.org/jira/browse/HBASE-21710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744414#comment-16744414
]
Hudson commented on HBASE-21710:
Results for branch master
[build #724 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/724/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/724//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/724//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/724//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add quota related methods to the Admin interface
>
>
> Key: HBASE-21710
> URL: https://issues.apache.org/jira/browse/HBASE-21710
> Project: HBase
> Issue Type: Task
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21710-v1.patch, HBASE-21710-v1.patch,
> HBASE-21710-v2.patch, HBASE-21710-v3.patch, HBASE-21710.patch
>
>
> So we can limit the ClusterConnection usage inside the sync client
> implementation. Now we will use ClusterConnection in QuotaTableUtil, which
> will be a pain when we want to replace the implement of the old sync client
> with the new async client.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20854) Wrong retries number in RpcRetryingCaller's log message
[
https://issues.apache.org/jira/browse/HBASE-20854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744389#comment-16744389
]
Hudson commented on HBASE-20854:
Results for branch branch-2
[build #1615 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1615/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1615//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1615//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1615//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Wrong retries number in RpcRetryingCaller's log message
> ---
>
> Key: HBASE-20854
> URL: https://issues.apache.org/jira/browse/HBASE-20854
> Project: HBase
> Issue Type: Bug
> Components: Client, logging
>Affects Versions: 2.0.0
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Minor
> Fix For: 3.0.0, 2.1.0, 2.2.0, 2.0.2
>
> Attachments: HBASE-20854.branch-2.0.patch
>
>
> Just a small bug fix. In the error log message of RpcRetryingCallerImpl.
> Tries number is passed to both tries and retries. Causing a bit of confusing.
> {code:java}
> 2018-07-05 21:04:46,343 INFO [Thread-20]
> org.apache.hadoop.hbase.client.RpcRetryingCallerImpl: Call exception,
> tries=6, retries=6, started=4174 ms ago, cancelled=false,
> msg=org.apache.hadoop.hbase.exce
> ptions.RegionOpeningException: Region
> IntegrationTestBigLinkedList,\x7F\xFF\xFF\xFF\xFF\xFF\xFF\xFE,1530795739116.0cfd339596648348ac13d979150eb2bf.
> is opening on e010125049164.bja,60020,1530795698451
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21626) log the regions blocking WAL from being archived
[ https://issues.apache.org/jira/browse/HBASE-21626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-21626: - Attachment: HBASE-21626.ADDENDUM.patch > log the regions blocking WAL from being archived > > > Key: HBASE-21626 > URL: https://issues.apache.org/jira/browse/HBASE-21626 > Project: HBase > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-21626.01.patch, HBASE-21626.02.patch, > HBASE-21626.ADDENDUM.patch, HBASE-21626.patch > > > The WALs not being archived for a long time can result in a long recovery > later. It's useful to know what regions are blocking the WALs from being > archived, to be able to debug flush logic and tune configuration. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20209) Do Not Use Both Map containsKey and get Methods in Replication Sink
[
https://issues.apache.org/jira/browse/HBASE-20209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744399#comment-16744399
]
stack commented on HBASE-20209:
---
Just saw [~busbey] comment. Ignore mine [~belugabehr]
> Do Not Use Both Map containsKey and get Methods in Replication Sink
> ---
>
> Key: HBASE-20209
> URL: https://issues.apache.org/jira/browse/HBASE-20209
> Project: HBase
> Issue Type: Improvement
> Components: Replication
>Affects Versions: 2.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Trivial
> Attachments: HBASE-20209.1.patch, HBASE-20209.2.patch,
> HBASE-20209.3.patch, HBASE-20209.4.patch
>
>
> {code:title=ReplicationSink.java}
> String tableName = table.getNameWithNamespaceInclAsString();
> if (bulkLoadHFileMap.containsKey(tableName)) {
> List>> familyHFilePathsList =
> bulkLoadHFileMap.get(tableName);
> boolean foundFamily = false;
> for (int i = 0; i < familyHFilePathsList.size(); i++) {
> Pair> familyHFilePathsPair =
> familyHFilePathsList.get(i);
> if (Bytes.equals(familyHFilePathsPair.getFirst(), family)) {
> // Found family already present, just add the path to the
> existing list
> familyHFilePathsPair.getSecond().add(pathToHfileFromNS);
> foundFamily = true;
> break;
> }
> }
> {code}
> I propose that this code does not use the Map methods _containsKey_ *and*
> _get_. Simply use the _get_ method once and check a _null_ return value to
> check for existence. Saves a trip to the Map data structure for each call.
> Also, use enhanced for loop.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21712) Make submit-patch.py python3 compatible
[
https://issues.apache.org/jira/browse/HBASE-21712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744387#comment-16744387
]
Hudson commented on HBASE-21712:
Results for branch branch-2
[build #1615 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1615/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1615//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1615//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1615//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Make submit-patch.py python3 compatible
> ---
>
> Key: HBASE-21712
> URL: https://issues.apache.org/jira/browse/HBASE-21712
> Project: HBase
> Issue Type: Improvement
> Components: tooling
>Reporter: Tommy Li
>Assignee: Tommy Li
>Priority: Minor
> Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5
>
> Attachments: HBASE-21712.master.001.patch,
> HBASE-21712.master.002.patch, HBASE-21722.master.001.patch.txt
>
>
> Attached patch was submitted with `python3 dev-support/submit-patch.py -b
> master -srb -jid HBASE-21712`
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21710) Add quota related methods to the Admin interface
[
https://issues.apache.org/jira/browse/HBASE-21710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744393#comment-16744393
]
Hudson commented on HBASE-21710:
Results for branch branch-2
[build #1615 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1615/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1615//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1615//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1615//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add quota related methods to the Admin interface
>
>
> Key: HBASE-21710
> URL: https://issues.apache.org/jira/browse/HBASE-21710
> Project: HBase
> Issue Type: Task
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21710-v1.patch, HBASE-21710-v1.patch,
> HBASE-21710-v2.patch, HBASE-21710-v3.patch, HBASE-21710.patch
>
>
> So we can limit the ClusterConnection usage inside the sync client
> implementation. Now we will use ClusterConnection in QuotaTableUtil, which
> will be a pain when we want to replace the implement of the old sync client
> with the new async client.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21731) Do not need to use ClusterConnection in IntegrationTestBigLinkedListWithVisibility
[
https://issues.apache.org/jira/browse/HBASE-21731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744392#comment-16744392
]
Hudson commented on HBASE-21731:
Results for branch branch-2
[build #1615 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1615/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1615//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1615//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1615//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Do not need to use ClusterConnection in
> IntegrationTestBigLinkedListWithVisibility
> --
>
> Key: HBASE-21731
> URL: https://issues.apache.org/jira/browse/HBASE-21731
> Project: HBase
> Issue Type: Task
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5
>
> Attachments: HBASE-21731-v1.patch, HBASE-21731.patch
>
>
> We could just use the RegionLocator instead of
> ClusterConnection.relocateRegion
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21612) Add developer debug options in HBase Config for REST server
[
https://issues.apache.org/jira/browse/HBASE-21612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744386#comment-16744386
]
Hudson commented on HBASE-21612:
Results for branch branch-2
[build #1615 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1615/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1615//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1615//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1615//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add developer debug options in HBase Config for REST server
>
>
> Key: HBASE-21612
> URL: https://issues.apache.org/jira/browse/HBASE-21612
> Project: HBase
> Issue Type: Wish
> Components: Operability, REST, scripts
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: HBASE-21612.patch
>
>
> Add developer debug options in HBase Config for REST server.
> Currently we have,
> {noformat}
> # export HBASE_MASTER_OPTS="$HBASE_MASTER_OPTS -Xdebug
> -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8070"
> # export HBASE_REGIONSERVER_OPTS="$HBASE_REGIONSERVER_OPTS -Xdebug
> -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8071"
> # export HBASE_THRIFT_OPTS="$HBASE_THRIFT_OPTS -Xdebug
> -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8072"
> # export HBASE_ZOOKEEPER_OPTS="$HBASE_ZOOKEEPER_OPTS -Xdebug
> -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8073"
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21732) Should call toUpperCase before using Enum.valueOf in some methods for ColumnFamilyDescriptor
[
https://issues.apache.org/jira/browse/HBASE-21732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744391#comment-16744391
]
Hudson commented on HBASE-21732:
Results for branch branch-2
[build #1615 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1615/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1615//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1615//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1615//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Should call toUpperCase before using Enum.valueOf in some methods for
> ColumnFamilyDescriptor
>
>
> Key: HBASE-21732
> URL: https://issues.apache.org/jira/browse/HBASE-21732
> Project: HBase
> Issue Type: Bug
> Components: Client
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5
>
> Attachments: HBASE-21732.patch
>
>
> When upgrading we faced a problem that the some regions can not be opened due
> to the region server can not recognize the lower case 'snappy' config. In
> code for branch-1, we have done this
> {code}
> public Compression.Algorithm getCompression() {
> String n = getValue(COMPRESSION);
> if (n == null) {
> return Compression.Algorithm.NONE;
> }
> return Compression.Algorithm.valueOf(n.toUpperCase(Locale.ROOT));
> }
> {code}
> But in the code of 2.0+, we just call valueOf.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21711) Remove references to git.apache.org/hbase.git
[
https://issues.apache.org/jira/browse/HBASE-21711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744388#comment-16744388
]
Hudson commented on HBASE-21711:
Results for branch branch-2
[build #1615 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1615/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1615//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1615//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1615//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Remove references to git.apache.org/hbase.git
> -
>
> Key: HBASE-21711
> URL: https://issues.apache.org/jira/browse/HBASE-21711
> Project: HBase
> Issue Type: Sub-task
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Critical
> Fix For: 3.0.0, 1.5.0, 2.2.0, 1.4.10, 2.1.3, 2.0.5, 1.3.4, 1.2.11
>
> Attachments: HBASE-21711.branch-1.001.patch,
> HBASE-21711.master.001.patch
>
>
> With the GitBox migration not only git-wip-us was removed but also
> git.apache.org/hbase.git is not available anymore. (INFRA-17640)
> We need to remove all references to this url.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21727) Simplify documentation around client timeout
[
https://issues.apache.org/jira/browse/HBASE-21727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744390#comment-16744390
]
Hudson commented on HBASE-21727:
Results for branch branch-2
[build #1615 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1615/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1615//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1615//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1615//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Simplify documentation around client timeout
>
>
> Key: HBASE-21727
> URL: https://issues.apache.org/jira/browse/HBASE-21727
> Project: HBase
> Issue Type: Sub-task
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Minor
> Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5
>
> Attachments: HBASE-21727.master.001.patch
>
>
> Client rpc timeouts are not easy to understand from the documentation.
> [~stack] also had an idea to point to doc when exception is thrown.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21626) log the regions blocking WAL from being archived
[ https://issues.apache.org/jira/browse/HBASE-21626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744382#comment-16744382 ] Sergey Shelukhin commented on HBASE-21626: -- [~stack] looks like the initial condition was reversed :( does addendum make sense to you? > log the regions blocking WAL from being archived > > > Key: HBASE-21626 > URL: https://issues.apache.org/jira/browse/HBASE-21626 > Project: HBase > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-21626.01.patch, HBASE-21626.02.patch, > HBASE-21626.ADDENDUM.patch, HBASE-21626.patch > > > The WALs not being archived for a long time can result in a long recovery > later. It's useful to know what regions are blocking the WALs from being > archived, to be able to debug flush logic and tune configuration. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21731) Do not need to use ClusterConnection in IntegrationTestBigLinkedListWithVisibility
[
https://issues.apache.org/jira/browse/HBASE-21731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744381#comment-16744381
]
Hudson commented on HBASE-21731:
Results for branch branch-2.1
[build #777 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/777/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/777//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/777//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/777//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Do not need to use ClusterConnection in
> IntegrationTestBigLinkedListWithVisibility
> --
>
> Key: HBASE-21731
> URL: https://issues.apache.org/jira/browse/HBASE-21731
> Project: HBase
> Issue Type: Task
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5
>
> Attachments: HBASE-21731-v1.patch, HBASE-21731.patch
>
>
> We could just use the RegionLocator instead of
> ClusterConnection.relocateRegion
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Reopened] (HBASE-21626) log the regions blocking WAL from being archived
[ https://issues.apache.org/jira/browse/HBASE-21626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin reopened HBASE-21626: -- There's an issue in this patch that causes the message to never be logged. > log the regions blocking WAL from being archived > > > Key: HBASE-21626 > URL: https://issues.apache.org/jira/browse/HBASE-21626 > Project: HBase > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-21626.01.patch, HBASE-21626.02.patch, > HBASE-21626.patch > > > The WALs not being archived for a long time can result in a long recovery > later. It's useful to know what regions are blocking the WALs from being > archived, to be able to debug flush logic and tune configuration. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21732) Should call toUpperCase before using Enum.valueOf in some methods for ColumnFamilyDescriptor
[
https://issues.apache.org/jira/browse/HBASE-21732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744380#comment-16744380
]
Hudson commented on HBASE-21732:
Results for branch branch-2.1
[build #777 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/777/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/777//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/777//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/777//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Should call toUpperCase before using Enum.valueOf in some methods for
> ColumnFamilyDescriptor
>
>
> Key: HBASE-21732
> URL: https://issues.apache.org/jira/browse/HBASE-21732
> Project: HBase
> Issue Type: Bug
> Components: Client
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5
>
> Attachments: HBASE-21732.patch
>
>
> When upgrading we faced a problem that the some regions can not be opened due
> to the region server can not recognize the lower case 'snappy' config. In
> code for branch-1, we have done this
> {code}
> public Compression.Algorithm getCompression() {
> String n = getValue(COMPRESSION);
> if (n == null) {
> return Compression.Algorithm.NONE;
> }
> return Compression.Algorithm.valueOf(n.toUpperCase(Locale.ROOT));
> }
> {code}
> But in the code of 2.0+, we just call valueOf.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20209) Do Not Use Both Map containsKey and get Methods in Replication Sink
[
https://issues.apache.org/jira/browse/HBASE-20209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744375#comment-16744375
]
stack commented on HBASE-20209:
---
[~belugabehr] Mind making version of patch using the format-patch or
./dev-support/submit-patch.py? It makes the patch easier to apply sir w/ proper
accreditation. Thanks.
> Do Not Use Both Map containsKey and get Methods in Replication Sink
> ---
>
> Key: HBASE-20209
> URL: https://issues.apache.org/jira/browse/HBASE-20209
> Project: HBase
> Issue Type: Improvement
> Components: Replication
>Affects Versions: 2.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Trivial
> Attachments: HBASE-20209.1.patch, HBASE-20209.2.patch,
> HBASE-20209.3.patch, HBASE-20209.4.patch
>
>
> {code:title=ReplicationSink.java}
> String tableName = table.getNameWithNamespaceInclAsString();
> if (bulkLoadHFileMap.containsKey(tableName)) {
> List>> familyHFilePathsList =
> bulkLoadHFileMap.get(tableName);
> boolean foundFamily = false;
> for (int i = 0; i < familyHFilePathsList.size(); i++) {
> Pair> familyHFilePathsPair =
> familyHFilePathsList.get(i);
> if (Bytes.equals(familyHFilePathsPair.getFirst(), family)) {
> // Found family already present, just add the path to the
> existing list
> familyHFilePathsPair.getSecond().add(pathToHfileFromNS);
> foundFamily = true;
> break;
> }
> }
> {code}
> I propose that this code does not use the Map methods _containsKey_ *and*
> _get_. Simply use the _get_ method once and check a _null_ return value to
> check for existence. Saves a trip to the Map data structure for each call.
> Also, use enhanced for loop.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21564) race condition in WAL rolling resulting in size-based rolling getting stuck
[ https://issues.apache.org/jira/browse/HBASE-21564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744359#comment-16744359 ] Sergey Shelukhin commented on HBASE-21564: -- [~Apache9] ping? I've addressed your CR feedback > race condition in WAL rolling resulting in size-based rolling getting stuck > --- > > Key: HBASE-21564 > URL: https://issues.apache.org/jira/browse/HBASE-21564 > Project: HBase > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HBASE-21564.master.001.patch, > HBASE-21564.master.002.patch, HBASE-21564.master.003.patch, > HBASE-21564.master.004.patch, HBASE-21564.master.005.patch > > > Manifests at least with AsyncFsWriter. > There's a window after LogRoller replaces the writer in the WAL, but before > it sets the rollLog boolean to false in the finally, where the WAL class can > request another log roll (it can happen in particular when the logs are > getting archived in the LogRoller thread, and there's high write volume > causing the logs to roll quickly). > LogRoller will blindly reset the rollLog flag in finally and "forget" about > this request. > AsyncWAL in turn never requests it again because its own rollRequested field > is set and it expects a callback. Logs don't get rolled until a periodic roll > is triggered after that. > The acknowledgment of roll requests by LogRoller should be atomic. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21402) Backport parent "HBASE-21325 Force to terminate regionserver when abort hang in somewhere"
[ https://issues.apache.org/jira/browse/HBASE-21402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744324#comment-16744324 ] Pankaj Kumar edited comment on HBASE-21402 at 1/16/19 6:51 PM: --- Attached branch-2.0/2.1 patch. was (Author: pankaj2461): Attached the patch, this patch can be applied to branch-2.0 also. > Backport parent "HBASE-21325 Force to terminate regionserver when abort hang > in somewhere" > -- > > Key: HBASE-21402 > URL: https://issues.apache.org/jira/browse/HBASE-21402 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.3, 2.1.2 >Reporter: stack >Priority: Major > Fix For: 2.1.3, 2.0.5 > > Attachments: HBASE-21402.branch-2.0.patch, > HBASE-21402.branch-2.1.patch > > > Backport parent. Looks like good idea for fixing long-standing problem. I can > do the backport [~zghaobac]. Just FYI. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21402) Backport parent "HBASE-21325 Force to terminate regionserver when abort hang in somewhere"
[ https://issues.apache.org/jira/browse/HBASE-21402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pankaj Kumar updated HBASE-21402: - Attachment: HBASE-21402.branch-2.0.patch > Backport parent "HBASE-21325 Force to terminate regionserver when abort hang > in somewhere" > -- > > Key: HBASE-21402 > URL: https://issues.apache.org/jira/browse/HBASE-21402 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.3, 2.1.2 >Reporter: stack >Priority: Major > Fix For: 2.1.3 > > Attachments: HBASE-21402.branch-2.0.patch, > HBASE-21402.branch-2.1.patch > > > Backport parent. Looks like good idea for fixing long-standing problem. I can > do the backport [~zghaobac]. Just FYI. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21402) Backport parent "HBASE-21325 Force to terminate regionserver when abort hang in somewhere"
[ https://issues.apache.org/jira/browse/HBASE-21402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pankaj Kumar updated HBASE-21402: - Fix Version/s: 2.0.5 > Backport parent "HBASE-21325 Force to terminate regionserver when abort hang > in somewhere" > -- > > Key: HBASE-21402 > URL: https://issues.apache.org/jira/browse/HBASE-21402 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.3, 2.1.2 >Reporter: stack >Priority: Major > Fix For: 2.1.3, 2.0.5 > > Attachments: HBASE-21402.branch-2.0.patch, > HBASE-21402.branch-2.1.patch > > > Backport parent. Looks like good idea for fixing long-standing problem. I can > do the backport [~zghaobac]. Just FYI. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21731) Do not need to use ClusterConnection in IntegrationTestBigLinkedListWithVisibility
[
https://issues.apache.org/jira/browse/HBASE-21731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744334#comment-16744334
]
Hudson commented on HBASE-21731:
Results for branch branch-2.0
[build #1261 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1261/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1261//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1261//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1261//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Do not need to use ClusterConnection in
> IntegrationTestBigLinkedListWithVisibility
> --
>
> Key: HBASE-21731
> URL: https://issues.apache.org/jira/browse/HBASE-21731
> Project: HBase
> Issue Type: Task
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5
>
> Attachments: HBASE-21731-v1.patch, HBASE-21731.patch
>
>
> We could just use the RegionLocator instead of
> ClusterConnection.relocateRegion
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21732) Should call toUpperCase before using Enum.valueOf in some methods for ColumnFamilyDescriptor
[
https://issues.apache.org/jira/browse/HBASE-21732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744333#comment-16744333
]
Hudson commented on HBASE-21732:
Results for branch branch-2.0
[build #1261 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1261/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1261//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1261//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1261//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Should call toUpperCase before using Enum.valueOf in some methods for
> ColumnFamilyDescriptor
>
>
> Key: HBASE-21732
> URL: https://issues.apache.org/jira/browse/HBASE-21732
> Project: HBase
> Issue Type: Bug
> Components: Client
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5
>
> Attachments: HBASE-21732.patch
>
>
> When upgrading we faced a problem that the some regions can not be opened due
> to the region server can not recognize the lower case 'snappy' config. In
> code for branch-1, we have done this
> {code}
> public Compression.Algorithm getCompression() {
> String n = getValue(COMPRESSION);
> if (n == null) {
> return Compression.Algorithm.NONE;
> }
> return Compression.Algorithm.valueOf(n.toUpperCase(Locale.ROOT));
> }
> {code}
> But in the code of 2.0+, we just call valueOf.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21402) Backport parent "HBASE-21325 Force to terminate regionserver when abort hang in somewhere"
[ https://issues.apache.org/jira/browse/HBASE-21402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744324#comment-16744324 ] Pankaj Kumar commented on HBASE-21402: -- Attached the patch, this patch can be applied to branch-2.0 also. > Backport parent "HBASE-21325 Force to terminate regionserver when abort hang > in somewhere" > -- > > Key: HBASE-21402 > URL: https://issues.apache.org/jira/browse/HBASE-21402 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.3, 2.1.2 >Reporter: stack >Priority: Major > Fix For: 2.1.3 > > Attachments: HBASE-21402.branch-2.1.patch > > > Backport parent. Looks like good idea for fixing long-standing problem. I can > do the backport [~zghaobac]. Just FYI. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21070) SnapshotFileCache won't update for snapshots stored in S3
[ https://issues.apache.org/jira/browse/HBASE-21070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744323#comment-16744323 ] Mingliang Liu commented on HBASE-21070: --- I myself has been working on something else in public cloud. I refreshed my understanding and the patch makes sense to me. I'll review this week. Thanks [~zyork]! > SnapshotFileCache won't update for snapshots stored in S3 > - > > Key: HBASE-21070 > URL: https://issues.apache.org/jira/browse/HBASE-21070 > Project: HBase > Issue Type: Bug > Components: snapshots >Affects Versions: 3.0.0, 2.1.1, 1.4.7 >Reporter: Zach York >Assignee: Zach York >Priority: Critical > Labels: FSRedo > Fix For: 3.0.0, 1.5.0, 2.2.0 > > Attachments: HBASE-21070.master.001.patch, > HBASE-21070.master.002.patch, HBASE-21070.master.003.patch > > > The SnapshotFileCache depends on last modified time to determine whether to > update the Snapshot HFile cache. However, in S3, real 'folders' don't exist. > S3 filesystems create a dummy file in place of a folder, but the dummy file > last modified time is not updated when files are changed 'under' it. This > means that the SnapshotFileCache doesn't pick up new snapshot HFiles and > these files aren't removed from the HFileCleaner and can be eligible for > deletion. > > My patch removes the lastmodified assumption. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21675) Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a lot of time) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744311#comment-16744311 ] Andrew Purtell commented on HBASE-21675: Committing patch v4, which fixes CopyTable to use the temporary root directory specified in configuration. > Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a > lot of time) to branch-1 > > > Key: HBASE-21675 > URL: https://issues.apache.org/jira/browse/HBASE-21675 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Zheng Hu >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-21675-branch-1-WIP.patch, > HBASE-21675.branch-1.v2.patch, HBASE-21675.branch-1.v3.patch, > HBASE-21675.branch-1.v4.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21402) Backport parent "HBASE-21325 Force to terminate regionserver when abort hang in somewhere"
[ https://issues.apache.org/jira/browse/HBASE-21402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pankaj Kumar updated HBASE-21402: - Fix Version/s: 2.1.3 Status: Patch Available (was: Open) > Backport parent "HBASE-21325 Force to terminate regionserver when abort hang > in somewhere" > -- > > Key: HBASE-21402 > URL: https://issues.apache.org/jira/browse/HBASE-21402 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.1.2, 2.0.3 >Reporter: stack >Priority: Major > Fix For: 2.1.3 > > Attachments: HBASE-21402.branch-2.1.patch > > > Backport parent. Looks like good idea for fixing long-standing problem. I can > do the backport [~zghaobac]. Just FYI. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
