[jira] [Commented] (HBASE-22422) Retain an ByteBuff with refCnt=0 when getBlock from LRUCache
[ https://issues.apache.org/jira/browse/HBASE-22422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16848024#comment-16848024 ] Zheng Hu commented on HBASE-22422: -- Upload a picture to show the current YCSB result ( see https://issues.apache.org/jira/secure/attachment/12969731/HBASE-22422-qps-after-fix-the-zero-retain-bug.png). At least, the QPS wouldn't drop to hundreds. but the sawtooth curve still looks some strange, anyway, will continue the digging. > Retain an ByteBuff with refCnt=0 when getBlock from LRUCache > > > Key: HBASE-22422 > URL: https://issues.apache.org/jira/browse/HBASE-22422 > Project: HBase > Issue Type: Sub-task > Components: BlockCache >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Attachments: 0001-debug2.patch, 0001-debug2.patch, 0001-debug2.patch, > 0001-debug3.patch, 0001-debug4.patch, > HBASE-22422-qps-after-fix-the-zero-retain-bug.png, > HBASE-22422.HBASE-21879.v01.patch, LRUBlockCache-getBlock.png, debug.patch, > failed-to-check-positive-on-web-ui.png, image-2019-05-15-12-00-03-641.png > > > After runing YCSB scan/get benchmark in our XiaoMi cluster, we found the get > QPS dropped from 25000/s to hunderds per second in a cluster with five > nodes. > After enable the debug log at YCSB client side, I found the following > stacktrace , see > https://issues.apache.org/jira/secure/attachment/12968745/image-2019-05-15-12-00-03-641.png. > > After looking into the stractrace, I can ensure that the zero refCnt block is > an intermedia index block, see [2] http://hbase.apache.org/images/hfilev2.png > Need a patch to fix this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22422) Retain an ByteBuff with refCnt=0 when getBlock from LRUCache
[ https://issues.apache.org/jira/browse/HBASE-22422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-22422: - Attachment: HBASE-22422-qps-after-fix-the-zero-retain-bug.png > Retain an ByteBuff with refCnt=0 when getBlock from LRUCache > > > Key: HBASE-22422 > URL: https://issues.apache.org/jira/browse/HBASE-22422 > Project: HBase > Issue Type: Sub-task > Components: BlockCache >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Attachments: 0001-debug2.patch, 0001-debug2.patch, 0001-debug2.patch, > 0001-debug3.patch, 0001-debug4.patch, > HBASE-22422-qps-after-fix-the-zero-retain-bug.png, > HBASE-22422.HBASE-21879.v01.patch, LRUBlockCache-getBlock.png, debug.patch, > failed-to-check-positive-on-web-ui.png, image-2019-05-15-12-00-03-641.png > > > After runing YCSB scan/get benchmark in our XiaoMi cluster, we found the get > QPS dropped from 25000/s to hunderds per second in a cluster with five > nodes. > After enable the debug log at YCSB client side, I found the following > stacktrace , see > https://issues.apache.org/jira/secure/attachment/12968745/image-2019-05-15-12-00-03-641.png. > > After looking into the stractrace, I can ensure that the zero refCnt block is > an intermedia index block, see [2] http://hbase.apache.org/images/hfilev2.png > Need a patch to fix this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [hbase] openinx commented on issue #242: HBASE-22422 Retain an ByteBuff with refCnt=0 when getBlock from LRUCache
openinx commented on issue #242: HBASE-22422 Retain an ByteBuff with refCnt=0 when getBlock from LRUCache URL: https://github.com/apache/hbase/pull/242#issuecomment-495834356 Seems no hadoop QA feedback ? It's strange... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-15867) Move HBase replication tracking from ZooKeeper to HBase
[ https://issues.apache.org/jira/browse/HBASE-15867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16848020#comment-16848020 ] Zheng Hu commented on HBASE-15867: -- [~xucang], Thanks for your attention. For this feature, the core obstacle is the problem described in HBASE-20166. We've also have a discussion in mail list with title: *[DISCUSS] A Problem When Start HBase Cluster Using Table Based Replication* , You can see that. Thought about the solution before, we may need to refactor the master/RS startup procedure, seems lots of change. After some evaluation we thought that the HBase 2.x stability and performance was worth spending more time at that time, so we made the priority of this feature not so high. Maybe it's the time for now :-) If you have some time now, yeah, can resume this effort. For myself, I'm mainly absorbed in HBASE-21879. Once all those subtasks get resolved , maybe I give a hand for this feature. Thanks. > Move HBase replication tracking from ZooKeeper to HBase > --- > > Key: HBASE-15867 > URL: https://issues.apache.org/jira/browse/HBASE-15867 > Project: HBase > Issue Type: New Feature > Components: Replication >Affects Versions: 2.1.0 >Reporter: Joseph >Assignee: Zheng Hu >Priority: Major > Fix For: 2.3.0 > > > Move the WAL file and offset tracking out of ZooKeeper and into an HBase > table called hbase:replication. > The largest three new changes will be two classes ReplicationTableBase, > TableBasedReplicationQueues, and TableBasedReplicationQueuesClient. As of now > ReplicationPeers and HFileRef's tracking will not be implemented. Subtasks > have been filed for these two jobs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [hbase] Apache9 commented on issue #249: HBASE-22455 Split TestReplicationStatus
Apache9 commented on issue #249: HBASE-22455 Split TestReplicationStatus URL: https://github.com/apache/hbase/pull/249#issuecomment-495821335 The failed UT is not related. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22467) WebUI changes to enable Apache Knox UI proxying
[
https://issues.apache.org/jira/browse/HBASE-22467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847986#comment-16847986
]
HBase QA commented on HBASE-22467:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
49s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
41s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
49s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 53s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.0.3 3.1.2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
15s{color} | {color:green} hbase-http in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}252m
14s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
45s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}312m 28s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce base:
https://builds.apache.org/job/PreCommit-HBASE-Build/417/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-22467 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12969683/HBASE-22467.003.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 785eaeb531db 4.4.0-143-generic #169~14.04.2-Ubuntu SMP Wed Feb
13 15:00:41 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 96d2aabfad |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181
[jira] [Commented] (HBASE-20782) Fix duplication of TestServletFilter.access
[
https://issues.apache.org/jira/browse/HBASE-20782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847931#comment-16847931
]
HBase QA commented on HBASE-20782:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
4s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
36s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
41s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 29s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.0.3 3.1.2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
58s{color} | {color:green} hbase-http in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 45m 13s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce base:
https://builds.apache.org/job/PreCommit-HBASE-Build/418/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-20782 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12969693/HBASE-20782.master.006.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux e12d7f8fd628 4.4.0-143-generic #169~14.04.2-Ubuntu SMP Wed Feb
13 15:00:41 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 96d2aabfad |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.11 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/418/testReport/ |
| Max. process+thread count | 270 (vs. ulimit of 1) |
| modules | C: hbase-http U: hbase-http |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/418/console |
| Powered by | Apache Yetus 0.9.0 http://yetus.apache.org |
This message was automatically generated.
> Fix duplication of
[jira] [Comment Edited] (HBASE-15867) Move HBase replication tracking from ZooKeeper to HBase
[ https://issues.apache.org/jira/browse/HBASE-15867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847916#comment-16847916 ] Xu Cang edited comment on HBASE-15867 at 5/24/19 9:51 PM: -- Can someone briefly summarize what's the current state of this JIRA? Are there any architectural blockers or big concerns? Thank you! I am very interested in this Jira since it's extremely useful. If I want to resume this effort, do you have some suggestions or tips? [~Apache9] [~openinx] was (Author: xucang): Can someone briefly summarize what's the current state of this JIRA? Are there any architectural blockers or big concerns? Thank you! I am very interested in this Jira since it's extremely useful. If I want to resume this effort, do you hame some suggestions or tips? [~Apache9] [~openinx] > Move HBase replication tracking from ZooKeeper to HBase > --- > > Key: HBASE-15867 > URL: https://issues.apache.org/jira/browse/HBASE-15867 > Project: HBase > Issue Type: New Feature > Components: Replication >Affects Versions: 2.1.0 >Reporter: Joseph >Assignee: Zheng Hu >Priority: Major > Fix For: 2.3.0 > > > Move the WAL file and offset tracking out of ZooKeeper and into an HBase > table called hbase:replication. > The largest three new changes will be two classes ReplicationTableBase, > TableBasedReplicationQueues, and TableBasedReplicationQueuesClient. As of now > ReplicationPeers and HFileRef's tracking will not be implemented. Subtasks > have been filed for these two jobs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-15867) Move HBase replication tracking from ZooKeeper to HBase
[ https://issues.apache.org/jira/browse/HBASE-15867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847916#comment-16847916 ] Xu Cang commented on HBASE-15867: - Can someone briefly summarize what's the current state of this JIRA? Are there any architectural blockers or big concerns? Thank you! I am very interested in this Jira since it's extremely useful. If I want to resume this effort, do you hame some suggestions or tips? [~Apache9] [~openinx] > Move HBase replication tracking from ZooKeeper to HBase > --- > > Key: HBASE-15867 > URL: https://issues.apache.org/jira/browse/HBASE-15867 > Project: HBase > Issue Type: New Feature > Components: Replication >Affects Versions: 2.1.0 >Reporter: Joseph >Assignee: Zheng Hu >Priority: Major > Fix For: 2.3.0 > > > Move the WAL file and offset tracking out of ZooKeeper and into an HBase > table called hbase:replication. > The largest three new changes will be two classes ReplicationTableBase, > TableBasedReplicationQueues, and TableBasedReplicationQueuesClient. As of now > ReplicationPeers and HFileRef's tracking will not be implemented. Subtasks > have been filed for these two jobs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-19665) Add table based replication peers/queues storage back
[ https://issues.apache.org/jira/browse/HBASE-19665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xu Cang updated HBASE-19665: Comment: was deleted (was: Hi [~Apache9] I was reading this Jira and found it's very useful and found out the commit got reverted at 00095a2ef9442e3fd86c04876c9d91f2f8b23ad8 Could not find any context for the revert. could you please provide some info why this is reverted? Thanks! [~openinx] ) > Add table based replication peers/queues storage back > - > > Key: HBASE-19665 > URL: https://issues.apache.org/jira/browse/HBASE-19665 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Guanghao Zhang >Assignee: Zheng Hu >Priority: Major > Fix For: HBASE-15867 > > Attachments: HBASE-19665.v1.patch, HBASE-19665.v2.patch, > HBASE-19665.v3.patch, HBASE-19665.v3.patch, HBASE-19665.v4.patch, > HBASE-19665.v4.patch > > > We removed them after HBASE-19397. So open a issue to track this thing. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19665) Add table based replication peers/queues storage back
[ https://issues.apache.org/jira/browse/HBASE-19665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847912#comment-16847912 ] Xu Cang commented on HBASE-19665: - Hi [~Apache9] I was reading this Jira and found it's very useful and found out the commit got reverted at 00095a2ef9442e3fd86c04876c9d91f2f8b23ad8 Could not find any context for the revert. could you please provide some info why this is reverted? Thanks! [~openinx] > Add table based replication peers/queues storage back > - > > Key: HBASE-19665 > URL: https://issues.apache.org/jira/browse/HBASE-19665 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Guanghao Zhang >Assignee: Zheng Hu >Priority: Major > Fix For: HBASE-15867 > > Attachments: HBASE-19665.v1.patch, HBASE-19665.v2.patch, > HBASE-19665.v3.patch, HBASE-19665.v3.patch, HBASE-19665.v4.patch, > HBASE-19665.v4.patch > > > We removed them after HBASE-19397. So open a issue to track this thing. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22469) replace md5 checksum in saveVersion script with sha512 for hbase version information
[
https://issues.apache.org/jira/browse/HBASE-22469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847904#comment-16847904
]
Josh Elser commented on HBASE-22469:
[~dbist13], {{RSStatusTmpl.jamon}} and {{MasterStatusTmpl.jamon}} need to be
updated, too, as they currently say "HBase source MD5 checksum" which is wrong
after your change.
Can you put up a v2 with that fixed, too?
> replace md5 checksum in saveVersion script with sha512 for hbase version
> information
>
>
> Key: HBASE-22469
> URL: https://issues.apache.org/jira/browse/HBASE-22469
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0
> Environment: {code:java}
> Apache Maven 3.6.1 (d66c9c0b3152b2e69ee9bac180bb8fcc8e6af555;
> 2019-04-04T15:00:29-04:00)
> Maven home: /opt/apache-maven-3.6.1
> Java version: 1.8.0_172, vendor: Oracle Corporation, runtime:
> /Library/Java/JavaVirtualMachines/jdk1.8.0_172.jdk/Contents/Home/jre
> Default locale: en_US, platform encoding: UTF-8
> OS name: "mac os x", version: "10.13.6", arch: "x86_64", family: "mac"{code}
>Reporter: Artem Ervits
>Assignee: Artem Ervits
>Priority: Minor
> Attachments: HBASE-22469.v01.patch
>
>
> noticed that hbase version always prints out an md5 for source but since we
> long retired md5 from the releases, it makes sense to replace hbase version
> information as well. I also noticed that revision doesn't always show, case
> in point HBase 2.2.0 RC4 vote, revision=Unknown. In my tests, revision only
> appears if the compiled code was cloned from git and not a release candidate
> download file. In that sense, release candidates should be checked out rather
> than downloaded. This Jira only addresses replacement of the md5 sum.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20782) Fix duplication of TestServletFilter.access
[
https://issues.apache.org/jira/browse/HBASE-20782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xu Cang updated HBASE-20782:
Attachment: HBASE-20782.master.006.patch
> Fix duplication of TestServletFilter.access
> ---
>
> Key: HBASE-20782
> URL: https://issues.apache.org/jira/browse/HBASE-20782
> Project: HBase
> Issue Type: Improvement
>Reporter: Jan Hentschel
>Assignee: Xu Cang
>Priority: Minor
> Attachments: HBASE-20782.master.001.patch,
> HBASE-20782.master.002.patch, HBASE-20782.master.003.patch,
> HBASE-20782.master.003.patch, HBASE-20782.master.003.patch,
> HBASE-20782.master.003.patch, HBASE-20782.master.004.patch,
> HBASE-20782.master.004.patch, HBASE-20782.master.005.patch,
> HBASE-20782.master.006.patch
>
>
> The {{access}} method in {{TestServletFilter}} is duplicated in
> {{TestPathFilter}}. The method should be moved into a common place.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20782) Fix duplication of TestServletFilter.access
[
https://issues.apache.org/jira/browse/HBASE-20782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847902#comment-16847902
]
Xu Cang commented on HBASE-20782:
-
uploaded patch .006
> Fix duplication of TestServletFilter.access
> ---
>
> Key: HBASE-20782
> URL: https://issues.apache.org/jira/browse/HBASE-20782
> Project: HBase
> Issue Type: Improvement
>Reporter: Jan Hentschel
>Assignee: Xu Cang
>Priority: Minor
> Attachments: HBASE-20782.master.001.patch,
> HBASE-20782.master.002.patch, HBASE-20782.master.003.patch,
> HBASE-20782.master.003.patch, HBASE-20782.master.003.patch,
> HBASE-20782.master.003.patch, HBASE-20782.master.004.patch,
> HBASE-20782.master.004.patch, HBASE-20782.master.005.patch,
> HBASE-20782.master.006.patch
>
>
> The {{access}} method in {{TestServletFilter}} is duplicated in
> {{TestPathFilter}}. The method should be moved into a common place.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22467) WebUI changes to enable Apache Knox UI proxying
[ https://issues.apache.org/jira/browse/HBASE-22467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-22467: --- Fix Version/s: 2.2.1 2.1.5 3.0.0 > WebUI changes to enable Apache Knox UI proxying > --- > > Key: HBASE-22467 > URL: https://issues.apache.org/jira/browse/HBASE-22467 > Project: HBase > Issue Type: Improvement > Components: UI >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 3.0.0, 2.1.5, 2.2.1 > > Attachments: HBASE-22467.001.patch, HBASE-22467.002.patch, > HBASE-22467.003.patch > > > Apache Knox's gateway is nice in that it can provide centralized > authentication and authorization for a collection of service. Additionally, > it can expose "private" nodes through a single point (e.g. a gateway). This > makes our life as developers much easier in cloud deployments where it's not > straightforward to access nodes (e.g. they're running on some private > network). > KNOX-1866 captures the changes over there required to make HBase's proxying > actually work (definition lives there), but there were a few things we do in > our UI which made it hard/impossible to proxy it correctly. > * ProfilerServlet was dropping extra query parameters in the URL > * JSON task output on master/regionserver couldn't be disambiguated > * Some missing /master-status and /rs-status links couldn't be disambiguated > properly due to a lack of context > * Missing content-type set on a profiler servlet response -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-11062) htop
[ https://issues.apache.org/jira/browse/HBASE-11062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847866#comment-16847866 ] Andrew Purtell commented on HBASE-11062: +1 for hbtop. Fits with other naming > htop > > > Key: HBASE-11062 > URL: https://issues.apache.org/jira/browse/HBASE-11062 > Project: HBase > Issue Type: New Feature > Components: hbase-operator-tools >Reporter: Andrew Purtell >Assignee: Toshihiro Suzuki >Priority: Major > > A top-like monitor could be useful for testing, debugging, operations of > clusters of moderate size, and possibly for diagnosing issues in large > clusters. > Consider a curses interface like the one presented by atop > (http://www.atoptool.nl/images/screenshots/genericw.png) - with aggregate > metrics collected over a monitoring interval in the upper portion of the > pane, and a listing of discrete measurements sorted and filtered by various > criteria in the bottom part of the pane. One might imagine a cluster overview > with cluster aggregate metrics above and a list of regionservers sorted by > utilization below; and a regionserver view with process metrics above and a > list of metrics by operation type below, or a list of client connections, or > a list of threads, sorted by utilization, throughput, or latency. > Generically 'htop' is taken but would be distinctive in the HBase context, a > utility org.apache.hadoop.hbase.HTop > No need necessarily for a curses interface. Could be an external monitor with > a web front end as has been discussed before. I do like the idea of a process > that runs in a terminal because I interact with dev and test HBase clusters > exclusively by SSH. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-11062) htop
[ https://issues.apache.org/jira/browse/HBASE-11062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847865#comment-16847865 ] Josh Elser commented on HBASE-11062: Coming here after Toshi's demo IRL at NoSQL day – I'd echo JMS' warning about confusion with [https://hisham.hm/htop/]. Author's choice, of course, but I'd throw out "hbtop" as a name instead. You have a patch/PR for what you demo'ed (even with the LGPL dependency)? > htop > > > Key: HBASE-11062 > URL: https://issues.apache.org/jira/browse/HBASE-11062 > Project: HBase > Issue Type: New Feature > Components: hbase-operator-tools >Reporter: Andrew Purtell >Assignee: Toshihiro Suzuki >Priority: Major > > A top-like monitor could be useful for testing, debugging, operations of > clusters of moderate size, and possibly for diagnosing issues in large > clusters. > Consider a curses interface like the one presented by atop > (http://www.atoptool.nl/images/screenshots/genericw.png) - with aggregate > metrics collected over a monitoring interval in the upper portion of the > pane, and a listing of discrete measurements sorted and filtered by various > criteria in the bottom part of the pane. One might imagine a cluster overview > with cluster aggregate metrics above and a list of regionservers sorted by > utilization below; and a regionserver view with process metrics above and a > list of metrics by operation type below, or a list of client connections, or > a list of threads, sorted by utilization, throughput, or latency. > Generically 'htop' is taken but would be distinctive in the HBase context, a > utility org.apache.hadoop.hbase.HTop > No need necessarily for a curses interface. Could be an external monitor with > a web front end as has been discussed before. I do like the idea of a process > that runs in a terminal because I interact with dev and test HBase clusters > exclusively by SSH. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22434) Improve clear meta cache
[ https://issues.apache.org/jira/browse/HBASE-22434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847863#comment-16847863 ] Xu Cang commented on HBASE-22434: - https://issues.apache.org/jira/browse/HBASE-22381 This one described similar issue and [~openinx] expressed sounding concern regarding adding TimeoutIOException into the specialException set. > Improve clear meta cache > > > Key: HBASE-22434 > URL: https://issues.apache.org/jira/browse/HBASE-22434 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 2.2.0 >Reporter: binlijin >Assignee: binlijin >Priority: Major > Attachments: HBASE-22434-master-v2.patch, > HBASE-22434-master-v3.patch, cleanServerCache.jpg, cleanServerCache2.jpg > > > 2.2.0 clear meta cache more aggressive and give more load to meta region, > need to improve the clear logic. This is relate to the > AsyncRequestFutureImpl#cleanServerCache logic. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22466) Try running create-release against branch-1
[ https://issues.apache.org/jira/browse/HBASE-22466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847862#comment-16847862 ] stack commented on HBASE-22466: --- [~apurt...@yahoo.com] You come across this below? 17:37:55 [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (hadoop-profile-min-maven-min-java-banned-xerces) @ hbase-annotations --- 17:37:55 [INFO] Restricted to JDK 1.7 yet jdk.tools:jdk.tools:jar:1.8:system contains org/relaxng/datatype/Datatype.class targeted to JDK 1.8 17:37:55 [WARNING] Rule 4: org.apache.maven.plugins.enforcer.EnforceBytecodeVersion failed with message: HBase has unsupported dependencies. HBase requires that all dependencies be compiled with version 1.7 or earlier of the JDK to properly build from source. You appear to be using a newer dependency. You can use either "mvn -version" or "mvn enforcer:display-info" to verify what version is active. Non-release builds can temporarily build with a newer JDK version by setting the 'compileSource' property (eg. mvn -DcompileSource=1.8 clean package). Found Banned Dependency: jdk.tools:jdk.tools:jar:1.8 Use 'mvn dependency:tree' to locate the source of the banned dependencies. 17:37:55 [INFO] 17:37:55 [INFO] Reactor Summary: 17:37:55 [INFO] 17:37:55 [INFO] Apache HBase ... SUCCESS [02:03 min] 17:37:55 [INFO] Apache HBase - Checkstyle .. SUCCESS [ 9.125 s] 17:37:55 [INFO] Apache HBase - Resource Bundle . SUCCESS [ 3.966 s] 17:37:55 [INFO] Apache HBase -
[GitHub] [hbase] Apache-HBase commented on issue #249: HBASE-22455 Split TestReplicationStatus
Apache-HBase commented on issue #249: HBASE-22455 Split TestReplicationStatus URL: https://github.com/apache/hbase/pull/249#issuecomment-495763128 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 54 | Docker mode activated. | ||| _ Prechecks _ | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 7 new or modified test files. | ||| _ master Compile Tests _ | | +1 | mvninstall | 238 | master passed | | +1 | compile | 52 | master passed | | +1 | checkstyle | 65 | master passed | | +1 | shadedjars | 270 | branch has no errors when building our shaded downstream artifacts. | | +1 | findbugs | 193 | master passed | | +1 | javadoc | 33 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 241 | the patch passed | | +1 | compile | 51 | the patch passed | | +1 | javac | 51 | the patch passed | | +1 | checkstyle | 67 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 265 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 983 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.0.3 3.1.2. | | +1 | findbugs | 207 | the patch passed | | +1 | javadoc | 33 | the patch passed | ||| _ Other Tests _ | | -1 | unit | 15596 | hbase-server in the patch failed. | | +1 | asflicense | 22 | The patch does not generate ASF License warnings. | | | | 18662 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.regionserver.TestRegionReplicasWithModifyTable | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-249/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/249 | | Optional Tests | dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 692a2149afb7 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | master / 96d2aabfad | | maven | version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) | | Default Java | 1.8.0_181 | | findbugs | v3.1.11 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-249/3/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-249/3/testReport/ | | Max. process+thread count | 5330 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-249/3/console | | Powered by | Apache Yetus 0.9.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-22467) WebUI changes to enable Apache Knox UI proxying
[ https://issues.apache.org/jira/browse/HBASE-22467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-22467: --- Attachment: HBASE-22467.003.patch > WebUI changes to enable Apache Knox UI proxying > --- > > Key: HBASE-22467 > URL: https://issues.apache.org/jira/browse/HBASE-22467 > Project: HBase > Issue Type: Improvement > Components: UI >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Attachments: HBASE-22467.001.patch, HBASE-22467.002.patch, > HBASE-22467.003.patch > > > Apache Knox's gateway is nice in that it can provide centralized > authentication and authorization for a collection of service. Additionally, > it can expose "private" nodes through a single point (e.g. a gateway). This > makes our life as developers much easier in cloud deployments where it's not > straightforward to access nodes (e.g. they're running on some private > network). > KNOX-1866 captures the changes over there required to make HBase's proxying > actually work (definition lives there), but there were a few things we do in > our UI which made it hard/impossible to proxy it correctly. > * ProfilerServlet was dropping extra query parameters in the URL > * JSON task output on master/regionserver couldn't be disambiguated > * Some missing /master-status and /rs-status links couldn't be disambiguated > properly due to a lack of context > * Missing content-type set on a profiler servlet response -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22467) WebUI changes to enable Apache Knox UI proxying
[ https://issues.apache.org/jira/browse/HBASE-22467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847846#comment-16847846 ] Josh Elser commented on HBASE-22467: .003 adds license, category, and class rule to new test class. > WebUI changes to enable Apache Knox UI proxying > --- > > Key: HBASE-22467 > URL: https://issues.apache.org/jira/browse/HBASE-22467 > Project: HBase > Issue Type: Improvement > Components: UI >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Attachments: HBASE-22467.001.patch, HBASE-22467.002.patch, > HBASE-22467.003.patch > > > Apache Knox's gateway is nice in that it can provide centralized > authentication and authorization for a collection of service. Additionally, > it can expose "private" nodes through a single point (e.g. a gateway). This > makes our life as developers much easier in cloud deployments where it's not > straightforward to access nodes (e.g. they're running on some private > network). > KNOX-1866 captures the changes over there required to make HBase's proxying > actually work (definition lives there), but there were a few things we do in > our UI which made it hard/impossible to proxy it correctly. > * ProfilerServlet was dropping extra query parameters in the URL > * JSON task output on master/regionserver couldn't be disambiguated > * Some missing /master-status and /rs-status links couldn't be disambiguated > properly due to a lack of context > * Missing content-type set on a profiler servlet response -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22467) WebUI changes to enable Apache Knox UI proxying
[
https://issues.apache.org/jira/browse/HBASE-22467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847831#comment-16847831
]
HBase QA commented on HBASE-22467:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
43s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
41s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red} 2m
38s{color} | {color:red} patch has 10 errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 34s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.0.3 3.1.2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
13s{color} | {color:green} hbase-http in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}203m 58s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
42s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}260m 47s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.TestCheckTestClasses |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce base:
https://builds.apache.org/job/PreCommit-HBASE-Build/416/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-22467 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12969651/HBASE-22467.002.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 69c5b106705c 4.4.0-143-generic #169~14.04.2-Ubuntu SMP Wed Feb
13 15:00:41 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 96d2aabfad |
| maven | version: Apache Maven 3.5.4
[jira] [Commented] (HBASE-22457) Harden the HBase HFile reader reference counting
[ https://issues.apache.org/jira/browse/HBASE-22457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847806#comment-16847806 ] Andrew Purtell commented on HBASE-22457: [~lhofhansl] do you want to really submit that patch? put this in PA status etc? > Harden the HBase HFile reader reference counting > > > Key: HBASE-22457 > URL: https://issues.apache.org/jira/browse/HBASE-22457 > Project: HBase > Issue Type: Brainstorming >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Major > Attachments: 22457-random-1.5.txt > > > The problem that any coprocessor hook that replaces a passed scanner without > closing it can cause an incorrect reference count. > This was bad and wrong before of course, but now it has pretty bad > consequences, since an incorrect reference could will prevent HFiles from > being archived indefinitely. > All hooks that are passed a scanner and return a scanner are suspect, since > the returned scanner may or may not close the passed scanner: > * preCompact > * preCompactScannerOpen > * preFlush > * preFlushScannerOpen > * preScannerOpen > * preStoreScannerOpen > * preStoreFileReaderOpen...? (not sure about this one, it could mess with the > reader) > I sampled the Phoenix and also Tephra code, and found a few instances where > this is happening. > And for those I filed issued: TEPHRA-300, PHOENIX-5291 > (We're not using Tephra) > The Phoenix ones should be rare. In our case we are seeing readers with > refCount > 1000. > Perhaps there are other issues, a path where not all exceptions are caught > and scanner is left open that way perhaps. (Generally I am not a fan of > reference counting in complex systems - it's too easy to miss something. But > that's a different discussion. :) ). > Let's brainstorm some way in which we can harden this. > [~ram_krish], [~anoop.hbase], [~apurtell] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [hbase] apurtell commented on issue #248: HBASE-22459 Expose store reader reference count
apurtell commented on issue #248: HBASE-22459 Expose store reader reference count URL: https://github.com/apache/hbase/pull/248#issuecomment-495741964 No problem, will remove from Store and leave in HStore only. Back soon. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] apurtell commented on issue #248: HBASE-22459 Expose store reader reference count
apurtell commented on issue #248: HBASE-22459 Expose store reader reference count URL: https://github.com/apache/hbase/pull/248#issuecomment-495742124 TestMasterReplication issue does not seem related, but will check This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] xcangCRM commented on issue #248: HBASE-22459 Expose store reader reference count
xcangCRM commented on issue #248: HBASE-22459 Expose store reader reference count URL: https://github.com/apache/hbase/pull/248#issuecomment-495738500 Other than Duo's comment , LGTM. +1 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22434) Improve clear meta cache
[
https://issues.apache.org/jira/browse/HBASE-22434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847749#comment-16847749
]
Xu Cang commented on HBASE-22434:
-
{code:java}
public static boolean isCleanServerCacheException(Throwable cur)
{code}
I found this method is confusing to me. If there is no exception found, it
returns true and causes clear cache action.
Is this the real intention?
And this part
{code:java}
boolean notCleanServerException =
(isSpeialException(cur) || cur instanceof DoNotRetryIOException);
// Exception like ConnectException should clean server's meta cache
return !notCleanServerException;{code}
Can this be cleant up a bit?
we can do something like
if (isSpecialException(curr) || cur instanceof DoNotRetryIOException) {
return fasle;
}
Meta cache behavior is crucial regarding HBase region access performance which
has high impact. We should treat this cautiously. Thanks!
> Improve clear meta cache
>
>
> Key: HBASE-22434
> URL: https://issues.apache.org/jira/browse/HBASE-22434
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 2.2.0
>Reporter: binlijin
>Assignee: binlijin
>Priority: Major
> Attachments: HBASE-22434-master-v2.patch,
> HBASE-22434-master-v3.patch, cleanServerCache.jpg, cleanServerCache2.jpg
>
>
> 2.2.0 clear meta cache more aggressive and give more load to meta region,
> need to improve the clear logic. This is relate to the
> AsyncRequestFutureImpl#cleanServerCache logic.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22403) Balance in RSGroup should consider throttling and a failure affects the whole
[ https://issues.apache.org/jira/browse/HBASE-22403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847670#comment-16847670 ] Xu Cang commented on HBASE-22403: - Good catch! lgtm. +1. Can you port this to branch 1 too? Thanks! > Balance in RSGroup should consider throttling and a failure affects the whole > - > > Key: HBASE-22403 > URL: https://issues.apache.org/jira/browse/HBASE-22403 > Project: HBase > Issue Type: Improvement > Components: rsgroup >Affects Versions: 2.2.0 >Reporter: Xiaolin Ha >Assignee: Xiaolin Ha >Priority: Major > Attachments: HBASE-22403.branch-2.2.001.patch, > HBASE-22403.branch-2.2.002.patch, HBASE-22403.master.001.patch, > HBASE-22403.master.002.patch > > > balanceRSGroup(groupName) excutes region move plans concurrently, which will > affect the availability of relevant tables. And a plan fails will cause the > whole balance plan abort. > As mentioned in master balance issues, HBASE-17178, HBASE-21260 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21879) Read HFile's block to ByteBuffer directly instead of to byte for reducing young gc purpose
[
https://issues.apache.org/jira/browse/HBASE-21879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847621#comment-16847621
]
Hudson commented on HBASE-21879:
Results for branch HBASE-21879
[build #112 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/112/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/112//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/112//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/112//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Read HFile's block to ByteBuffer directly instead of to byte for reducing
> young gc purpose
> --
>
> Key: HBASE-21879
> URL: https://issues.apache.org/jira/browse/HBASE-21879
> Project: HBase
> Issue Type: Improvement
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: HBASE-21879.v1.patch, HBASE-21879.v1.patch,
> QPS-latencies-before-HBASE-21879.png, gc-data-before-HBASE-21879.png
>
>
> In HFileBlock#readBlockDataInternal, we have the following:
> {code}
> @VisibleForTesting
> protected HFileBlock readBlockDataInternal(FSDataInputStream is, long offset,
> long onDiskSizeWithHeaderL, boolean pread, boolean verifyChecksum,
> boolean updateMetrics)
> throws IOException {
> // .
> // TODO: Make this ByteBuffer-based. Will make it easier to go to HDFS with
> BBPool (offheap).
> byte [] onDiskBlock = new byte[onDiskSizeWithHeader + hdrSize];
> int nextBlockOnDiskSize = readAtOffset(is, onDiskBlock, preReadHeaderSize,
> onDiskSizeWithHeader - preReadHeaderSize, true, offset +
> preReadHeaderSize, pread);
> if (headerBuf != null) {
> // ...
> }
> // ...
> }
> {code}
> In the read path, we still read the block from hfile to on-heap byte[], then
> copy the on-heap byte[] to offheap bucket cache asynchronously, and in my
> 100% get performance test, I also observed some frequent young gc, The
> largest memory footprint in the young gen should be the on-heap block byte[].
> In fact, we can read HFile's block to ByteBuffer directly instead of to
> byte[] for reducing young gc purpose. we did not implement this before,
> because no ByteBuffer reading interface in the older HDFS client, but 2.7+
> has supported this now, so we can fix this now. I think.
> Will provide an patch and some perf-comparison for this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] [hbase] openinx commented on issue #242: HBASE-22422 Retain an ByteBuff with refCnt=0 when getBlock from LRUCache
openinx commented on issue #242: HBASE-22422 Retain an ByteBuff with refCnt=0 when getBlock from LRUCache URL: https://github.com/apache/hbase/pull/242#issuecomment-495664563 Update the patch with the final fix and UT, FYI @Apache9 @ramkrish86 @anoopsjohn This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22467) WebUI changes to enable Apache Knox UI proxying
[ https://issues.apache.org/jira/browse/HBASE-22467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847594#comment-16847594 ] Josh Elser commented on HBASE-22467: .002 fixes the checkstyle and tries to make the findbugs dings happy. Not sure if Findbugs will be happy enough with my quick and dirty sanitizing method. > WebUI changes to enable Apache Knox UI proxying > --- > > Key: HBASE-22467 > URL: https://issues.apache.org/jira/browse/HBASE-22467 > Project: HBase > Issue Type: Improvement > Components: UI >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Attachments: HBASE-22467.001.patch, HBASE-22467.002.patch > > > Apache Knox's gateway is nice in that it can provide centralized > authentication and authorization for a collection of service. Additionally, > it can expose "private" nodes through a single point (e.g. a gateway). This > makes our life as developers much easier in cloud deployments where it's not > straightforward to access nodes (e.g. they're running on some private > network). > KNOX-1866 captures the changes over there required to make HBase's proxying > actually work (definition lives there), but there were a few things we do in > our UI which made it hard/impossible to proxy it correctly. > * ProfilerServlet was dropping extra query parameters in the URL > * JSON task output on master/regionserver couldn't be disambiguated > * Some missing /master-status and /rs-status links couldn't be disambiguated > properly due to a lack of context > * Missing content-type set on a profiler servlet response -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22467) WebUI changes to enable Apache Knox UI proxying
[ https://issues.apache.org/jira/browse/HBASE-22467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-22467: --- Attachment: HBASE-22467.002.patch > WebUI changes to enable Apache Knox UI proxying > --- > > Key: HBASE-22467 > URL: https://issues.apache.org/jira/browse/HBASE-22467 > Project: HBase > Issue Type: Improvement > Components: UI >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Attachments: HBASE-22467.001.patch, HBASE-22467.002.patch > > > Apache Knox's gateway is nice in that it can provide centralized > authentication and authorization for a collection of service. Additionally, > it can expose "private" nodes through a single point (e.g. a gateway). This > makes our life as developers much easier in cloud deployments where it's not > straightforward to access nodes (e.g. they're running on some private > network). > KNOX-1866 captures the changes over there required to make HBase's proxying > actually work (definition lives there), but there were a few things we do in > our UI which made it hard/impossible to proxy it correctly. > * ProfilerServlet was dropping extra query parameters in the URL > * JSON task output on master/regionserver couldn't be disambiguated > * Some missing /master-status and /rs-status links couldn't be disambiguated > properly due to a lack of context > * Missing content-type set on a profiler servlet response -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22469) replace md5 checksum in saveVersion script with sha512 for hbase version information
[
https://issues.apache.org/jira/browse/HBASE-22469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847560#comment-16847560
]
Josh Elser commented on HBASE-22469:
{quote}I also noticed that revision doesn't always show, case in point HBase
2.2.0 RC4 vote, revision=Unknown. In my tests, revision only appears if the
compiled code was cloned from git and not a release candidate download file
{quote}
Yeah, this is a result of the maven/git plugin(s) we have configured, IIRC.
Like you said, if you don't build from a git checkout, we can't know what the
revision was.
That said, it looks what saveVersion.sh does is conditionally regenerates the
Version.java file which has a checksum over all of the source files. Switching
that from md5 to sha512 seems fine, but I don't think it's giving us anything
new (other than eliminate glaring eyes from thinking that there is a problem).
It's not like a collision or someone figuring out somethign else that generates
our checksum would invalidate our security model.
+1 on qa, I think.
> replace md5 checksum in saveVersion script with sha512 for hbase version
> information
>
>
> Key: HBASE-22469
> URL: https://issues.apache.org/jira/browse/HBASE-22469
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0
> Environment: {code:java}
> Apache Maven 3.6.1 (d66c9c0b3152b2e69ee9bac180bb8fcc8e6af555;
> 2019-04-04T15:00:29-04:00)
> Maven home: /opt/apache-maven-3.6.1
> Java version: 1.8.0_172, vendor: Oracle Corporation, runtime:
> /Library/Java/JavaVirtualMachines/jdk1.8.0_172.jdk/Contents/Home/jre
> Default locale: en_US, platform encoding: UTF-8
> OS name: "mac os x", version: "10.13.6", arch: "x86_64", family: "mac"{code}
>Reporter: Artem Ervits
>Assignee: Artem Ervits
>Priority: Minor
> Attachments: HBASE-22469.v01.patch
>
>
> noticed that hbase version always prints out an md5 for source but since we
> long retired md5 from the releases, it makes sense to replace hbase version
> information as well. I also noticed that revision doesn't always show, case
> in point HBase 2.2.0 RC4 vote, revision=Unknown. In my tests, revision only
> appears if the compiled code was cloned from git and not a release candidate
> download file. In that sense, release candidates should be checked out rather
> than downloaded. This Jira only addresses replacement of the md5 sum.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22469) replace md5 checksum in saveVersion script with sha512 for hbase version information
[
https://issues.apache.org/jira/browse/HBASE-22469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847556#comment-16847556
]
HBase QA commented on HBASE-22469:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
50s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m
0s{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
0s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 1m 16s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce base:
https://builds.apache.org/job/PreCommit-HBASE-Build/415/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-22469 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12969642/HBASE-22469.v01.patch
|
| Optional Tests | dupname asflicense shellcheck shelldocs |
| uname | Linux 9e8d48d0ce22 4.4.0-143-generic #169~14.04.2-Ubuntu SMP Wed Feb
13 15:00:41 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 96d2aabfad |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| shellcheck | v0.4.4 |
| Max. process+thread count | 40 (vs. ulimit of 1) |
| modules | C: hbase-common U: hbase-common |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/415/console |
| Powered by | Apache Yetus 0.9.0 http://yetus.apache.org |
This message was automatically generated.
> replace md5 checksum in saveVersion script with sha512 for hbase version
> information
>
>
> Key: HBASE-22469
> URL: https://issues.apache.org/jira/browse/HBASE-22469
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0
> Environment: {code:java}
> Apache Maven 3.6.1 (d66c9c0b3152b2e69ee9bac180bb8fcc8e6af555;
> 2019-04-04T15:00:29-04:00)
> Maven home: /opt/apache-maven-3.6.1
> Java version: 1.8.0_172, vendor: Oracle Corporation, runtime:
> /Library/Java/JavaVirtualMachines/jdk1.8.0_172.jdk/Contents/Home/jre
> Default locale: en_US, platform encoding: UTF-8
> OS name: "mac os x", version: "10.13.6", arch: "x86_64", family: "mac"{code}
>Reporter: Artem Ervits
>Assignee: Artem Ervits
>Priority: Minor
> Attachments: HBASE-22469.v01.patch
>
>
> noticed that hbase version always prints out an md5 for source but since we
> long retired md5 from the releases, it makes sense to replace hbase version
> information as well. I also noticed that revision doesn't always show, case
> in point HBase 2.2.0 RC4 vote, revision=Unknown. In my tests, revision only
> appears if the compiled code was cloned from git and not a release candidate
> download file. In that sense, release candidates should be checked out rather
> than downloaded. This Jira only addresses replacement of the md5 sum.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22469) replace md5 checksum in saveVersion script with sha512 for hbase version information
[
https://issues.apache.org/jira/browse/HBASE-22469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated HBASE-22469:
-
Status: Patch Available (was: Open)
> replace md5 checksum in saveVersion script with sha512 for hbase version
> information
>
>
> Key: HBASE-22469
> URL: https://issues.apache.org/jira/browse/HBASE-22469
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0
> Environment: {code:java}
> Apache Maven 3.6.1 (d66c9c0b3152b2e69ee9bac180bb8fcc8e6af555;
> 2019-04-04T15:00:29-04:00)
> Maven home: /opt/apache-maven-3.6.1
> Java version: 1.8.0_172, vendor: Oracle Corporation, runtime:
> /Library/Java/JavaVirtualMachines/jdk1.8.0_172.jdk/Contents/Home/jre
> Default locale: en_US, platform encoding: UTF-8
> OS name: "mac os x", version: "10.13.6", arch: "x86_64", family: "mac"{code}
>Reporter: Artem Ervits
>Assignee: Artem Ervits
>Priority: Minor
> Attachments: HBASE-22469.v01.patch
>
>
> noticed that hbase version always prints out an md5 for source but since we
> long retired md5 from the releases, it makes sense to replace hbase version
> information as well. I also noticed that revision doesn't always show, case
> in point HBase 2.2.0 RC4 vote, revision=Unknown. In my tests, revision only
> appears if the compiled code was cloned from git and not a release candidate
> download file. In that sense, release candidates should be checked out rather
> than downloaded. This Jira only addresses replacement of the md5 sum.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22469) replace md5 checksum in saveVersion script with sha512 for hbase version information
[
https://issues.apache.org/jira/browse/HBASE-22469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated HBASE-22469:
-
Attachment: HBASE-22469.v01.patch
> replace md5 checksum in saveVersion script with sha512 for hbase version
> information
>
>
> Key: HBASE-22469
> URL: https://issues.apache.org/jira/browse/HBASE-22469
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0
> Environment: {code:java}
> Apache Maven 3.6.1 (d66c9c0b3152b2e69ee9bac180bb8fcc8e6af555;
> 2019-04-04T15:00:29-04:00)
> Maven home: /opt/apache-maven-3.6.1
> Java version: 1.8.0_172, vendor: Oracle Corporation, runtime:
> /Library/Java/JavaVirtualMachines/jdk1.8.0_172.jdk/Contents/Home/jre
> Default locale: en_US, platform encoding: UTF-8
> OS name: "mac os x", version: "10.13.6", arch: "x86_64", family: "mac"{code}
>Reporter: Artem Ervits
>Assignee: Artem Ervits
>Priority: Minor
> Attachments: HBASE-22469.v01.patch
>
>
> noticed that hbase version always prints out an md5 for source but since we
> long retired md5 from the releases, it makes sense to replace hbase version
> information as well. I also noticed that revision doesn't always show, case
> in point HBase 2.2.0 RC4 vote, revision=Unknown. In my tests, revision only
> appears if the compiled code was cloned from git and not a release candidate
> download file. In that sense, release candidates should be checked out rather
> than downloaded. This Jira only addresses replacement of the md5 sum.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-22469) replace md5 checksum in saveVersion script with sha512 for hbase version information
Artem Ervits created HBASE-22469:
Summary: replace md5 checksum in saveVersion script with sha512
for hbase version information
Key: HBASE-22469
URL: https://issues.apache.org/jira/browse/HBASE-22469
Project: HBase
Issue Type: Improvement
Affects Versions: 3.0.0
Environment: {code:java}
Apache Maven 3.6.1 (d66c9c0b3152b2e69ee9bac180bb8fcc8e6af555;
2019-04-04T15:00:29-04:00)
Maven home: /opt/apache-maven-3.6.1
Java version: 1.8.0_172, vendor: Oracle Corporation, runtime:
/Library/Java/JavaVirtualMachines/jdk1.8.0_172.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.13.6", arch: "x86_64", family: "mac"{code}
Reporter: Artem Ervits
Assignee: Artem Ervits
noticed that hbase version always prints out an md5 for source but since we
long retired md5 from the releases, it makes sense to replace hbase version
information as well. I also noticed that revision doesn't always show, case in
point HBase 2.2.0 RC4 vote, revision=Unknown. In my tests, revision only
appears if the compiled code was cloned from git and not a release candidate
download file. In that sense, release candidates should be checked out rather
than downloaded. This Jira only addresses replacement of the md5 sum.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21512) Introduce an AsyncClusterConnection and replace the usage of ClusterConnection
[
https://issues.apache.org/jira/browse/HBASE-21512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847540#comment-16847540
]
Hudson commented on HBASE-21512:
Results for branch HBASE-21512
[build #239 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/239/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/239//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/239//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/239//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/239//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Introduce an AsyncClusterConnection and replace the usage of ClusterConnection
> --
>
> Key: HBASE-21512
> URL: https://issues.apache.org/jira/browse/HBASE-21512
> Project: HBase
> Issue Type: Umbrella
>Reporter: Duo Zhang
>Priority: Major
> Fix For: 3.0.0
>
>
> At least for the RSProcedureDispatcher, with CompletableFuture we do not need
> to set a delay and use a thread pool any more, which could reduce the
> resource usage and also the latency.
> Once this is done, I think we can remove the ClusterConnection completely,
> and start to rewrite the old sync client based on the async client, which
> could reduce the code base a lot for our client.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22466) Try running create-release against branch-1
[
https://issues.apache.org/jira/browse/HBASE-22466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847462#comment-16847462
]
Hudson commented on HBASE-22466:
Results for branch HBASE-22466
[build #9 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22466/9/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22466/9//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22466/9//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22466/9//JDK8_Nightly_Build_Report_(Hadoop3)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> Try running create-release against branch-1
> ---
>
> Key: HBASE-22466
> URL: https://issues.apache.org/jira/browse/HBASE-22466
> Project: HBase
> Issue Type: Sub-task
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: HBASE-22466-1.5.0
>
>
> The parent create-release candidate just ran unaccompanied end to end to make
> a 2.1.5 RC. Let me see if it will work on branch-1.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22462) Should run a 'mvn install' at the end of hadoop check in pre commit job
[
https://issues.apache.org/jira/browse/HBASE-22462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847436#comment-16847436
]
Hudson commented on HBASE-22462:
Results for branch branch-2.2
[build #283 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/283/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/283//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/283//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/283//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/283//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Should run a 'mvn install' at the end of hadoop check in pre commit job
> ---
>
> Key: HBASE-22462
> URL: https://issues.apache.org/jira/browse/HBASE-22462
> Project: HBase
> Issue Type: Bug
> Components: build
>Reporter: Duo Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 1.4.10, 2.3.0, 2.0.6, 2.1.5, 2.2.1
>
> Attachments: HBASE-22462.master.001.patch,
> HBASE-22462.master.002.patch, HBASE-22462.master.003.patch,
> HBASE-22462.master.004.patch, HBASE-22462.master.addendum.patch,
> HBASE-22462.master.test.patch
>
>
> Now for branch-2.x, we will build with hadoop 3.x in the hadoop check stage,
> so in later unit check, if we run from a sub module, then the other hbase
> modules we depend on will depend on hadoop 3.x while the module we build will
> depend on 2.x, this will cause the following problem
> {noformat}
> 2019-05-23 04:47:41,156 WARN [RS_CLOSE_META-regionserver/b001f91a596c:0-0]
> handler.AssignRegionHandler(157): Fatal error occurred while opening region
> hbase:meta,,1.1588230740, aborting...
> java.lang.IncompatibleClassChangeError: Found class
> org.apache.hadoop.hdfs.protocol.HdfsFileStatus, but interface was expected
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createOutput(FanOutOneBlockAsyncDFSOutputHelper.java:496)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$400(FanOutOneBlockAsyncDFSOutputHelper.java:116)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$8.doCall(FanOutOneBlockAsyncDFSOutputHelper.java:576)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$8.doCall(FanOutOneBlockAsyncDFSOutputHelper.java:571)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createOutput(FanOutOneBlockAsyncDFSOutputHelper.java:584)
> at
> org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:51)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:169)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:166)
> at
> org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:105)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createAsyncWriter(AsyncFSWAL.java:664)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:670)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:128)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:832)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:538)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:479)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:156)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:61)
> at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:293)
> at
>
[jira] [Commented] (HBASE-22411) Refactor codes of moving reigons in RSGroup
[
https://issues.apache.org/jira/browse/HBASE-22411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847405#comment-16847405
]
HBase QA commented on HBASE-22411:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
30s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
33s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
39s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 19s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.0.3 3.1.2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
58s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 46m 44s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce base:
https://builds.apache.org/job/PreCommit-HBASE-Build/414/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-22411 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12969606/HBASE-22411.master.011.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 1c69eb7ae9d4 4.4.0-137-generic #163~14.04.1-Ubuntu SMP Mon Sep
24 17:14:57 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 96d2aabfad |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.11 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/414/testReport/ |
| Max. process+thread count | 4472 (vs. ulimit of 1) |
| modules | C: hbase-rsgroup U: hbase-rsgroup |
| Console output |
[jira] [Commented] (HBASE-22462) Should run a 'mvn install' at the end of hadoop check in pre commit job
[
https://issues.apache.org/jira/browse/HBASE-22462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847361#comment-16847361
]
Hudson commented on HBASE-22462:
Results for branch branch-2.1
[build #1183 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1183/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1183//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1183//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1183//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Should run a 'mvn install' at the end of hadoop check in pre commit job
> ---
>
> Key: HBASE-22462
> URL: https://issues.apache.org/jira/browse/HBASE-22462
> Project: HBase
> Issue Type: Bug
> Components: build
>Reporter: Duo Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 1.4.10, 2.3.0, 2.0.6, 2.1.5, 2.2.1
>
> Attachments: HBASE-22462.master.001.patch,
> HBASE-22462.master.002.patch, HBASE-22462.master.003.patch,
> HBASE-22462.master.004.patch, HBASE-22462.master.addendum.patch,
> HBASE-22462.master.test.patch
>
>
> Now for branch-2.x, we will build with hadoop 3.x in the hadoop check stage,
> so in later unit check, if we run from a sub module, then the other hbase
> modules we depend on will depend on hadoop 3.x while the module we build will
> depend on 2.x, this will cause the following problem
> {noformat}
> 2019-05-23 04:47:41,156 WARN [RS_CLOSE_META-regionserver/b001f91a596c:0-0]
> handler.AssignRegionHandler(157): Fatal error occurred while opening region
> hbase:meta,,1.1588230740, aborting...
> java.lang.IncompatibleClassChangeError: Found class
> org.apache.hadoop.hdfs.protocol.HdfsFileStatus, but interface was expected
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createOutput(FanOutOneBlockAsyncDFSOutputHelper.java:496)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$400(FanOutOneBlockAsyncDFSOutputHelper.java:116)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$8.doCall(FanOutOneBlockAsyncDFSOutputHelper.java:576)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$8.doCall(FanOutOneBlockAsyncDFSOutputHelper.java:571)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createOutput(FanOutOneBlockAsyncDFSOutputHelper.java:584)
> at
> org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:51)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:169)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:166)
> at
> org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:105)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createAsyncWriter(AsyncFSWAL.java:664)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:670)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:128)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:832)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:538)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:479)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:156)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:61)
> at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:293)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getWAL(HRegionServer.java:2170)
> at
> org.apache.hadoop.hbase.regionserver.handler.AssignRegionHandler.process(AssignRegionHandler.java:133)
> at
>
[jira] [Commented] (HBASE-22462) Should run a 'mvn install' at the end of hadoop check in pre commit job
[
https://issues.apache.org/jira/browse/HBASE-22462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847357#comment-16847357
]
Hudson commented on HBASE-22462:
Results for branch branch-1.4
[build #812 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/812/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/812//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/812//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/812//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Should run a 'mvn install' at the end of hadoop check in pre commit job
> ---
>
> Key: HBASE-22462
> URL: https://issues.apache.org/jira/browse/HBASE-22462
> Project: HBase
> Issue Type: Bug
> Components: build
>Reporter: Duo Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 1.4.10, 2.3.0, 2.0.6, 2.1.5, 2.2.1
>
> Attachments: HBASE-22462.master.001.patch,
> HBASE-22462.master.002.patch, HBASE-22462.master.003.patch,
> HBASE-22462.master.004.patch, HBASE-22462.master.addendum.patch,
> HBASE-22462.master.test.patch
>
>
> Now for branch-2.x, we will build with hadoop 3.x in the hadoop check stage,
> so in later unit check, if we run from a sub module, then the other hbase
> modules we depend on will depend on hadoop 3.x while the module we build will
> depend on 2.x, this will cause the following problem
> {noformat}
> 2019-05-23 04:47:41,156 WARN [RS_CLOSE_META-regionserver/b001f91a596c:0-0]
> handler.AssignRegionHandler(157): Fatal error occurred while opening region
> hbase:meta,,1.1588230740, aborting...
> java.lang.IncompatibleClassChangeError: Found class
> org.apache.hadoop.hdfs.protocol.HdfsFileStatus, but interface was expected
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createOutput(FanOutOneBlockAsyncDFSOutputHelper.java:496)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$400(FanOutOneBlockAsyncDFSOutputHelper.java:116)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$8.doCall(FanOutOneBlockAsyncDFSOutputHelper.java:576)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$8.doCall(FanOutOneBlockAsyncDFSOutputHelper.java:571)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createOutput(FanOutOneBlockAsyncDFSOutputHelper.java:584)
> at
> org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:51)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:169)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:166)
> at
> org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:105)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createAsyncWriter(AsyncFSWAL.java:664)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:670)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:128)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:832)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:538)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:479)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:156)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:61)
> at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:293)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getWAL(HRegionServer.java:2170)
> at
> org.apache.hadoop.hbase.regionserver.handler.AssignRegionHandler.process(AssignRegionHandler.java:133)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
> at
>
[jira] [Updated] (HBASE-22411) Refactor codes of moving reigons in RSGroup
[ https://issues.apache.org/jira/browse/HBASE-22411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaolin Ha updated HBASE-22411: --- Attachment: HBASE-22411.master.011.patch > Refactor codes of moving reigons in RSGroup > --- > > Key: HBASE-22411 > URL: https://issues.apache.org/jira/browse/HBASE-22411 > Project: HBase > Issue Type: Improvement > Components: rsgroup >Affects Versions: 2.2.0 >Reporter: Xiaolin Ha >Assignee: Xiaolin Ha >Priority: Major > Attachments: HBASE-22411.branch-2.2.001.patch, > HBASE-22411.master.001.patch, HBASE-22411.master.002.patch, > HBASE-22411.master.003.patch, HBASE-22411.master.004.patch, > HBASE-22411.master.005.patch, HBASE-22411.master.006.patch, > HBASE-22411.master.007.patch, HBASE-22411.master.007.patch, > HBASE-22411.master.007.patch, HBASE-22411.master.007.patch, > HBASE-22411.master.008.patch, HBASE-22411.master.009.patch, > HBASE-22411.master.010.patch, HBASE-22411.master.011.patch > > > Essentially RSGroup managed regions. Organizing tables or servers' RSGroups > is to move relevant regions. Codes of moving regions can be refactored. > So that some problems caused by moving regions can be fixed elegantly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-22468) There is no ARM CI for HBASE
zhao bo created HBASE-22468: --- Summary: There is no ARM CI for HBASE Key: HBASE-22468 URL: https://issues.apache.org/jira/browse/HBASE-22468 Project: HBase Issue Type: Improvement Reporter: zhao bo Currently, the CI of HBASE(in github [https://github.com/apache/hbase]) is handled by Apache Jenkins([https://builds.apache.org|https://builds.apache.org/]) . But most jobs are running under X86 hw ARCH, the arm ARCH is missing, even though there is already a arm server([https://builds.apache.org/computer/arm1/]). This leads an problem that we don't have a good way to test every pull request that the proposal can be deployed, compiled or tested success on ARM. We should add a CI system that support ARM ARCH for more resources and less human resource invest. Using it, HBASE can officially support arm release in the future. Here I'd like to introduce OpenLab to the community. [OpenLab |[https://openlabtesting.org/]] is a open source CI system that can test any open source software on either x86 or arm ARCH, it's mainly used by github projects. Now some [projects|[https://github.com/theopenlab/openlab-zuul-jobs/blob/master/zuul.d/jobs.yaml]] has integrated it already. Such as containerd (a CNCF project), terraform and so on. OpenLab uses the open source CI software [Zuul |[https://github.com/openstack-infra/zuul]] for CI system. Zuul is used by OpenStack community as well. integrating with OpenLab is quite easy using its github app. All config info is open source as well. If HBASE team has interested with it, I can help for the integration. Thank you. Other words: I have test to build HBASE on centos(arm ARCH). It can be built and runned successful after install protobuf manually, especially version 2.5.0, which didn't provider a arm release. That means support arm is viable and good to do. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22462) Should run a 'mvn install' at the end of hadoop check in pre commit job
[
https://issues.apache.org/jira/browse/HBASE-22462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847345#comment-16847345
]
Hudson commented on HBASE-22462:
Results for branch branch-2
[build #1914 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1914/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1914//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1914//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1914//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1914//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Should run a 'mvn install' at the end of hadoop check in pre commit job
> ---
>
> Key: HBASE-22462
> URL: https://issues.apache.org/jira/browse/HBASE-22462
> Project: HBase
> Issue Type: Bug
> Components: build
>Reporter: Duo Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 1.4.10, 2.3.0, 2.0.6, 2.1.5, 2.2.1
>
> Attachments: HBASE-22462.master.001.patch,
> HBASE-22462.master.002.patch, HBASE-22462.master.003.patch,
> HBASE-22462.master.004.patch, HBASE-22462.master.addendum.patch,
> HBASE-22462.master.test.patch
>
>
> Now for branch-2.x, we will build with hadoop 3.x in the hadoop check stage,
> so in later unit check, if we run from a sub module, then the other hbase
> modules we depend on will depend on hadoop 3.x while the module we build will
> depend on 2.x, this will cause the following problem
> {noformat}
> 2019-05-23 04:47:41,156 WARN [RS_CLOSE_META-regionserver/b001f91a596c:0-0]
> handler.AssignRegionHandler(157): Fatal error occurred while opening region
> hbase:meta,,1.1588230740, aborting...
> java.lang.IncompatibleClassChangeError: Found class
> org.apache.hadoop.hdfs.protocol.HdfsFileStatus, but interface was expected
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createOutput(FanOutOneBlockAsyncDFSOutputHelper.java:496)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$400(FanOutOneBlockAsyncDFSOutputHelper.java:116)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$8.doCall(FanOutOneBlockAsyncDFSOutputHelper.java:576)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$8.doCall(FanOutOneBlockAsyncDFSOutputHelper.java:571)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createOutput(FanOutOneBlockAsyncDFSOutputHelper.java:584)
> at
> org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:51)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:169)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:166)
> at
> org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:105)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createAsyncWriter(AsyncFSWAL.java:664)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:670)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:128)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:832)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:538)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:479)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:156)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:61)
> at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:293)
> at
>
[jira] [Commented] (HBASE-19663) site build fails complaining "javadoc: error - class file for javax.annotation.meta.TypeQualifierNickname not found"
[
https://issues.apache.org/jira/browse/HBASE-19663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847331#comment-16847331
]
Hudson commented on HBASE-19663:
Results for branch HBASE-22466
[build #7 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22466/7/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22466/7//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22466/7//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22466/7//JDK8_Nightly_Build_Report_(Hadoop3)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> site build fails complaining "javadoc: error - class file for
> javax.annotation.meta.TypeQualifierNickname not found"
>
>
> Key: HBASE-19663
> URL: https://issues.apache.org/jira/browse/HBASE-19663
> Project: HBase
> Issue Type: Bug
> Components: website
>Reporter: stack
>Assignee: stack
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: script.sh
>
>
> Cryptic failure trying to build beta-1 RC. Fails like this:
> {code}
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 03:54 min
> [INFO] Finished at: 2017-12-29T01:13:15-08:00
> [INFO] Final Memory: 381M/9165M
> [INFO]
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-site-plugin:3.4:site (default-site) on project
> hbase: Error generating maven-javadoc-plugin:2.10.3:aggregate:
> [ERROR] Exit code: 1 - warning: unknown enum constant When.ALWAYS
> [ERROR] reason: class file for javax.annotation.meta.When not found
> [ERROR] warning: unknown enum constant When.UNKNOWN
> [ERROR] warning: unknown enum constant When.MAYBE
> [ERROR]
> /home/stack/hbase.git/hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java:762:
> warning - Tag @link: malformed: "#matchingRows(Cell, byte[]))"
> [ERROR]
> /home/stack/hbase.git/hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java:762:
> warning - Tag @link: reference not found: #matchingRows(Cell, byte[]))
> [ERROR]
> /home/stack/hbase.git/hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java:762:
> warning - Tag @link: reference not found: #matchingRows(Cell, byte[]))
> [ERROR] javadoc: warning - Class javax.annotation.Nonnull not found.
> [ERROR] javadoc: error - class file for
> javax.annotation.meta.TypeQualifierNickname not found
> [ERROR]
> [ERROR] Command line was: /home/stack/bin/jdk1.8.0_151/jre/../bin/javadoc
> -J-Xmx2G @options @packages
> [ERROR]
> [ERROR] Refer to the generated Javadoc files in
> '/home/stack/hbase.git/target/site/apidocs' dir.
> [ERROR] -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
> {code}
> javax.annotation.meta.TypeQualifierNickname is out of jsr305 but we don't
> include this anywhere according to mvn dependency.
> Happens building the User API both test and main.
> Excluding these lines gets us passing again:
> {code}
> 3511
> 3512
> org.apache.yetus.audience.tools.IncludePublicAnnotationsStandardDoclet
> 3513
> 3514
> 3515 org.apache.yetus
> 3516 audience-annotations
> 3517 ${audience-annotations.version}
> 3518
> + 3519 true
> {code}
> Tried upgrading to newer mvn site (ours is three years old) but that a
> different set of problems.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22462) Should run a 'mvn install' at the end of hadoop check in pre commit job
[
https://issues.apache.org/jira/browse/HBASE-22462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847323#comment-16847323
]
Hudson commented on HBASE-22462:
Results for branch branch-2.0
[build #1614 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1614/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1614//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1614//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1614//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Should run a 'mvn install' at the end of hadoop check in pre commit job
> ---
>
> Key: HBASE-22462
> URL: https://issues.apache.org/jira/browse/HBASE-22462
> Project: HBase
> Issue Type: Bug
> Components: build
>Reporter: Duo Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 1.4.10, 2.3.0, 2.0.6, 2.1.5, 2.2.1
>
> Attachments: HBASE-22462.master.001.patch,
> HBASE-22462.master.002.patch, HBASE-22462.master.003.patch,
> HBASE-22462.master.004.patch, HBASE-22462.master.addendum.patch,
> HBASE-22462.master.test.patch
>
>
> Now for branch-2.x, we will build with hadoop 3.x in the hadoop check stage,
> so in later unit check, if we run from a sub module, then the other hbase
> modules we depend on will depend on hadoop 3.x while the module we build will
> depend on 2.x, this will cause the following problem
> {noformat}
> 2019-05-23 04:47:41,156 WARN [RS_CLOSE_META-regionserver/b001f91a596c:0-0]
> handler.AssignRegionHandler(157): Fatal error occurred while opening region
> hbase:meta,,1.1588230740, aborting...
> java.lang.IncompatibleClassChangeError: Found class
> org.apache.hadoop.hdfs.protocol.HdfsFileStatus, but interface was expected
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createOutput(FanOutOneBlockAsyncDFSOutputHelper.java:496)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$400(FanOutOneBlockAsyncDFSOutputHelper.java:116)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$8.doCall(FanOutOneBlockAsyncDFSOutputHelper.java:576)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$8.doCall(FanOutOneBlockAsyncDFSOutputHelper.java:571)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createOutput(FanOutOneBlockAsyncDFSOutputHelper.java:584)
> at
> org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:51)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:169)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:166)
> at
> org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:105)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createAsyncWriter(AsyncFSWAL.java:664)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:670)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:128)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:832)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:538)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:479)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:156)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:61)
> at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:293)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getWAL(HRegionServer.java:2170)
> at
> org.apache.hadoop.hbase.regionserver.handler.AssignRegionHandler.process(AssignRegionHandler.java:133)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
> at
>
[jira] [Updated] (HBASE-22463) Some paths in HFileScannerImpl did not consider block#release which will exhaust the ByteBuffAllocator
[ https://issues.apache.org/jira/browse/HBASE-22463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-22463: - Attachment: HBASE-22463.HBASE-21879.v1.patch > Some paths in HFileScannerImpl did not consider block#release which will > exhaust the ByteBuffAllocator > > > Key: HBASE-22463 > URL: https://issues.apache.org/jira/browse/HBASE-22463 > Project: HBase > Issue Type: Sub-task >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Attachments: HBASE-22463.HBASE-21879.v1.patch, > HBASE-22463.HBASE-21879.v1.patch, HBASE-22463.HBASE-21879.v1.patch > > > When I debug the issue HBASE-22422, I observed that the > ByteBuffAllocator#usedBufCount will was always increasing and all direct > ByteBuffers would be exhausted, which lead to may heap allocation happen. > The comment here [1] is also related to this problem. > Check the code path, the HFileScannerImpl is the biggest suspect, so create > issue to address this. > 1. > https://issues.apache.org/jira/browse/HBASE-22387?focusedCommentId=16838446=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16838446 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22462) Should run a 'mvn install' at the end of hadoop check in pre commit job
[
https://issues.apache.org/jira/browse/HBASE-22462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847305#comment-16847305
]
Hudson commented on HBASE-22462:
Results for branch branch-1
[build #849 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/849/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/849//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/849//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/849//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> Should run a 'mvn install' at the end of hadoop check in pre commit job
> ---
>
> Key: HBASE-22462
> URL: https://issues.apache.org/jira/browse/HBASE-22462
> Project: HBase
> Issue Type: Bug
> Components: build
>Reporter: Duo Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 1.4.10, 2.3.0, 2.0.6, 2.1.5, 2.2.1
>
> Attachments: HBASE-22462.master.001.patch,
> HBASE-22462.master.002.patch, HBASE-22462.master.003.patch,
> HBASE-22462.master.004.patch, HBASE-22462.master.addendum.patch,
> HBASE-22462.master.test.patch
>
>
> Now for branch-2.x, we will build with hadoop 3.x in the hadoop check stage,
> so in later unit check, if we run from a sub module, then the other hbase
> modules we depend on will depend on hadoop 3.x while the module we build will
> depend on 2.x, this will cause the following problem
> {noformat}
> 2019-05-23 04:47:41,156 WARN [RS_CLOSE_META-regionserver/b001f91a596c:0-0]
> handler.AssignRegionHandler(157): Fatal error occurred while opening region
> hbase:meta,,1.1588230740, aborting...
> java.lang.IncompatibleClassChangeError: Found class
> org.apache.hadoop.hdfs.protocol.HdfsFileStatus, but interface was expected
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createOutput(FanOutOneBlockAsyncDFSOutputHelper.java:496)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$400(FanOutOneBlockAsyncDFSOutputHelper.java:116)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$8.doCall(FanOutOneBlockAsyncDFSOutputHelper.java:576)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$8.doCall(FanOutOneBlockAsyncDFSOutputHelper.java:571)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createOutput(FanOutOneBlockAsyncDFSOutputHelper.java:584)
> at
> org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:51)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:169)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:166)
> at
> org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:105)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createAsyncWriter(AsyncFSWAL.java:664)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:670)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:128)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:832)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:538)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:479)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:156)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:61)
> at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:293)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getWAL(HRegionServer.java:2170)
> at
> org.apache.hadoop.hbase.regionserver.handler.AssignRegionHandler.process(AssignRegionHandler.java:133)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
> at
>
[jira] [Commented] (HBASE-22454) refactor WALSplitter
[
https://issues.apache.org/jira/browse/HBASE-22454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847301#comment-16847301
]
HBase QA commented on HBASE-22454:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
46s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 11 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
21s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
17s{color} | {color:green} hbase-server: The patch generated 0 new + 512
unchanged - 30 fixed = 512 total (was 542) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
28s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
16m 24s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.0.3 3.1.2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}238m 50s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
30s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}289m 45s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.procedure.TestSCPWithReplicas |
| | hadoop.hbase.client.TestAdmin1 |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce base:
https://builds.apache.org/job/PreCommit-HBASE-Build/410/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-22454 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12969576/HBASE-22454.master.003.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 7660b1ab83b1 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / de579cd243 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.11 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/410/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/410/testReport/
[jira] [Commented] (HBASE-22411) Refactor codes of moving reigons in RSGroup
[
https://issues.apache.org/jira/browse/HBASE-22411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847296#comment-16847296
]
HBase QA commented on HBASE-22411:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
39s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
27s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
37s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 39s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.0.3 3.1.2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
57s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 54m 35s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce base:
https://builds.apache.org/job/PreCommit-HBASE-Build/413/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-22411 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12969588/HBASE-22411.master.010.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 6b3711c64223 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 96d2aabfad |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.11 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/413/testReport/ |
| Max. process+thread count | 4428 (vs. ulimit of 1) |
| modules | C: hbase-rsgroup U: hbase-rsgroup |
| Console output |
[jira] [Commented] (HBASE-22462) Should run a 'mvn install' at the end of hadoop check in pre commit job
[
https://issues.apache.org/jira/browse/HBASE-22462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847293#comment-16847293
]
Hudson commented on HBASE-22462:
Results for branch master
[build #1032 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1032/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1032//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1032//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1032//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://builds.apache.org/job/HBase%20Nightly/job/master/1032//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Should run a 'mvn install' at the end of hadoop check in pre commit job
> ---
>
> Key: HBASE-22462
> URL: https://issues.apache.org/jira/browse/HBASE-22462
> Project: HBase
> Issue Type: Bug
> Components: build
>Reporter: Duo Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 1.4.10, 2.3.0, 2.0.6, 2.1.5, 2.2.1
>
> Attachments: HBASE-22462.master.001.patch,
> HBASE-22462.master.002.patch, HBASE-22462.master.003.patch,
> HBASE-22462.master.004.patch, HBASE-22462.master.addendum.patch,
> HBASE-22462.master.test.patch
>
>
> Now for branch-2.x, we will build with hadoop 3.x in the hadoop check stage,
> so in later unit check, if we run from a sub module, then the other hbase
> modules we depend on will depend on hadoop 3.x while the module we build will
> depend on 2.x, this will cause the following problem
> {noformat}
> 2019-05-23 04:47:41,156 WARN [RS_CLOSE_META-regionserver/b001f91a596c:0-0]
> handler.AssignRegionHandler(157): Fatal error occurred while opening region
> hbase:meta,,1.1588230740, aborting...
> java.lang.IncompatibleClassChangeError: Found class
> org.apache.hadoop.hdfs.protocol.HdfsFileStatus, but interface was expected
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createOutput(FanOutOneBlockAsyncDFSOutputHelper.java:496)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$400(FanOutOneBlockAsyncDFSOutputHelper.java:116)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$8.doCall(FanOutOneBlockAsyncDFSOutputHelper.java:576)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$8.doCall(FanOutOneBlockAsyncDFSOutputHelper.java:571)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createOutput(FanOutOneBlockAsyncDFSOutputHelper.java:584)
> at
> org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:51)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:169)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:166)
> at
> org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:105)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createAsyncWriter(AsyncFSWAL.java:664)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:670)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:128)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:832)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:538)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:479)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:156)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:61)
> at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:293)
> at
>
[jira] [Commented] (HBASE-22403) Balance in RSGroup should consider throttling and a failure affects the whole
[ https://issues.apache.org/jira/browse/HBASE-22403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847273#comment-16847273 ] Xiaolin Ha commented on HBASE-22403: [~zghaobac]，[~xucang]Could you review the diff codes? > Balance in RSGroup should consider throttling and a failure affects the whole > - > > Key: HBASE-22403 > URL: https://issues.apache.org/jira/browse/HBASE-22403 > Project: HBase > Issue Type: Improvement > Components: rsgroup >Affects Versions: 2.2.0 >Reporter: Xiaolin Ha >Assignee: Xiaolin Ha >Priority: Major > Attachments: HBASE-22403.branch-2.2.001.patch, > HBASE-22403.branch-2.2.002.patch, HBASE-22403.master.001.patch, > HBASE-22403.master.002.patch > > > balanceRSGroup(groupName) excutes region move plans concurrently, which will > affect the availability of relevant tables. And a plan fails will cause the > whole balance plan abort. > As mentioned in master balance issues, HBASE-17178, HBASE-21260 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-17165) Add retry to LoadIncrementalHFiles tool
[ https://issues.apache.org/jira/browse/HBASE-17165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847272#comment-16847272 ] Y. SREENIVASULU REDDY commented on HBASE-17165: --- [~stack] and [~grimesmi] There is any plan to push this changes in branch-1 also. As discussed in the above to make a subtask. https://issues.apache.org/jira/browse/HBASE-17165?focusedCommentId=16397730=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16397730 > Add retry to LoadIncrementalHFiles tool > --- > > Key: HBASE-17165 > URL: https://issues.apache.org/jira/browse/HBASE-17165 > Project: HBase > Issue Type: Improvement > Components: HFile, tooling >Affects Versions: 1.2.3, 2.0.0 >Reporter: Mike Grimes >Assignee: Mike Grimes >Priority: Critical > Labels: s3 > Fix For: 2.0.0 > > Attachments: HBASE-17165.branch-1.001.patch, > HBASE-17165.branch-1.001.patch, HBASE-17165.branch-1.002.patch, > HBASE-17165.branch-1.002.patch, HBASE-17165.branch-1.2.001.patch, > HBASE-17165.branch-1.2.002.patch, HBASE-17165.branch-1.2.003.patch, > HBASE-17165.branch-1.2.004.patch, HBASE-17165.master.001.patch, > HBASE-17165.master.002.patch, HBASE-17165.master.003.patch, > HBASE-17165.master.004.patch, HBASE-17165.master.004.patch > > Original Estimate: 0h > Remaining Estimate: 0h > > As using the LoadIncrementalHFiles tool with S3 as the filesystem is prone to > failing due to FileNotFoundExceptions due to inconsistency, simple, > configurable retry logic was added. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22346) scanner priorities/deadline units are invalid for non-huge scanners
[
https://issues.apache.org/jira/browse/HBASE-22346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847269#comment-16847269
]
Hudson commented on HBASE-22346:
Results for branch HBASE-22346
[build #19 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22346/19/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22346/19//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22346/19//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22346/19//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> scanner priorities/deadline units are invalid for non-huge scanners
> ---
>
> Key: HBASE-22346
> URL: https://issues.apache.org/jira/browse/HBASE-22346
> Project: HBase
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HBASE-22346.01.patch, HBASE-22346.patch
>
>
> I was looking at using the priority (deadline) queue for scanner requests;
> what I see is that AnnotationReadingPriorityFunction, the only impl of the
> deadline function available, implements getDeadline as sqrt of the number of
> next() calls, from HBASE-10993.
> However, CallPriorityComparator.compare, its only caller, adds that
> "deadline" value to the callA.getReceiveTime() in milliseconds...
> That results in some sort of a meaningless value that I assume only make
> sense "by coincidence" for telling apart broad and specific classes of
> scanners... in practice next calls must be in the 1000s before it becomes
> meaningful vs small differences in ReceivedTime
> When there's contention from many scanners, e.g. small scanners for meta, or
> just users creating tons of scanners to the point where requests queue up,
> the actual deadline is not accounted for and the priority function itself is
> meaningless... In fact as queueing increases, it becomes worse because
> receivedtime differences grow.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22403) Balance in RSGroup should consider throttling and a failure affects the whole
[
https://issues.apache.org/jira/browse/HBASE-22403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847265#comment-16847265
]
HBase QA commented on HBASE-22403:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
26s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
36s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
42s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 31s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.0.3 3.1.2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}130m
54s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
18s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
37s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}192m 23s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce base:
https://builds.apache.org/job/PreCommit-HBASE-Build/411/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-22403 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12969577/HBASE-22403.master.002.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 01472b038851 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / de579cd243 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java |
[jira] [Updated] (HBASE-22411) Refactor codes of moving reigons in RSGroup
[ https://issues.apache.org/jira/browse/HBASE-22411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaolin Ha updated HBASE-22411: --- Attachment: HBASE-22411.master.010.patch > Refactor codes of moving reigons in RSGroup > --- > > Key: HBASE-22411 > URL: https://issues.apache.org/jira/browse/HBASE-22411 > Project: HBase > Issue Type: Improvement > Components: rsgroup >Affects Versions: 2.2.0 >Reporter: Xiaolin Ha >Assignee: Xiaolin Ha >Priority: Major > Attachments: HBASE-22411.branch-2.2.001.patch, > HBASE-22411.master.001.patch, HBASE-22411.master.002.patch, > HBASE-22411.master.003.patch, HBASE-22411.master.004.patch, > HBASE-22411.master.005.patch, HBASE-22411.master.006.patch, > HBASE-22411.master.007.patch, HBASE-22411.master.007.patch, > HBASE-22411.master.007.patch, HBASE-22411.master.007.patch, > HBASE-22411.master.008.patch, HBASE-22411.master.009.patch, > HBASE-22411.master.010.patch > > > Essentially RSGroup managed regions. Organizing tables or servers' RSGroups > is to move relevant regions. Codes of moving regions can be refactored. > So that some problems caused by moving regions can be fixed elegantly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22411) Refactor codes of moving reigons in RSGroup
[ https://issues.apache.org/jira/browse/HBASE-22411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847262#comment-16847262 ] Xiaolin Ha commented on HBASE-22411: OK，I‘ll change it back. > Refactor codes of moving reigons in RSGroup > --- > > Key: HBASE-22411 > URL: https://issues.apache.org/jira/browse/HBASE-22411 > Project: HBase > Issue Type: Improvement > Components: rsgroup >Affects Versions: 2.2.0 >Reporter: Xiaolin Ha >Assignee: Xiaolin Ha >Priority: Major > Attachments: HBASE-22411.branch-2.2.001.patch, > HBASE-22411.master.001.patch, HBASE-22411.master.002.patch, > HBASE-22411.master.003.patch, HBASE-22411.master.004.patch, > HBASE-22411.master.005.patch, HBASE-22411.master.006.patch, > HBASE-22411.master.007.patch, HBASE-22411.master.007.patch, > HBASE-22411.master.007.patch, HBASE-22411.master.007.patch, > HBASE-22411.master.008.patch, HBASE-22411.master.009.patch > > > Essentially RSGroup managed regions. Organizing tables or servers' RSGroups > is to move relevant regions. Codes of moving regions can be refactored. > So that some problems caused by moving regions can be fixed elegantly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
