[jira] [Resolved] (HBASE-18163) rubocop and ruby-lint are not available
[ https://issues.apache.org/jira/browse/HBASE-18163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey resolved HBASE-18163. - Resolution: Duplicate > rubocop and ruby-lint are not available > --- > > Key: HBASE-18163 > URL: https://issues.apache.org/jira/browse/HBASE-18163 > Project: HBase > Issue Type: Bug > Components: build >Reporter: Mike Drob > Labels: beginner > > From the yetus output: > {noformat} > executable for 'rubocop' was not specified. > executable for 'Ruby-lint' was not specified. > {noformat} > which results in this during the build > | {color:blue}0{color} | {color:blue} rubocop {color} | {color:blue} 0m 11s > {color} | {color:blue} rubocop was not available. {color} | > | {color:blue}0{color} | {color:blue} ruby-lint {color} | {color:blue} 0m 11s > {color} | {color:blue} Ruby-lint was not available. {color} | > We should make those available. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18427) minor cleanup around AssignmentManager
Umesh Agashe created HBASE-18427: Summary: minor cleanup around AssignmentManager Key: HBASE-18427 URL: https://issues.apache.org/jira/browse/HBASE-18427 Project: HBase Issue Type: Bug Components: amv2 Reporter: Umesh Agashe Assignee: Umesh Agashe Priority: Minor Minor cleanup around AssignmentManager. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18426) nightly job should use independent stages to check supported jdks
Sean Busbey created HBASE-18426: --- Summary: nightly job should use independent stages to check supported jdks Key: HBASE-18426 URL: https://issues.apache.org/jira/browse/HBASE-18426 Project: HBase Issue Type: Improvement Components: community, test Reporter: Sean Busbey Assignee: Sean Busbey Priority: Critical follow on from HBASE-18147 to handle the lack of multijdk support for unit tests. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18425) Fix TestMasterFailover
Vladimir Rodionov created HBASE-18425: - Summary: Fix TestMasterFailover Key: HBASE-18425 URL: https://issues.apache.org/jira/browse/HBASE-18425 Project: HBase Issue Type: Sub-task Reporter: Vladimir Rodionov Assignee: Vladimir Rodionov -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18424) Fix TestAsyncTableGetMultiThreaded
Vladimir Rodionov created HBASE-18424: - Summary: Fix TestAsyncTableGetMultiThreaded Key: HBASE-18424 URL: https://issues.apache.org/jira/browse/HBASE-18424 Project: HBase Issue Type: Sub-task Reporter: Vladimir Rodionov Assignee: Vladimir Rodionov -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18423) Fix TestMetaWithReplicas
Vladimir Rodionov created HBASE-18423: - Summary: Fix TestMetaWithReplicas Key: HBASE-18423 URL: https://issues.apache.org/jira/browse/HBASE-18423 Project: HBase Issue Type: Sub-task Reporter: Vladimir Rodionov Assignee: Vladimir Rodionov -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18422) Fix TestRegionRebalancing
Vladimir Rodionov created HBASE-18422: - Summary: Fix TestRegionRebalancing Key: HBASE-18422 URL: https://issues.apache.org/jira/browse/HBASE-18422 Project: HBase Issue Type: Sub-task Reporter: Vladimir Rodionov Assignee: Vladimir Rodionov -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: [ANNOUNCE] Asynchronous client is available in HBase 2.0
Very sweet Guanghao. Thanks for all the hard work (you, Duo and all who contributed to the async work). When you think we should move the sync client to be on top of the async chassis? Thanks, St.Ack P.S. It looks like you updated its status in this doc, https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9iEu_ktczrlKHK8N4SZzs/edit#, thanks. On Thu, Jul 20, 2017 at 4:51 AM, Guanghao Zhangwrote: > Dear all, > > Asynchronous client aims to provide the ability to access HBase > asynchronously. You can obtain an AsyncConnection from ConnectionFactory, > and then get an asynchronous table instance (for DML operations) or an > asynchronous admin instance (for DDL operations) from it to access HBase. > For the asynchronous table or admin, most methods have the same meaning > with the old Table or Admin interface, expect that the return value is > wrapped with a CompletableFuture (java 8 or higher is required) usually. > You can get more details from the documentation[1][2]. > > There are two examples about the asynchronous client. AsyncClientExample[3] > is a simple example to show you how to use AsyncTable. HttpProxyExample[4] > is an example for advance user to show you how to use RawAsyncTable to > write a fully asynchronous HTTP proxy server. Welcome to try the > asynchronous client in your project. > > We tested the performance of asynchronous client by > the PerformanceEvaluation tool. And it showed that async client has almost > same latency with sync client[5]. Notice: when we did performance test for > asynchronous client. We found some performance issue when use jdk 1.8_111. > And it was fixed when use jdk 1.8_131 [6]. So suggest use a bigger version > than jdk8_131 when you use the asynchronous client. > > The major umbrella issues include: > HBASE-16432 Revisit the asynchronous ipc implementation[7] > HBASE-16833 Implement asynchronous hbase client based on HBASE-15921[8] > HBASE-17359 Implement async admin[9] > > There is also an issue HBASE-17856 to track the further improvements for > asynchronous hbase client[10]. > > Finally, thanks Duo Zhang and Zheng Hu who worked with me together to > implement the asynchronous client. > > Best Regards, > Guanghao > > [1] https://hbase.apache.org/book.html#async.client > [2] https://hbase.apache.org/book.html#async.admin > [3] > https://github.com/apache/hbase/blob/master/hbase- > examples/src/main/java/org/apache/hadoop/hbase/client/ > example/AsyncClientExample.java > [4] > https://github.com/apache/hbase/blob/master/hbase- > examples/src/main/java/org/apache/hadoop/hbase/client/ > example/HttpProxyExample.java > [5] https://www.slideshare.net/HBaseCon/hbase-at-xiaomi > [6] > http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/687fd7c7986d/src/share/ > classes/java/util/concurrent/CompletableFuture.java > [7] https://issues.apache.org/jira/browse/HBASE-16432 > [8] https://issues.apache.org/jira/browse/HBASE-16833 > [9] https://issues.apache.org/jira/browse/HBASE-17359 > [10] https://issues.apache.org/jira/browse/HBASE-17856 >
Re: Ident String
On Thu, Jul 20, 2017 at 9:20 PM, Lars Georgewrote: > Ah ok, I missed that HBASE_IDENT_STR is used to personalize the log and > pid file names. Hadoop does the same, so that makes sense. But the > -Dhbase.id.str is not used, neither is it in Hadoop. No worries, just > wanted to see if anyone had an idea if that was ever used. Does not seem to > be the case. > > Should we remove it then boss? St.Ack > Cheers, > Lars > > Sent from my iPad > > > On 19. Jul 2017, at 22:42, Lars George wrote: > > > > It dates back to Hadoop: > > > > https://github.com/apache/hbase/commit/24b065cc91f7bcdab25fc363469965 > 7ac2f27104 > > > > See this https://github.com/apache/hadoop/blame/trunk/hadoop- > common-project/hadoop-common/src/main/conf/hadoop-env.sh#L202 > > > > It is used there for logs (according to the comment, haven't checked if > it really does). Are we doing that? Will check... > > > > > > Sent from my iPad > > > >> On 19. Jul 2017, at 16:43, Stack wrote: > >> > >> It shows when you do a long processing listing. It is a convention from > >> hadoop. It does similar. Here's a few example snippets Lars: > >> > >> For hbase master process listing, you'll find this in the long line...: > >> -Dproc_master > >> > >> Etc: > >> > >> -Dproc_zookeeper > >> > >> -Dproc_resourcemanager > >> > >> St.Ack > >> > >> > >>> On Fri, Jul 14, 2017 at 11:55 AM, Lars George > wrote: > >>> > >>> Hi, > >>> > >>> Was coming across `HBASE_IDENT_STR` (in hbase-env.sh) which sets > >>> `hbase.id.str` as a command line parameter to the daemon. But I cannot > >>> find where that is used. Could someone point me in the right > >>> direction? > >>> > >>> Cheers, > >>> Lars > >>> >
[jira] [Created] (HBASE-18421) update hadoop prerequisites docs to call out 2.8.1
Sean Busbey created HBASE-18421: --- Summary: update hadoop prerequisites docs to call out 2.8.1 Key: HBASE-18421 URL: https://issues.apache.org/jira/browse/HBASE-18421 Project: HBase Issue Type: Task Components: community, dependencies, documentation Reporter: Sean Busbey The Hadoop PMC has release 2.8.1 with the same "not ready for production" caveat as 2.8.0 ([ref announce email|https://lists.apache.org/thread.html/06609e491325c16debbdef4fbe8cecc6160b8909c06683c4805ad4af@%3Cgeneral.hadoop.apache.org%3E]) We should update our docs proactively. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18420) Some methods of Admin don't use ColumnFamilyDescriptor
Chia-Ping Tsai created HBASE-18420: -- Summary: Some methods of Admin don't use ColumnFamilyDescriptor Key: HBASE-18420 URL: https://issues.apache.org/jira/browse/HBASE-18420 Project: HBase Issue Type: Bug Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: Ident String
Ah ok, I missed that HBASE_IDENT_STR is used to personalize the log and pid file names. Hadoop does the same, so that makes sense. But the -Dhbase.id.str is not used, neither is it in Hadoop. No worries, just wanted to see if anyone had an idea if that was ever used. Does not seem to be the case. Cheers, Lars Sent from my iPad > On 19. Jul 2017, at 22:42, Lars Georgewrote: > > It dates back to Hadoop: > > https://github.com/apache/hbase/commit/24b065cc91f7bcdab25fc3634699657ac2f27104 > > See this > https://github.com/apache/hadoop/blame/trunk/hadoop-common-project/hadoop-common/src/main/conf/hadoop-env.sh#L202 > > It is used there for logs (according to the comment, haven't checked if it > really does). Are we doing that? Will check... > > > Sent from my iPad > >> On 19. Jul 2017, at 16:43, Stack wrote: >> >> It shows when you do a long processing listing. It is a convention from >> hadoop. It does similar. Here's a few example snippets Lars: >> >> For hbase master process listing, you'll find this in the long line...: >> -Dproc_master >> >> Etc: >> >> -Dproc_zookeeper >> >> -Dproc_resourcemanager >> >> St.Ack >> >> >>> On Fri, Jul 14, 2017 at 11:55 AM, Lars George wrote: >>> >>> Hi, >>> >>> Was coming across `HBASE_IDENT_STR` (in hbase-env.sh) which sets >>> `hbase.id.str` as a command line parameter to the daemon. But I cannot >>> find where that is used. Could someone point me in the right >>> direction? >>> >>> Cheers, >>> Lars >>>
[jira] [Created] (HBASE-18419) IntegrationTestIngestWithMob fails in standalone mode
Mike Drob created HBASE-18419: - Summary: IntegrationTestIngestWithMob fails in standalone mode Key: HBASE-18419 URL: https://issues.apache.org/jira/browse/HBASE-18419 Project: HBase Issue Type: Bug Components: integration tests Affects Versions: 2.0.0-alpha-1 Reporter: Mike Drob Priority: Critical When attempting to run ITIWM against a cluster I get the following error: {noformat} java.lang.UnsupportedOperationException: HColumnDescriptor is read-only at org.apache.hadoop.hbase.client.ImmutableHColumnDescriptor.getDelegateeForModification(ImmutableHColumnDescriptor.java:44) at org.apache.hadoop.hbase.HColumnDescriptor.setMobEnabled(HColumnDescriptor.java:735) at org.apache.hadoop.hbase.IntegrationTestIngestWithMOB.initTable(IntegrationTestIngestWithMOB.java:122) at org.apache.hadoop.hbase.IntegrationTestIngest.setUpCluster(IntegrationTestIngest.java:92) at org.apache.hadoop.hbase.IntegrationTestBase.setUp(IntegrationTestBase.java:148) at org.apache.hadoop.hbase.IntegrationTestBase.doWork(IntegrationTestBase.java:131) at org.apache.hadoop.hbase.util.AbstractHBaseTool.run(AbstractHBaseTool.java:154) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at org.apache.hadoop.hbase.IntegrationTestIngestWithMOB.main(IntegrationTestIngestWithMOB.java:153) {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18418) Remove apache_hbase_topology from dev-support
Dima Spivak created HBASE-18418: --- Summary: Remove apache_hbase_topology from dev-support Key: HBASE-18418 URL: https://issues.apache.org/jira/browse/HBASE-18418 Project: HBase Issue Type: Task Reporter: Dima Spivak Assignee: Dima Spivak Priority: Minor As [~misty] noted on dev@, we still keep a folder containing the Apache HBase topology for [clusterdock|https://github.com/clusterdock/framework] in dev-support. Since the {{clusterdock}} org on GitHub will likely always have a more up-to-date iteration of this code (and since I plan on showing it some love there in the next few weeks), we'd probably be better off removing it from HBase. Any objections? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: Dockerfiles under dev-support
As a general best practice, it's best to call it Dockerfile and I'd argue for keeping it that way and just updating documentation/READMEs/comments in the files to explain any ambiguity. Yes, we can provide unique names and tell people to use special flags when running Docker builds, but in the packaging community, the presence of a Dockerfile in a directory suggests that the contents of that directory form the basis of a Docker image. +1 to removing the apache_hbase_topology folder since the dedicated repository for clusterdock is updated more often and will be getting some love in the coming weeks from yours truly. I can open a JIRA to do that myself. -Dima On Thu, Jul 20, 2017 at 8:59 AM, Misty Stanley-Joneswrote: > You can give them more descriptive names and use the -f flag to point to > the right path when invoking the docker CLI. > > On Thu, Jul 20, 2017 at 7:51 AM Sean Busbey wrote: > > > FWIW, I agree this is all confusing. > > > > > It looks like ./docker/Dockerfile is for Yetus, > > > > Yep! exactly. > > > > > ./Dockerfile > > > > I believe this one is used to provide dependencies needed for running > > the dev-support python scripts on builds.a.o. > > > > I think it's used here: > > > > https://builds.apache.org/view/H-L/view/HBase/job/HBase- > Find-Flaky-Tests/ > > > > > ./hbase_docker/Dockerfile > > > > This one has a README. It's meant for devs to use to easily get a > > standalone instance based on _master_ on their local machine. AFAIK > > it's not used in our builds.a.o stuff. > > > > > ./apache_hbase_topology/Dockerfile is for cluster testing > > > > This one is part of making it so folks can use clusterdock to deploy > > master. AFAIK it's not used in our builds.a.o stuff. I suspect we can > > remove it in favor of relying on the one that project publishes: > > > > https://github.com/clusterdock/topology_apache_hbase > > > > I think Andrew and Dima were the two I've seen relying on this in the > > past for checking RCs, so we should check with them before doing that. > > > > On Thu, Jul 20, 2017 at 9:22 AM, Mike Drob wrote: > > > Hi Devs, > > > > > > I noticed that we have 4 Dockerfiles under dev-support. Doing JIRA > > > spelunking, I wasn't able to figure out if they are actually > duplicative > > or > > > what the purposes for all of them are. > > > > > > mdrob@mdrob-MBP:~/IdeaProjects/hbase/dev-support$ find . -name > > Dockerfile > > > > > > ./apache_hbase_topology/Dockerfile > > > > > > ./docker/Dockerfile > > > > > > ./Dockerfile > > > > > > ./hbase_docker/Dockerfile > > > > > > It looks like ./docker/Dockerfile is for Yetus, > > > ./apache_hbase_topology/Dockerfile is for cluster testing, but I can't > > > figure out the purpose of the other two. Is there overlap? > > > > > > > > > Thanks, > > > > > > Mike > > >
Re: Dockerfiles under dev-support
You can give them more descriptive names and use the -f flag to point to the right path when invoking the docker CLI. On Thu, Jul 20, 2017 at 7:51 AM Sean Busbeywrote: > FWIW, I agree this is all confusing. > > > It looks like ./docker/Dockerfile is for Yetus, > > Yep! exactly. > > > ./Dockerfile > > I believe this one is used to provide dependencies needed for running > the dev-support python scripts on builds.a.o. > > I think it's used here: > > https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/ > > > ./hbase_docker/Dockerfile > > This one has a README. It's meant for devs to use to easily get a > standalone instance based on _master_ on their local machine. AFAIK > it's not used in our builds.a.o stuff. > > > ./apache_hbase_topology/Dockerfile is for cluster testing > > This one is part of making it so folks can use clusterdock to deploy > master. AFAIK it's not used in our builds.a.o stuff. I suspect we can > remove it in favor of relying on the one that project publishes: > > https://github.com/clusterdock/topology_apache_hbase > > I think Andrew and Dima were the two I've seen relying on this in the > past for checking RCs, so we should check with them before doing that. > > On Thu, Jul 20, 2017 at 9:22 AM, Mike Drob wrote: > > Hi Devs, > > > > I noticed that we have 4 Dockerfiles under dev-support. Doing JIRA > > spelunking, I wasn't able to figure out if they are actually duplicative > or > > what the purposes for all of them are. > > > > mdrob@mdrob-MBP:~/IdeaProjects/hbase/dev-support$ find . -name > Dockerfile > > > > ./apache_hbase_topology/Dockerfile > > > > ./docker/Dockerfile > > > > ./Dockerfile > > > > ./hbase_docker/Dockerfile > > > > It looks like ./docker/Dockerfile is for Yetus, > > ./apache_hbase_topology/Dockerfile is for cluster testing, but I can't > > figure out the purpose of the other two. Is there overlap? > > > > > > Thanks, > > > > Mike >
Re: Dockerfiles under dev-support
FWIW, I agree this is all confusing. > It looks like ./docker/Dockerfile is for Yetus, Yep! exactly. > ./Dockerfile I believe this one is used to provide dependencies needed for running the dev-support python scripts on builds.a.o. I think it's used here: https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/ > ./hbase_docker/Dockerfile This one has a README. It's meant for devs to use to easily get a standalone instance based on _master_ on their local machine. AFAIK it's not used in our builds.a.o stuff. > ./apache_hbase_topology/Dockerfile is for cluster testing This one is part of making it so folks can use clusterdock to deploy master. AFAIK it's not used in our builds.a.o stuff. I suspect we can remove it in favor of relying on the one that project publishes: https://github.com/clusterdock/topology_apache_hbase I think Andrew and Dima were the two I've seen relying on this in the past for checking RCs, so we should check with them before doing that. On Thu, Jul 20, 2017 at 9:22 AM, Mike Drobwrote: > Hi Devs, > > I noticed that we have 4 Dockerfiles under dev-support. Doing JIRA > spelunking, I wasn't able to figure out if they are actually duplicative or > what the purposes for all of them are. > > mdrob@mdrob-MBP:~/IdeaProjects/hbase/dev-support$ find . -name Dockerfile > > ./apache_hbase_topology/Dockerfile > > ./docker/Dockerfile > > ./Dockerfile > > ./hbase_docker/Dockerfile > > It looks like ./docker/Dockerfile is for Yetus, > ./apache_hbase_topology/Dockerfile is for cluster testing, but I can't > figure out the purpose of the other two. Is there overlap? > > > Thanks, > > Mike
Dockerfiles under dev-support
Hi Devs, I noticed that we have 4 Dockerfiles under dev-support. Doing JIRA spelunking, I wasn't able to figure out if they are actually duplicative or what the purposes for all of them are. mdrob@mdrob-MBP:~/IdeaProjects/hbase/dev-support$ find . -name Dockerfile ./apache_hbase_topology/Dockerfile ./docker/Dockerfile ./Dockerfile ./hbase_docker/Dockerfile It looks like ./docker/Dockerfile is for Yetus, ./apache_hbase_topology/Dockerfile is for cluster testing, but I can't figure out the purpose of the other two. Is there overlap? Thanks, Mike
[jira] [Reopened] (HBASE-17738) BucketCache startup is slow
[ https://issues.apache.org/jira/browse/HBASE-17738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan reopened HBASE-17738: > BucketCache startup is slow > --- > > Key: HBASE-17738 > URL: https://issues.apache.org/jira/browse/HBASE-17738 > Project: HBase > Issue Type: Sub-task > Components: BucketCache >Affects Versions: 2.0.0 >Reporter: stack >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0, 2.0.0-alpha-2 > > Attachments: HBASE-17738_2.patch, HBASE-17738_2.patch, > HBASE-17738_3.patch, HBASE-17738_4.patch, HBASE-17738_5_withoutUnsafe.patch, > HBASE-17738_6_withoutUnsafe.patch, HBASE-17738_8.patch, HBASE-17738_9.patch, > HBASE-17738.patch > > > If you set bucketcache size at 64G say and then start hbase, it takes a long > time. Can we do the allocations in parallel and not inline with the server > startup? > Related, prefetching on a bucketcache is slow. Speed it up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)