[jira] [Commented] (HBASE-20040) Master UI should include "Cluster Key" needed to use the cluster as a replication sink
[
https://issues.apache.org/jira/browse/HBASE-20040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16455791#comment-16455791
]
Sakthi commented on HBASE-20040:
Sure, [~busbey]
> Master UI should include "Cluster Key" needed to use the cluster as a
> replication sink
> --
>
> Key: HBASE-20040
> URL: https://issues.apache.org/jira/browse/HBASE-20040
> Project: HBase
> Issue Type: Improvement
> Components: Replication, Usability
>Reporter: Sean Busbey
>Assignee: Sakthi
>Priority: Minor
> Labels: beginner
> Fix For: 3.0.0, 2.1.0
>
> Attachments: hbase-20040.master.001.patch,
> hbase-20040.master.002.patch
>
>
> The ref guide defines a "Cluster Key" needed to add an hbase cluster as a
> replication peer
> {quote}
> CLUSTER_KEY: composed using the following template, with appropriate
> place-holders:
> {code}hbase.zookeeper.quorum:hbase.zookeeper.property.clientPort:zookeeper.znode.parent{code}
> {quote}
> the Master UI has all of the pieces displayed currently, but it should
> include a single field that operators can copy/paste.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20492) UnassignProcedure is stuck in retry loop on region stuck in OPENING state
[
https://issues.apache.org/jira/browse/HBASE-20492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16455782#comment-16455782
]
stack commented on HBASE-20492:
---
Made a subissue on how we can get into the retry loop. We can fix the found
'hole' but there are going to be others. The retries indicate something is
wrong. The ever-cycling procedure, if there are more than one, can clog up the
procedure executor but they flag that there is an issue. Not so good if many
hosed regions. Need a back-off. Dropping the procedure probably ain't the best
if it a sub-procedure of say, a disable. The disable could succeed though there
are regions still assigned. Will be back.
> UnassignProcedure is stuck in retry loop on region stuck in OPENING state
> -
>
> Key: HBASE-20492
> URL: https://issues.apache.org/jira/browse/HBASE-20492
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Affects Versions: 2.0.0
>Reporter: Umesh Agashe
>Priority: Critical
> Fix For: 2.0.1
>
>
> UnassignProcedure gets stuck in a retry loop for a region stuck in OPENING
> state. From logs:
> {code:java}
> 2018-04-25 15:59:53,825 WARN
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure:
> Retryable error trying to transition: pid=142564,
> state=RUNNABLE:REGION_TRANSITION_DISPATCH; UnassignProcedure
> table=IntegrationTestBigLinkedList_20180331004141,
> region=bd2fb2c7d39236c9b9085f350358df7c,
> server=vb1122.halxg.cloudera.com,22101,1522626198450; rit=OPENING,
> location=vb1122.halxg.cloudera.com,22101,1522626198450
> org.apache.hadoop.hbase.exceptions.UnexpectedStateException: Expected
> [SPLITTING, SPLIT, MERGING, OPEN, CLOSING] so could move to CLOSING but
> current state=OPENING
> at
> org.apache.hadoop.hbase.master.assignment.RegionStates$RegionStateNode.transitionState(RegionStates.java:158)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.markRegionAsClosing(AssignmentManager.java:1514)
> at
> org.apache.hadoop.hbase.master.assignment.UnassignProcedure.updateTransition(UnassignProcedure.java:179)
> at
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:309)
> at
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:85)
> at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1458)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1227)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1738)
> 2018-04-25 15:59:53,892 WARN
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure:
> Retryable error trying to transition: pid=142564,
> state=RUNNABLE:REGION_TRANSITION_DISPATCH; UnassignProcedure
> table=IntegrationTestBigLinkedList_20180331004141,
> region=bd2fb2c7d39236c9b9085f350358df7c,
> server=vb1122.halxg.cloudera.com,22101,1522626198450; rit=OPENING,
> location=vb1122.halxg.cloudera.com,22101,1522626198450
> org.apache.hadoop.hbase.exceptions.UnexpectedStateException: Expected
> [SPLITTING, SPLIT, MERGING, OPEN, CLOSING] so could move to CLOSING but
> current state=OPENING
> at
> org.apache.hadoop.hbase.master.assignment.RegionStates$RegionStateNode.transitionState(RegionStates.java:158)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.markRegionAsClosing(AssignmentManager.java:1514)
> at
> org.apache.hadoop.hbase.master.assignment.UnassignProcedure.updateTransition(UnassignProcedure.java:179)
> at
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:309)
> at
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:85)
> at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1458)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1227)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1738){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-20498) [AMv2] Stuck in UnexpectedStateException
stack created HBASE-20498: - Summary: [AMv2] Stuck in UnexpectedStateException Key: HBASE-20498 URL: https://issues.apache.org/jira/browse/HBASE-20498 Project: HBase Issue Type: Sub-task Reporter: stack Here is how we get into a stuck scenario in hbase-2.0.0RC2. * Assign a region. It is moved to OPENING state then RPCs the RS. * RS opens region. Tells Master. * Master tries to complete the assign by updating hbase:meta. * hbase:meta is hosed because I'd deployed a bad patch that blocked hbase:meta updates * Master is stuck retrying RPCs to RS hosting hbase:meta; we want to update our new OPEN state in hbase:meta. * I kill Master because I want to fix the broke patch. * On restart, a script sets table to be DISABLED. * As part of startup, we go to assign regions. * We skip assigning regions because the table is DISABLED; i.e. we skip the replay of the unfinished assign. * The region is now a free-agent; no lock held, so, the queued unassign that is part of the disable table can run * It fails because region is in OPENING state, an UnexpectedStateException is thrown. We loop complaining the above. Resolution requires finishing previous assign first, then we can disable. Let me try and write a test to manufacture this state. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20369) Document incompatibilities between HBase 1.1.2 and HBase 2.0
[ https://issues.apache.org/jira/browse/HBASE-20369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16455754#comment-16455754 ] Thiriguna Bharat Rao commented on HBASE-20369: -- [~busbey] First of all, many thanks for reviewing the content and providing feedback. Please find my responses to the feedback: # Rolling upgrade: When I created the first draft, there was no information available about the rolling upgrade in the hbase reference guide. I referenced the beta 1 discussion thread and the raw content. There was mention of: "Minimum of 1.2 for rolling upgrade to 2.0. No downtime. Works for secure cluster (delegation tokens)." So, I assumed that it's possible from 1.2. 2. Yes, I meant changes to InterfaceAudience.Private classes. My follow up query is, should we add a note that these classes are internal to the project. Also, in hindsight, I think, we can remove the general information heading and straightaway address the changes that they the users need to be aware of for reference. Please let me know your views. Appreciate your support and time. Best, Triguna > Document incompatibilities between HBase 1.1.2 and HBase 2.0 > > > Key: HBASE-20369 > URL: https://issues.apache.org/jira/browse/HBASE-20369 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Thiriguna Bharat Rao >Assignee: Thiriguna Bharat Rao >Priority: Critical > Labels: patch > Attachments: HBASE-20369.patch, HBASE-20369_v1.patch, > HBASE-20369_v2.patch, HBASE-20369_v3.patch, HBASE-20369_v4.patch, book.adoc > > > Hi, > I compiled a draft document for the HBase incompatibilities from the raw > source content that was available in HBase Beta 1 site. Can someone please > review and provide a feedback or share your comments on this document? > Appreciate your support and time. > > Best Regards, > Triguna -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20476) Fix the flaky TestReplicationSmallTests unit test
[
https://issues.apache.org/jira/browse/HBASE-20476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16455741#comment-16455741
]
Duo Zhang commented on HBASE-20476:
---
{noformat}
2018-04-27 03:08:21,768 DEBUG
[RS_REFRESH_PEER-regionserver/asf911:0-0.replicationSource,2.replicationSource.wal-reader.asf911.gq1.ygridcore.net%2C34977%2C1524798459531,2]
regionserver.SerialReplicationChecker(189): Replication barrier for
test/2dcce913bdddbe6bd46b11167744012b/3=[#edits: 0 = <>]:
ReplicationBarrierResult [barriers=[0, 2], state=OPEN, parentRegionNames=]
2018-04-27 03:08:21,769 DEBUG
[RS_REFRESH_PEER-regionserver/asf911:0-0.replicationSource,2.replicationSource.wal-reader.asf911.gq1.ygridcore.net%2C34977%2C1524798459531,2]
zookeeper.ZKUtil(704): regionserver:34977-0x1630511ec980004,
quorum=localhost:63109, baseZNode=/1 Unable to get data of znode
/1/replication/regions/2d/cc/e913bdddbe6bd46b11167744012b-2 because node does
not exist (not necessarily an error)
2018-04-27 03:08:21,769 DEBUG
[RS_REFRESH_PEER-regionserver/asf911:0-0.replicationSource,2.replicationSource.wal-reader.asf911.gq1.ygridcore.net%2C34977%2C1524798459531,2]
regionserver.SerialReplicationChecker(227): Previous range for
test/2dcce913bdddbe6bd46b11167744012b/3=[#edits: 0 = <>] has not been finished
yet, give up
2018-04-27 03:08:21,769 DEBUG
[RS_REFRESH_PEER-regionserver/asf911:0-0.replicationSource,2.replicationSource.wal-reader.asf911.gq1.ygridcore.net%2C34977%2C1524798459531,2]
regionserver.SerialReplicationChecker(269): Can not push
test/2dcce913bdddbe6bd46b11167744012b/3=[#edits: 0 = <>], wait
{noformat}
This is the root cause I think. Let me dig.
> Fix the flaky TestReplicationSmallTests unit test
> -
>
> Key: HBASE-20476
> URL: https://issues.apache.org/jira/browse/HBASE-20476
> Project: HBase
> Issue Type: Bug
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Attachments: HBASE-20476.patch
>
>
> See
> https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20476) Fix the flaky TestReplicationSmallTests unit test
[ https://issues.apache.org/jira/browse/HBASE-20476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16455733#comment-16455733 ] Duo Zhang commented on HBASE-20476: --- Seems not much help. > Fix the flaky TestReplicationSmallTests unit test > - > > Key: HBASE-20476 > URL: https://issues.apache.org/jira/browse/HBASE-20476 > Project: HBase > Issue Type: Bug >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Attachments: HBASE-20476.patch > > > See > https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20476) Fix the flaky TestReplicationSmallTests unit test
[ https://issues.apache.org/jira/browse/HBASE-20476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16455667#comment-16455667 ] Zheng Hu commented on HBASE-20476: -- OK, +1 for HBASE-20476.patch. Please commit first. > Fix the flaky TestReplicationSmallTests unit test > - > > Key: HBASE-20476 > URL: https://issues.apache.org/jira/browse/HBASE-20476 > Project: HBase > Issue Type: Bug >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Attachments: HBASE-20476.patch > > > See > https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20497) The getRecoveredQueueStartPos always return -1 in RecoveredReplicationSourceShipper
Zheng Hu created HBASE-20497: Summary: The getRecoveredQueueStartPos always return -1 in RecoveredReplicationSourceShipper Key: HBASE-20497 URL: https://issues.apache.org/jira/browse/HBASE-20497 Project: HBase Issue Type: Bug Affects Versions: 2.1.0 Reporter: Zheng Hu Assignee: Zheng Hu Fix For: 3.0.0, 2.1.0 See https://issues.apache.org/jira/browse/HBASE-20475?page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel=16454240#comment-16454240 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20476) Fix the flaky TestReplicationSmallTests unit test
[ https://issues.apache.org/jira/browse/HBASE-20476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16455666#comment-16455666 ] Duo Zhang commented on HBASE-20476: --- Anyway, I think we can commit this first to make the test more stable, so that we can get the actual problem on the flakey dashboard. > Fix the flaky TestReplicationSmallTests unit test > - > > Key: HBASE-20476 > URL: https://issues.apache.org/jira/browse/HBASE-20476 > Project: HBase > Issue Type: Bug >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Attachments: HBASE-20476.patch > > > See > https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20476) Fix the flaky TestReplicationSmallTests unit test
[
https://issues.apache.org/jira/browse/HBASE-20476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16455664#comment-16455664
]
Zheng Hu edited comment on HBASE-20476 at 4/27/18 2:09 AM:
---
Run several times under my local, the testDisableEnable/testDeleteTypes are
still not stable.
{code}
java.lang.AssertionError: Waited too much time for put replication
at org.junit.Assert.fail(Assert.java:88)
at
org.apache.hadoop.hbase.replication.TestReplicationSmallTests.testDisableEnable(TestReplicationSmallTests.java:238)
{code}
{code}
java.lang.AssertionError: Waited too much time for put replication
at org.junit.Assert.fail(Assert.java:88)
at
org.apache.hadoop.hbase.replication.TestReplicationSmallTests.testDeleteTypes(TestReplicationSmallTests.java:125)
{code}
Will go back to this issue after I fix the HBASE-20475
was (Author: openinx):
Run several times under my local, the testDisableEnable/testDeleteTypes are
still not stable.
{code}
java.lang.AssertionError: Waited too much time for put replication
at org.junit.Assert.fail(Assert.java:88)
at
org.apache.hadoop.hbase.replication.TestReplicationSmallTests.testDisableEnable(TestReplicationSmallTests.java:238)
{code}
Will go back to this issue after I fix the HBASE-20475
> Fix the flaky TestReplicationSmallTests unit test
> -
>
> Key: HBASE-20476
> URL: https://issues.apache.org/jira/browse/HBASE-20476
> Project: HBase
> Issue Type: Bug
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Attachments: HBASE-20476.patch
>
>
> See
> https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20476) Fix the flaky TestReplicationSmallTests unit test
[
https://issues.apache.org/jira/browse/HBASE-20476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16455664#comment-16455664
]
Zheng Hu commented on HBASE-20476:
--
Run several times under my local, the testDisableEnable/testDeleteTypes are
still not stable.
{code}
java.lang.AssertionError: Waited too much time for put replication
at org.junit.Assert.fail(Assert.java:88)
at
org.apache.hadoop.hbase.replication.TestReplicationSmallTests.testDisableEnable(TestReplicationSmallTests.java:238)
{code}
Will go back to this issue after I fix the HBASE-20475
> Fix the flaky TestReplicationSmallTests unit test
> -
>
> Key: HBASE-20476
> URL: https://issues.apache.org/jira/browse/HBASE-20476
> Project: HBase
> Issue Type: Bug
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Attachments: HBASE-20476.patch
>
>
> See
> https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19543) Abstract a replication storage interface to extract the zk specific code
[ https://issues.apache.org/jira/browse/HBASE-19543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16455662#comment-16455662 ] Duo Zhang commented on HBASE-19543: --- What's the actual purpose of the tool? Is it used to determine whether the replication system works? Or for a specific peer? > Abstract a replication storage interface to extract the zk specific code > > > Key: HBASE-19543 > URL: https://issues.apache.org/jira/browse/HBASE-19543 > Project: HBase > Issue Type: Sub-task > Components: proc-v2, Replication >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.1.0 > > Attachments: HBASE-19543-HBASE-19397-v1.patch, > HBASE-19543-HBASE-19397-v2.patch, HBASE-19543-HBASE-19397-v3.patch, > HBASE-19543-HBASE-19397.patch, HBASE-19543-HBASE-19397.patch > > > For now, we will do sanity checks at the same time when updating replication > peer. But this is not a safe way for procedure based replication peer > modification. > For the old zk watcher way, the only thing is updating the data on zk, so if > the data is updated and then we crashes, there is no problem. > For the new procedure way, we need to trigger refresh by ourselves after > updating zk. If we crashes after the updating and before we record the state > change of the procedure, we may fail with IllegalArgumentException when we > execute the procedure next time since the data on zk has already been updated. > So the current way is to do sanity checks in PRE_PEER_MODIFICATION state, and > in UPDATE_STORAGE state we will not do sanity checks any more, just > update(overwrite) the peer storage. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19543) Abstract a replication storage interface to extract the zk specific code
[ https://issues.apache.org/jira/browse/HBASE-19543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16455652#comment-16455652 ] Guanghao Zhang commented on HBASE-19543: [~mdrob] In 2.0, the replication operation will use the new procedure 2.0 framework. You can make sure replication started work after you add a new replication peer. > Abstract a replication storage interface to extract the zk specific code > > > Key: HBASE-19543 > URL: https://issues.apache.org/jira/browse/HBASE-19543 > Project: HBase > Issue Type: Sub-task > Components: proc-v2, Replication >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.1.0 > > Attachments: HBASE-19543-HBASE-19397-v1.patch, > HBASE-19543-HBASE-19397-v2.patch, HBASE-19543-HBASE-19397-v3.patch, > HBASE-19543-HBASE-19397.patch, HBASE-19543-HBASE-19397.patch > > > For now, we will do sanity checks at the same time when updating replication > peer. But this is not a safe way for procedure based replication peer > modification. > For the old zk watcher way, the only thing is updating the data on zk, so if > the data is updated and then we crashes, there is no problem. > For the new procedure way, we need to trigger refresh by ourselves after > updating zk. If we crashes after the updating and before we record the state > change of the procedure, we may fail with IllegalArgumentException when we > execute the procedure next time since the data on zk has already been updated. > So the current way is to do sanity checks in PRE_PEER_MODIFICATION state, and > in UPDATE_STORAGE state we will not do sanity checks any more, just > update(overwrite) the peer storage. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20478) move import checks from hbaseanti to checkstyle
[ https://issues.apache.org/jira/browse/HBASE-20478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob updated HBASE-20478: -- Attachment: HBASE-20478.4.patch > move import checks from hbaseanti to checkstyle > --- > > Key: HBASE-20478 > URL: https://issues.apache.org/jira/browse/HBASE-20478 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Sean Busbey >Assignee: Mike Drob >Priority: Minor > Attachments: HBASE-20478.0.patch, HBASE-20478.1.patch, > HBASE-20478.2.patch, HBASE-20478.3.patch, HBASE-20478.4.patch, > HBASE-20478.WIP.2.patch, HBASE-20478.WIP.2.patch, HBASE-20478.WIP.patch, > HBASE-anti-check.patch > > > came up in discussion on HBASE-20332. our check of "don't do this" things in > the codebase doesn't log the specifics of complaints anywhere, which forces > those who want to follow up to reverse engineer the check. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20478) move import checks from hbaseanti to checkstyle
[ https://issues.apache.org/jira/browse/HBASE-20478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16455636#comment-16455636 ] Mike Drob edited comment on HBASE-20478 at 4/27/18 1:37 AM: Checkstyle 7+ requires JDK 8+, so won't be able to backport such a fix to branch-1, AFAICT. v4: - update checkstyle version - update checkstyle plugin version - move the import checks from anti to checkstyle couldn't verify this locally because checkstyle doesn't run in the normal maven build anymore and for some reason test-patch was ignoring the checkstyle plugin for me (maybe related to HBASE-20467) edit: maybe i need dummy changes in other modules to get it picked up? was (Author: mdrob): Checkstyle 7+ requires JDK 8+, so won't be able to backport such a fix to branch-1, AFAICT. v4: - update checkstyle version - update checkstyle plugin version - move the import checks from anti to checkstyle couldn't verify this locally because checkstyle doesn't run in the normal maven build anymore and for some reason test-patch was ignoring the checkstyle plugin for me (maybe related to HBASE-20467) > move import checks from hbaseanti to checkstyle > --- > > Key: HBASE-20478 > URL: https://issues.apache.org/jira/browse/HBASE-20478 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Sean Busbey >Assignee: Mike Drob >Priority: Minor > Attachments: HBASE-20478.0.patch, HBASE-20478.1.patch, > HBASE-20478.2.patch, HBASE-20478.3.patch, HBASE-20478.4.patch, > HBASE-20478.WIP.2.patch, HBASE-20478.WIP.2.patch, HBASE-20478.WIP.patch, > HBASE-anti-check.patch > > > came up in discussion on HBASE-20332. our check of "don't do this" things in > the codebase doesn't log the specifics of complaints anywhere, which forces > those who want to follow up to reverse engineer the check. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20478) move import checks from hbaseanti to checkstyle
[ https://issues.apache.org/jira/browse/HBASE-20478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16455636#comment-16455636 ] Mike Drob commented on HBASE-20478: --- Checkstyle 7+ requires JDK 8+, so won't be able to backport such a fix to branch-1, AFAICT. v4: - update checkstyle version - update checkstyle plugin version - move the import checks from anti to checkstyle couldn't verify this locally because checkstyle doesn't run in the normal maven build anymore and for some reason test-patch was ignoring the checkstyle plugin for me (maybe related to HBASE-20467) > move import checks from hbaseanti to checkstyle > --- > > Key: HBASE-20478 > URL: https://issues.apache.org/jira/browse/HBASE-20478 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Sean Busbey >Assignee: Mike Drob >Priority: Minor > Attachments: HBASE-20478.0.patch, HBASE-20478.1.patch, > HBASE-20478.2.patch, HBASE-20478.3.patch, HBASE-20478.4.patch, > HBASE-20478.WIP.2.patch, HBASE-20478.WIP.2.patch, HBASE-20478.WIP.patch, > HBASE-anti-check.patch > > > came up in discussion on HBASE-20332. our check of "don't do this" things in > the codebase doesn't log the specifics of complaints anywhere, which forces > those who want to follow up to reverse engineer the check. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20476) Fix the flaky TestReplicationSmallTests unit test
[ https://issues.apache.org/jira/browse/HBASE-20476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16455595#comment-16455595 ] Duo Zhang commented on HBASE-20476: --- [~openinx] Could please try the patch to see if it works for you? > Fix the flaky TestReplicationSmallTests unit test > - > > Key: HBASE-20476 > URL: https://issues.apache.org/jira/browse/HBASE-20476 > Project: HBase > Issue Type: Bug >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Attachments: HBASE-20476.patch > > > See > https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20229) ConnectionImplementation.locateRegions() returns duplicated entries when region replication is on
[
https://issues.apache.org/jira/browse/HBASE-20229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16455512#comment-16455512
]
Hudson commented on HBASE-20229:
Results for branch branch-1.3
[build #309 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/309/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/309//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/309//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/309//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> ConnectionImplementation.locateRegions() returns duplicated entries when
> region replication is on
> -
>
> Key: HBASE-20229
> URL: https://issues.apache.org/jira/browse/HBASE-20229
> Project: HBase
> Issue Type: Bug
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: HBASE-20229.branch-1.001.patch,
> HBASE-20229.branch-1.002.patch, HBASE-20229.branch-1.002.patch,
> HBASE-20229.master.001.patch, HBASE-20229.master.002.patch,
> HBASE-20229.master.003.patch, HBASE-20229.master.004.patch
>
>
> The issue is when region replication is on,
> ConnectionImplementation.locateRegions() returns duplicated entries.
> In the test in my patch, the table should have 1 primary region + 2
> secondaries, but ConnectionImplementation.locateRegions() returns 9 regions.
> Every region repeats 3 times (3 = replicas count).
> I think this is because the following code calls locateRegion() even for
> replica regions and then the result triples.
> {code:java}
> for (RegionInfo regionInfo : regions) {
> RegionLocations list = locateRegion(tableName,
> regionInfo.getStartKey(), useCache, true);
> if (list != null) {
> for (HRegionLocation loc : list.getRegionLocations()) {
> if (loc != null) {
> locations.add(loc);
> }
> }
> }
> {code}
> The fix in my patch is to make it call locateRegion() only for a primary
> region.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18842) The hbase shell clone_snaphost command returns bad error message
[
https://issues.apache.org/jira/browse/HBASE-18842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16455513#comment-16455513
]
Hudson commented on HBASE-18842:
Results for branch branch-1.3
[build #309 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/309/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/309//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/309//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/309//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> The hbase shell clone_snaphost command returns bad error message
>
>
> Key: HBASE-18842
> URL: https://issues.apache.org/jira/browse/HBASE-18842
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Thoralf Gutierrez
>Assignee: Thoralf Gutierrez
>Priority: Minor
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 2.0.1, 1.4.5
>
> Attachments:
> 0001-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch,
> 0002-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch,
> 0003-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch,
> 0004-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch,
> 0005-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch
>
>
> When you call the hbase shell clone_snapshot command with a target namespace
> that doesn't exist, you get an error message, but the variable used to
> identify the inexistent namespace is wrong:
> {noformat}
> hbase(main):001:0> clone_snapshot 'someSnapshotName',
> 'someNamespaceName:someTableName'
> ERROR: Unknown namespace someSnapshotName!
> Create a new table by cloning the snapshot content.
> There're no copies of data involved.
> And writing on the newly created table will not influence the snapshot data.
> Examples:
> hbase> clone_snapshot 'snapshotName', 'tableName'
> hbase> clone_snapshot 'snapshotName', 'namespace:tableName'
> {noformat}
> It should rather say:
> {noformat}
> ERROR: Unknown namespace someNamespaceName!
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-20493) Port HBASE-19994 (Create a new class for RPC throttling exception, make it retryable) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-20493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-20493. Resolution: Fixed Hadoop Flags: Reviewed > Port HBASE-19994 (Create a new class for RPC throttling exception, make it > retryable) to branch-1 > - > > Key: HBASE-20493 > URL: https://issues.apache.org/jira/browse/HBASE-20493 > Project: HBase > Issue Type: Task >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 1.5.0 > > Attachments: HBASE-20493-branch-1.patch > > > Port HBASE-19994 (Create a new class for RPC throttling exception, make it > retryable). Need to preserve the current behavior where the client gets a > non-retryable ThrottlingException and only optionally throw back the > retryable RpcThrottlingException if explicitly allowed by configuration. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-9465) Push entries to peer clusters serially
[ https://issues.apache.org/jira/browse/HBASE-9465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-9465. --- Resolution: Fixed > Push entries to peer clusters serially > -- > > Key: HBASE-9465 > URL: https://issues.apache.org/jira/browse/HBASE-9465 > Project: HBase > Issue Type: New Feature > Components: regionserver, Replication >Affects Versions: 1.4.0, 2.0.0 >Reporter: Honghua Feng >Assignee: Phil Yang >Priority: Critical > Attachments: HBASE-9465-branch-1-v1.patch, > HBASE-9465-branch-1-v1.patch, HBASE-9465-branch-1-v2.patch, > HBASE-9465-branch-1-v3.patch, HBASE-9465-branch-1-v4.patch, > HBASE-9465-branch-1-v4.patch, HBASE-9465-branch-1.v4.revert.patch, > HBASE-9465-v1.patch, HBASE-9465-v2.patch, HBASE-9465-v2.patch, > HBASE-9465-v3.patch, HBASE-9465-v4.patch, HBASE-9465-v5.patch, > HBASE-9465-v6.patch, HBASE-9465-v6.patch, HBASE-9465-v7.patch, > HBASE-9465-v7.patch, HBASE-9465.pdf > > > When region-move or RS failure occurs in master cluster, the hlog entries > that are not pushed before region-move or RS-failure will be pushed by > original RS(for region move) or another RS which takes over the remained hlog > of dead RS(for RS failure), and the new entries for the same region(s) will > be pushed by the RS which now serves the region(s), but they push the hlog > entries of a same region concurrently without coordination. > This treatment can possibly lead to data inconsistency between master and > peer clusters: > 1. there are put and then delete written to master cluster > 2. due to region-move / RS-failure, they are pushed by different > replication-source threads to peer cluster > 3. if delete is pushed to peer cluster before put, and flush and > major-compact occurs in peer cluster before put is pushed to peer cluster, > the delete is collected and the put remains in peer cluster > In this scenario, the put remains in peer cluster, but in master cluster the > put is masked by the delete, hence data inconsistency between master and peer > clusters -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-9465) Push entries to peer clusters serially
[ https://issues.apache.org/jira/browse/HBASE-9465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16455492#comment-16455492 ] Andrew Purtell commented on HBASE-9465: --- Ok, I filed HBASE-20496 > Push entries to peer clusters serially > -- > > Key: HBASE-9465 > URL: https://issues.apache.org/jira/browse/HBASE-9465 > Project: HBase > Issue Type: New Feature > Components: regionserver, Replication >Affects Versions: 1.4.0, 2.0.0 >Reporter: Honghua Feng >Assignee: Phil Yang >Priority: Critical > Attachments: HBASE-9465-branch-1-v1.patch, > HBASE-9465-branch-1-v1.patch, HBASE-9465-branch-1-v2.patch, > HBASE-9465-branch-1-v3.patch, HBASE-9465-branch-1-v4.patch, > HBASE-9465-branch-1-v4.patch, HBASE-9465-branch-1.v4.revert.patch, > HBASE-9465-v1.patch, HBASE-9465-v2.patch, HBASE-9465-v2.patch, > HBASE-9465-v3.patch, HBASE-9465-v4.patch, HBASE-9465-v5.patch, > HBASE-9465-v6.patch, HBASE-9465-v6.patch, HBASE-9465-v7.patch, > HBASE-9465-v7.patch, HBASE-9465.pdf > > > When region-move or RS failure occurs in master cluster, the hlog entries > that are not pushed before region-move or RS-failure will be pushed by > original RS(for region move) or another RS which takes over the remained hlog > of dead RS(for RS failure), and the new entries for the same region(s) will > be pushed by the RS which now serves the region(s), but they push the hlog > entries of a same region concurrently without coordination. > This treatment can possibly lead to data inconsistency between master and > peer clusters: > 1. there are put and then delete written to master cluster > 2. due to region-move / RS-failure, they are pushed by different > replication-source threads to peer cluster > 3. if delete is pushed to peer cluster before put, and flush and > major-compact occurs in peer cluster before put is pushed to peer cluster, > the delete is collected and the put remains in peer cluster > In this scenario, the put remains in peer cluster, but in master cluster the > put is masked by the delete, hence data inconsistency between master and peer > clusters -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20496) TestGlobalThrottler failing on branch-1 since revert of HBASE-9465
Andrew Purtell created HBASE-20496:
--
Summary: TestGlobalThrottler failing on branch-1 since revert of
HBASE-9465
Key: HBASE-20496
URL: https://issues.apache.org/jira/browse/HBASE-20496
Project: HBase
Issue Type: Bug
Reporter: Andrew Purtell
Not sure why we didn't catch it earlier, but with my latest dev setup including
8u JVM, TestGlobalThrottler fails reliably, and a git bisect finds the problem
at this revert:
{noformat}
commit ba7a936f74985eb9d974fdc87b0d06cb8cd8473d
Author: Sean Busbey
Date: Tue Nov 7 23:50:35 2017 -0600
Revert "HBASE-9465 Push entries to peer clusters serially"
This reverts commit 441bc050b991c14c048617bc443b97f46e21b76f.
Conflicts:
hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
hbase-client/src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java
hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/WALProtos.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/Replication.java
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java
Signed-off-by: Andrew Purtell
{noformat}
For now I'm going to disable the test. Leaving this open for debugging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20489) Update Reference Guide that CLUSTER_KEY value is present on the Master UI info page.
[
https://issues.apache.org/jira/browse/HBASE-20489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16455473#comment-16455473
]
Hudson commented on HBASE-20489:
Results for branch branch-2
[build #663 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/663/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/663//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/663//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/663//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Update Reference Guide that CLUSTER_KEY value is present on the Master UI
> info page.
>
>
> Key: HBASE-20489
> URL: https://issues.apache.org/jira/browse/HBASE-20489
> Project: HBase
> Issue Type: Sub-task
> Components: documentation
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Minor
> Attachments: hbase-20489.master.001.patch
>
>
> CLUSTER_KEY info is now present on the Master UI info page. The reference
> guide's section that defines the CLUSTER_KEY, must contain the info that it
> is now present in the UI as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20040) Master UI should include "Cluster Key" needed to use the cluster as a replication sink
[
https://issues.apache.org/jira/browse/HBASE-20040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16455472#comment-16455472
]
Hudson commented on HBASE-20040:
Results for branch branch-2
[build #663 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/663/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/663//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/663//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/663//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Master UI should include "Cluster Key" needed to use the cluster as a
> replication sink
> --
>
> Key: HBASE-20040
> URL: https://issues.apache.org/jira/browse/HBASE-20040
> Project: HBase
> Issue Type: Improvement
> Components: Replication, Usability
>Reporter: Sean Busbey
>Assignee: Sakthi
>Priority: Minor
> Labels: beginner
> Fix For: 3.0.0, 2.1.0
>
> Attachments: hbase-20040.master.001.patch,
> hbase-20040.master.002.patch
>
>
> The ref guide defines a "Cluster Key" needed to add an hbase cluster as a
> replication peer
> {quote}
> CLUSTER_KEY: composed using the following template, with appropriate
> place-holders:
> {code}hbase.zookeeper.quorum:hbase.zookeeper.property.clientPort:zookeeper.znode.parent{code}
> {quote}
> the Master UI has all of the pieces displayed currently, but it should
> include a single field that operators can copy/paste.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20493) Port HBASE-19994 (Create a new class for RPC throttling exception, make it retryable) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-20493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20493: --- Release Note: Before this change when RPC quotas are exceeded the server will throw ThrottlingException to the client, which is unfriendly, because ThrottlingException is derived from DoNotRetryIOException. The application must handle throttling. With this change we introduce a new exception type for RPC quotas, RpcThrottlingException, which is retryable. If the configuration parameter 'hbase.quota.retryable.throttlingexception' (default: 'false') is set to 'true' the server will throw RpcThrottlingException back to the client instead, and the client will handle this exception like any other transient condition, retrying using the configured retry policy until exhausted. The application will still need to handle the case where, because of throttling, all retries have been exhausted, but this is the normal RetriesExhaustedException that applications have to deal with anyway. Throttling will no longer demand special handling. > Port HBASE-19994 (Create a new class for RPC throttling exception, make it > retryable) to branch-1 > - > > Key: HBASE-20493 > URL: https://issues.apache.org/jira/browse/HBASE-20493 > Project: HBase > Issue Type: Task >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 1.5.0 > > Attachments: HBASE-20493-branch-1.patch > > > Port HBASE-19994 (Create a new class for RPC throttling exception, make it > retryable). Need to preserve the current behavior where the client gets a > non-retryable ThrottlingException and only optionally throw back the > retryable RpcThrottlingException if explicitly allowed by configuration. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20493) Port HBASE-19994 (Create a new class for RPC throttling exception, make it retryable) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-20493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20493: --- Release Note: Before this change when RPC quotas are exceeded the server will throw ThrottlingException to the client, which is unfriendly, because ThrottlingException is derived from DoNotRetryIOException. The application must specially handle error conditions caused by throttling. With this change we introduce a new exception type for RPC quotas, RpcThrottlingException, which is retryable. If the configuration parameter 'hbase.quota.retryable.throttlingexception' (default: 'false') is set to 'true' the server will throw RpcThrottlingException back to the client instead, and the client will handle this exception like any other transient condition, retrying using the configured retry policy until exhausted. The application will still need to handle the case where, because of throttling, all retries have been exhausted, but this is the normal RetriesExhaustedException that applications have to deal with anyway. Throttling will no longer demand special handling. (was: Before this change when RPC quotas are exceeded the server will throw ThrottlingException to the client, which is unfriendly, because ThrottlingException is derived from DoNotRetryIOException. The application must handle throttling. With this change we introduce a new exception type for RPC quotas, RpcThrottlingException, which is retryable. If the configuration parameter 'hbase.quota.retryable.throttlingexception' (default: 'false') is set to 'true' the server will throw RpcThrottlingException back to the client instead, and the client will handle this exception like any other transient condition, retrying using the configured retry policy until exhausted. The application will still need to handle the case where, because of throttling, all retries have been exhausted, but this is the normal RetriesExhaustedException that applications have to deal with anyway. Throttling will no longer demand special handling.) > Port HBASE-19994 (Create a new class for RPC throttling exception, make it > retryable) to branch-1 > - > > Key: HBASE-20493 > URL: https://issues.apache.org/jira/browse/HBASE-20493 > Project: HBase > Issue Type: Task >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 1.5.0 > > Attachments: HBASE-20493-branch-1.patch > > > Port HBASE-19994 (Create a new class for RPC throttling exception, make it > retryable). Need to preserve the current behavior where the client gets a > non-retryable ThrottlingException and only optionally throw back the > retryable RpcThrottlingException if explicitly allowed by configuration. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20478) move import checks from hbaseanti to checkstyle
[
https://issues.apache.org/jira/browse/HBASE-20478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16455454#comment-16455454
]
Hadoop QA commented on HBASE-20478:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
33s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
0s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
3s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
14m 24s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}148m
13s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 19m
50s{color} | {color:green} hbase-mapreduce in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 11m
45s{color} | {color:green} hbase-backup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}232m 0s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20478 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12920848/HBASE-anti-check.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 93074e5615fe 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 90d103d6d5 |
|
[jira] [Commented] (HBASE-19924) hbase rpc throttling does not work for multi() with request count rater.
[
https://issues.apache.org/jira/browse/HBASE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16455165#comment-16455165
]
Hudson commented on HBASE-19924:
SUCCESS: Integrated in Jenkins build HBase-1.2-IT #1107 (See
[https://builds.apache.org/job/HBase-1.2-IT/1107/])
HBASE-19924 hbase rpc throttling does not work for multi() with request
(apurtell: rev 1519dc05b8faa0008c977a4ffbc6e066481fdbf9)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/QuotaLimiter.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/TimeBasedLimiter.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/DefaultOperationQuota.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/NoopQuotaLimiter.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestQuotaState.java
> hbase rpc throttling does not work for multi() with request count rater.
>
>
> Key: HBASE-19924
> URL: https://issues.apache.org/jira/browse/HBASE-19924
> Project: HBase
> Issue Type: Bug
> Components: rpc
>Affects Versions: 0.16.0, 1.2.6
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Major
> Fix For: 1.5.0, 1.2.7, 1.3.3, 2.0.0, 1.4.5
>
> Attachments: HBASE-19924-branch-1-v001.patch,
> HBASE-19924-master-v001.patch, HBASE-19924-master-v002.patch,
> HBASE-19924-master-v002.patch, HBASE-19924-master-v002.patch,
> HBASE-19924-master-v002.patch
>
>
> Basically, rpc throttling does not work for request count based rater for
> multi. for the following code, when it calls limiter's checkQuota(),
> numWrites/numReads is lost.
> {code:java}
> @Override
> public void checkQuota(int numWrites, int numReads, int numScans) throws
> ThrottlingException {
> writeConsumed = estimateConsume(OperationType.MUTATE, numWrites, 100);
> readConsumed = estimateConsume(OperationType.GET, numReads, 100);
> readConsumed += estimateConsume(OperationType.SCAN, numScans, 1000);
> writeAvailable = Long.MAX_VALUE;
> readAvailable = Long.MAX_VALUE;
> for (final QuotaLimiter limiter : limiters) {
> if (limiter.isBypass()) continue;
> limiter.checkQuota(writeConsumed, readConsumed);
> readAvailable = Math.min(readAvailable, limiter.getReadAvailable());
> writeAvailable = Math.min(writeAvailable, limiter.getWriteAvailable());
> }
> for (final QuotaLimiter limiter : limiters) {
> limiter.grabQuota(writeConsumed, readConsumed);
> }
> }{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19924) hbase rpc throttling does not work for multi() with request count rater.
[
https://issues.apache.org/jira/browse/HBASE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454919#comment-16454919
]
Hudson commented on HBASE-19924:
SUCCESS: Integrated in Jenkins build HBase-1.3-IT #400 (See
[https://builds.apache.org/job/HBase-1.3-IT/400/])
HBASE-19924 hbase rpc throttling does not work for multi() with request
(apurtell: rev 62a3434b9bb849395b45e90cf17f56a74eb96c00)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/TimeBasedLimiter.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/NoopQuotaLimiter.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/QuotaLimiter.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestQuotaState.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/DefaultOperationQuota.java
> hbase rpc throttling does not work for multi() with request count rater.
>
>
> Key: HBASE-19924
> URL: https://issues.apache.org/jira/browse/HBASE-19924
> Project: HBase
> Issue Type: Bug
> Components: rpc
>Affects Versions: 0.16.0, 1.2.6
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Major
> Fix For: 1.5.0, 1.2.7, 1.3.3, 2.0.0, 1.4.5
>
> Attachments: HBASE-19924-branch-1-v001.patch,
> HBASE-19924-master-v001.patch, HBASE-19924-master-v002.patch,
> HBASE-19924-master-v002.patch, HBASE-19924-master-v002.patch,
> HBASE-19924-master-v002.patch
>
>
> Basically, rpc throttling does not work for request count based rater for
> multi. for the following code, when it calls limiter's checkQuota(),
> numWrites/numReads is lost.
> {code:java}
> @Override
> public void checkQuota(int numWrites, int numReads, int numScans) throws
> ThrottlingException {
> writeConsumed = estimateConsume(OperationType.MUTATE, numWrites, 100);
> readConsumed = estimateConsume(OperationType.GET, numReads, 100);
> readConsumed += estimateConsume(OperationType.SCAN, numScans, 1000);
> writeAvailable = Long.MAX_VALUE;
> readAvailable = Long.MAX_VALUE;
> for (final QuotaLimiter limiter : limiters) {
> if (limiter.isBypass()) continue;
> limiter.checkQuota(writeConsumed, readConsumed);
> readAvailable = Math.min(readAvailable, limiter.getReadAvailable());
> writeAvailable = Math.min(writeAvailable, limiter.getWriteAvailable());
> }
> for (final QuotaLimiter limiter : limiters) {
> limiter.grabQuota(writeConsumed, readConsumed);
> }
> }{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18842) The hbase shell clone_snaphost command returns bad error message
[
https://issues.apache.org/jira/browse/HBASE-18842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454917#comment-16454917
]
Hudson commented on HBASE-18842:
Results for branch branch-1.2
[build #313 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/313/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/313//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/313//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/313//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> The hbase shell clone_snaphost command returns bad error message
>
>
> Key: HBASE-18842
> URL: https://issues.apache.org/jira/browse/HBASE-18842
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Thoralf Gutierrez
>Assignee: Thoralf Gutierrez
>Priority: Minor
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 2.0.1, 1.4.5
>
> Attachments:
> 0001-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch,
> 0002-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch,
> 0003-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch,
> 0004-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch,
> 0005-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch
>
>
> When you call the hbase shell clone_snapshot command with a target namespace
> that doesn't exist, you get an error message, but the variable used to
> identify the inexistent namespace is wrong:
> {noformat}
> hbase(main):001:0> clone_snapshot 'someSnapshotName',
> 'someNamespaceName:someTableName'
> ERROR: Unknown namespace someSnapshotName!
> Create a new table by cloning the snapshot content.
> There're no copies of data involved.
> And writing on the newly created table will not influence the snapshot data.
> Examples:
> hbase> clone_snapshot 'snapshotName', 'tableName'
> hbase> clone_snapshot 'snapshotName', 'namespace:tableName'
> {noformat}
> It should rather say:
> {noformat}
> ERROR: Unknown namespace someNamespaceName!
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19924) hbase rpc throttling does not work for multi() with request count rater.
[
https://issues.apache.org/jira/browse/HBASE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-19924:
---
Fix Version/s: 1.4.5
1.3.3
1.2.7
I think this is a bug so I picked it back to the release branches 1.2 and up.
Please let me know or revert if you disagree.
> hbase rpc throttling does not work for multi() with request count rater.
>
>
> Key: HBASE-19924
> URL: https://issues.apache.org/jira/browse/HBASE-19924
> Project: HBase
> Issue Type: Bug
> Components: rpc
>Affects Versions: 0.16.0, 1.2.6
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Major
> Fix For: 1.5.0, 1.2.7, 1.3.3, 2.0.0, 1.4.5
>
> Attachments: HBASE-19924-branch-1-v001.patch,
> HBASE-19924-master-v001.patch, HBASE-19924-master-v002.patch,
> HBASE-19924-master-v002.patch, HBASE-19924-master-v002.patch,
> HBASE-19924-master-v002.patch
>
>
> Basically, rpc throttling does not work for request count based rater for
> multi. for the following code, when it calls limiter's checkQuota(),
> numWrites/numReads is lost.
> {code:java}
> @Override
> public void checkQuota(int numWrites, int numReads, int numScans) throws
> ThrottlingException {
> writeConsumed = estimateConsume(OperationType.MUTATE, numWrites, 100);
> readConsumed = estimateConsume(OperationType.GET, numReads, 100);
> readConsumed += estimateConsume(OperationType.SCAN, numScans, 1000);
> writeAvailable = Long.MAX_VALUE;
> readAvailable = Long.MAX_VALUE;
> for (final QuotaLimiter limiter : limiters) {
> if (limiter.isBypass()) continue;
> limiter.checkQuota(writeConsumed, readConsumed);
> readAvailable = Math.min(readAvailable, limiter.getReadAvailable());
> writeAvailable = Math.min(writeAvailable, limiter.getWriteAvailable());
> }
> for (final QuotaLimiter limiter : limiters) {
> limiter.grabQuota(writeConsumed, readConsumed);
> }
> }{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20482) Print a link to the ref guide chapter for the shell during startup
[ https://issues.apache.org/jira/browse/HBASE-20482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454902#comment-16454902 ] Sakthi commented on HBASE-20482: Oh yes. In that case, we have 2 other ref guides(i.e. for 2.0 & 1.2, if I'm not wrong). So we need 2 other branch-specific patches with the link pointing towards the respective ref guides, [~busbey] ? > Print a link to the ref guide chapter for the shell during startup > -- > > Key: HBASE-20482 > URL: https://issues.apache.org/jira/browse/HBASE-20482 > Project: HBase > Issue Type: Task > Components: documentation, shell >Reporter: Sakthi >Assignee: Sakthi >Priority: Minor > Attachments: hbase-20482.master.001.patch, > hbase-20482.master.002.patch, hbase-20482.master.003.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20431) Store commit transaction for filesystems that do not support an atomic rename
[ https://issues.apache.org/jira/browse/HBASE-20431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454856#comment-16454856 ] Andrew Purtell commented on HBASE-20431: bq. S3Guard should give you a reliable listing of objects, and Get-After-Put is a documented guarantee in Amazon S3 these days, but do be aware that if you request an object before it exists, it might still appear to not exist after it's written. Thanks. That's a bit subtle so adding this comment to make sure we account for it. > Store commit transaction for filesystems that do not support an atomic rename > - > > Key: HBASE-20431 > URL: https://issues.apache.org/jira/browse/HBASE-20431 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Priority: Major > > HBase expects the Hadoop filesystem implementation to support an atomic > rename() operation. HDFS does. The S3 backed filesystems do not. The > fundamental issue is the non-atomic and eventually consistent nature of the > S3 service. A S3 bucket is not a filesystem. S3 is not always immediately > read-your-writes. Object metadata can be temporarily inconsistent just after > new objects are stored. There can be a settling period to ride over. > Renaming/moving objects from one path to another are copy operations with > O(file) complexity and O(data) time followed by a series of deletes with > O(file) complexity. Failures at any point prior to completion will leave the > operation in an inconsistent state. The missing atomic rename semantic opens > opportunities for corruption and data loss, which may or may not be > repairable with HBCK. > Handling this at the HBase level could be done with a new multi-step > filesystem transaction framework. Call it StoreCommitTransaction. > SplitTransaction and MergeTransaction are well established cases where even > on HDFS we have non-atomic filesystem changes and are our implementation > template for the new work. In this new StoreCommitTransaction we'd be moving > flush and compaction temporaries out of the temporary directory into the > region store directory. On HDFS the implementation would be easy. We can rely > on the filesystem's atomic rename semantics. On S3 it would be work: First we > would build the list of objects to move, then copy each object into the > destination, and then finally delete all objects at the original path. We > must handle transient errors with retry strategies appropriate for the action > at hand. We must handle serious or permanent errors where the RS doesn't need > to be aborted with a rollback that cleans it all up. Finally, we must handle > permanent errors where the RS must be aborted with a rollback during region > open/recovery. Note that after all objects have been copied and we are > deleting obsolete source objects we must roll forward, not back. To support > recovery after an abort we must utilize the WAL to track transaction > progress. Put markers in for StoreCommitTransaction start and completion > state, with details of the store file(s) involved, so it can be rolled back > during region recovery at open. This will be significant work in HFile, > HStore, flusher, compactor, and HRegion. Wherever we use HDFS's rename now we > would substitute the running of this new multi-step filesystem transaction. > We need to determine this for certain, but I believe on S3 the PUT or > multipart upload of an object must complete before the object is visible, so > we don't have to worry about the case where an object is visible before fully > uploaded as part of normal operations. So an individual object copy will > either happen entirely and the target will then become visible, or it won't > and the target won't exist. > S3 has an optimization, PUT COPY > (https://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectCOPY.html), which > the AmazonClient embedded in S3A utilizes for moves. When designing the > StoreCommitTransaction be sure to allow for filesystem implementations that > leverage a server side copy operation. Doing a get-then-put should be > optional. (Not sure Hadoop has an interface that advertises this capability > yet; we can add one if not.) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20431) Store commit transaction for filesystems that do not support an atomic rename
[
https://issues.apache.org/jira/browse/HBASE-20431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454836#comment-16454836
]
Sean Mackrory edited comment on HBASE-20431 at 4/26/18 8:39 PM:
{quote}we can, but it'd restrict the code to requiring a DDB, which would make
the WDC and Ceph groups sad. I think Andrew could get by without it, if a
single file is all that's needed for the commit.{quote}
It could be an extension to the MetadataStore interface that can be ported to
any implementation, but yeah - I only bring this up for completeness of
discussion, since I've thought about the problem a lot. If different
applications depend on different characteristics of a true atomic rename, you'd
need different rename strategies to support all of them, so if the application
can avoid the problem entirely by depending on different semantics that's
probably better.
{quote}I think S3Guard may still be needed on AWS to ensure that once an object
has become visible it remains visible, right? When enumerating a bucket we need
to get back a list of committed objects aka "files" that always includes
everything that has been committed.{quote}
S3Guard should give you a reliable listing of objects, and Get-After-Put is a
documented guarantee in Amazon S3 these days, but do be aware that if you
request an object before it exists, it might still appear to not exist after
it's written (I think of this as Get-After-Put-After-Get consistency). There's
nothing S3Guard could do about that without actually duplicating the data
itself - the best we can do is block requests for objects that S3Guard doesn't
know exist (which would require what we call "authoritative mode" to be
implemented and some other work) or if you don't request anything you don't
discover by listing then you don't have that problem.
was (Author: mackrorysd):
{quote}we can, but it'd restrict the code to requiring a DDB, which would make
the WDC and Ceph groups sad. I think Andrew could get by without it, if a
single file is all that's needed for the commit.{quote}
It could be an extension to the MetadataStore interface that can be ported to
any implementation, but yeah - I only bring this up for completeness of
discussion, since I've thought about the problem a lot. If different
applications depend on different characteristics of a true atomic rename, you'd
need different rename strategies to support all of them, so if the application
can avoid the problem entirely by depending on different semantics that's
probably better.
{quote}I think S3Guard may still be needed on AWS to ensure that once an object
has become visible it remains visible, right? When enumerating a bucket we need
to get back a list of committed objects aka "files" that always includes
everything that has been committed.{quote}
S3Guard should give you a reliable listing of objects, and Get-After-Put is a
documented guarantee in Amazon S3 these days, but do be aware that if you
request an object before it exists, it might still appear to not exist after
it's written. There's nothing S3Guard could do about that without actually
duplicating the data itself - the best we can do is block requests for objects
that S3Guard doesn't know exist (which would require what we call
"authoritative mode" to be implemented and some other work) or if you don't
request anything you don't discover by listing then you don't have that problem.
> Store commit transaction for filesystems that do not support an atomic rename
> -
>
> Key: HBASE-20431
> URL: https://issues.apache.org/jira/browse/HBASE-20431
> Project: HBase
> Issue Type: Sub-task
>Reporter: Andrew Purtell
>Priority: Major
>
> HBase expects the Hadoop filesystem implementation to support an atomic
> rename() operation. HDFS does. The S3 backed filesystems do not. The
> fundamental issue is the non-atomic and eventually consistent nature of the
> S3 service. A S3 bucket is not a filesystem. S3 is not always immediately
> read-your-writes. Object metadata can be temporarily inconsistent just after
> new objects are stored. There can be a settling period to ride over.
> Renaming/moving objects from one path to another are copy operations with
> O(file) complexity and O(data) time followed by a series of deletes with
> O(file) complexity. Failures at any point prior to completion will leave the
> operation in an inconsistent state. The missing atomic rename semantic opens
> opportunities for corruption and data loss, which may or may not be
> repairable with HBCK.
> Handling this at the HBase level could be done with a new multi-step
> filesystem transaction framework. Call it StoreCommitTransaction.
> SplitTransaction and MergeTransaction are well
[jira] [Commented] (HBASE-20431) Store commit transaction for filesystems that do not support an atomic rename
[
https://issues.apache.org/jira/browse/HBASE-20431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454836#comment-16454836
]
Sean Mackrory commented on HBASE-20431:
---
{quote}we can, but it'd restrict the code to requiring a DDB, which would make
the WDC and Ceph groups sad. I think Andrew could get by without it, if a
single file is all that's needed for the commit.{quote}
It could be an extension to the MetadataStore interface that can be ported to
any implementation, but yeah - I only bring this up for completeness of
discussion, since I've thought about the problem a lot. If different
applications depend on different characteristics of a true atomic rename, you'd
need different rename strategies to support all of them, so if the application
can avoid the problem entirely by depending on different semantics that's
probably better.
{quote}I think S3Guard may still be needed on AWS to ensure that once an object
has become visible it remains visible, right? When enumerating a bucket we need
to get back a list of committed objects aka "files" that always includes
everything that has been committed.{quote}
S3Guard should give you a reliable listing of objects, and Get-After-Put is a
documented guarantee in Amazon S3 these days, but do be aware that if you
request an object before it exists, it might still appear to not exist after
it's written. There's nothing S3Guard could do about that without actually
duplicating the data itself - the best we can do is block requests for objects
that S3Guard doesn't know exist (which would require what we call
"authoritative mode" to be implemented and some other work) or if you don't
request anything you don't discover by listing then you don't have that problem.
> Store commit transaction for filesystems that do not support an atomic rename
> -
>
> Key: HBASE-20431
> URL: https://issues.apache.org/jira/browse/HBASE-20431
> Project: HBase
> Issue Type: Sub-task
>Reporter: Andrew Purtell
>Priority: Major
>
> HBase expects the Hadoop filesystem implementation to support an atomic
> rename() operation. HDFS does. The S3 backed filesystems do not. The
> fundamental issue is the non-atomic and eventually consistent nature of the
> S3 service. A S3 bucket is not a filesystem. S3 is not always immediately
> read-your-writes. Object metadata can be temporarily inconsistent just after
> new objects are stored. There can be a settling period to ride over.
> Renaming/moving objects from one path to another are copy operations with
> O(file) complexity and O(data) time followed by a series of deletes with
> O(file) complexity. Failures at any point prior to completion will leave the
> operation in an inconsistent state. The missing atomic rename semantic opens
> opportunities for corruption and data loss, which may or may not be
> repairable with HBCK.
> Handling this at the HBase level could be done with a new multi-step
> filesystem transaction framework. Call it StoreCommitTransaction.
> SplitTransaction and MergeTransaction are well established cases where even
> on HDFS we have non-atomic filesystem changes and are our implementation
> template for the new work. In this new StoreCommitTransaction we'd be moving
> flush and compaction temporaries out of the temporary directory into the
> region store directory. On HDFS the implementation would be easy. We can rely
> on the filesystem's atomic rename semantics. On S3 it would be work: First we
> would build the list of objects to move, then copy each object into the
> destination, and then finally delete all objects at the original path. We
> must handle transient errors with retry strategies appropriate for the action
> at hand. We must handle serious or permanent errors where the RS doesn't need
> to be aborted with a rollback that cleans it all up. Finally, we must handle
> permanent errors where the RS must be aborted with a rollback during region
> open/recovery. Note that after all objects have been copied and we are
> deleting obsolete source objects we must roll forward, not back. To support
> recovery after an abort we must utilize the WAL to track transaction
> progress. Put markers in for StoreCommitTransaction start and completion
> state, with details of the store file(s) involved, so it can be rolled back
> during region recovery at open. This will be significant work in HFile,
> HStore, flusher, compactor, and HRegion. Wherever we use HDFS's rename now we
> would substitute the running of this new multi-step filesystem transaction.
> We need to determine this for certain, but I believe on S3 the PUT or
> multipart upload of an object must complete before the object is visible, so
> we don't have to worry about the case where an object is
[jira] [Commented] (HBASE-20495) REST unit test fails with NoClassDefFoundError against hadoop3
[
https://issues.apache.org/jira/browse/HBASE-20495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454827#comment-16454827
]
Ted Yu commented on HBASE-20495:
[~busbey]:
Do you have other comment ?
> REST unit test fails with NoClassDefFoundError against hadoop3
> --
>
> Key: HBASE-20495
> URL: https://issues.apache.org/jira/browse/HBASE-20495
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 20495.v1.txt
>
>
> The following was first observed in the test output of rest.TestDeleteRow
> against hadoop3:
> {code}
> java.lang.NoClassDefFoundError:
> com/sun/jersey/core/spi/factory/AbstractRuntimeDelegate
> Caused by: java.lang.ClassNotFoundException:
> com.sun.jersey.core.spi.factory.AbstractRuntimeDelegate
> {code}
> This was due to the following transitive dependency on jersey 1.19:
> {code}
> [INFO] +- org.apache.hbase:hbase-testing-util:jar:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-zookeeper:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-hadoop-compat:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-hadoop2-compat:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:3.0.0:compile
> [INFO] | | \-
> org.apache.hadoop:hadoop-mapreduce-client-common:jar:3.0.0:compile
> [INFO] | +- org.apache.hadoop:hadoop-hdfs:test-jar:tests:3.0.0:test
> [INFO] | | \- com.sun.jersey:jersey-server:jar:1.19:compile
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20229) ConnectionImplementation.locateRegions() returns duplicated entries when region replication is on
[
https://issues.apache.org/jira/browse/HBASE-20229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454791#comment-16454791
]
Hudson commented on HBASE-20229:
Results for branch branch-1.4
[build #302 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/302/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/302//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/302//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/302//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> ConnectionImplementation.locateRegions() returns duplicated entries when
> region replication is on
> -
>
> Key: HBASE-20229
> URL: https://issues.apache.org/jira/browse/HBASE-20229
> Project: HBase
> Issue Type: Bug
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: HBASE-20229.branch-1.001.patch,
> HBASE-20229.branch-1.002.patch, HBASE-20229.branch-1.002.patch,
> HBASE-20229.master.001.patch, HBASE-20229.master.002.patch,
> HBASE-20229.master.003.patch, HBASE-20229.master.004.patch
>
>
> The issue is when region replication is on,
> ConnectionImplementation.locateRegions() returns duplicated entries.
> In the test in my patch, the table should have 1 primary region + 2
> secondaries, but ConnectionImplementation.locateRegions() returns 9 regions.
> Every region repeats 3 times (3 = replicas count).
> I think this is because the following code calls locateRegion() even for
> replica regions and then the result triples.
> {code:java}
> for (RegionInfo regionInfo : regions) {
> RegionLocations list = locateRegion(tableName,
> regionInfo.getStartKey(), useCache, true);
> if (list != null) {
> for (HRegionLocation loc : list.getRegionLocations()) {
> if (loc != null) {
> locations.add(loc);
> }
> }
> }
> {code}
> The fix in my patch is to make it call locateRegion() only for a primary
> region.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18842) The hbase shell clone_snaphost command returns bad error message
[
https://issues.apache.org/jira/browse/HBASE-18842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454792#comment-16454792
]
Hudson commented on HBASE-18842:
Results for branch branch-1.4
[build #302 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/302/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/302//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/302//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/302//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> The hbase shell clone_snaphost command returns bad error message
>
>
> Key: HBASE-18842
> URL: https://issues.apache.org/jira/browse/HBASE-18842
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Thoralf Gutierrez
>Assignee: Thoralf Gutierrez
>Priority: Minor
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 2.0.1, 1.4.5
>
> Attachments:
> 0001-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch,
> 0002-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch,
> 0003-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch,
> 0004-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch,
> 0005-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch
>
>
> When you call the hbase shell clone_snapshot command with a target namespace
> that doesn't exist, you get an error message, but the variable used to
> identify the inexistent namespace is wrong:
> {noformat}
> hbase(main):001:0> clone_snapshot 'someSnapshotName',
> 'someNamespaceName:someTableName'
> ERROR: Unknown namespace someSnapshotName!
> Create a new table by cloning the snapshot content.
> There're no copies of data involved.
> And writing on the newly created table will not influence the snapshot data.
> Examples:
> hbase> clone_snapshot 'snapshotName', 'tableName'
> hbase> clone_snapshot 'snapshotName', 'namespace:tableName'
> {noformat}
> It should rather say:
> {noformat}
> ERROR: Unknown namespace someNamespaceName!
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20431) Store commit transaction for filesystems that do not support an atomic rename
[ https://issues.apache.org/jira/browse/HBASE-20431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454775#comment-16454775 ] Andrew Purtell edited comment on HBASE-20431 at 4/26/18 8:03 PM: - I think S3Guard is still needed on AWS to ensure that once an object has become visible it remains visible, right? When enumerating a bucket we need to get back a list of committed objects aka "files" that always includes everything that has been committed. Sounds like Ceph RGW has saner semantics and since it is more likely we'd run HBase on Ceph than HBase on AWS, although both are in design scope, this is heartening. was (Author: apurtell): I think S3Guard is still needed on AWS to ensure that once an object has become visible it remains visible, right? Sounds like Ceph RGW has saner semantics and since it is more likely we'd run HBase on Ceph than HBase on AWS, although both are in design scope, this is heartening. > Store commit transaction for filesystems that do not support an atomic rename > - > > Key: HBASE-20431 > URL: https://issues.apache.org/jira/browse/HBASE-20431 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Priority: Major > > HBase expects the Hadoop filesystem implementation to support an atomic > rename() operation. HDFS does. The S3 backed filesystems do not. The > fundamental issue is the non-atomic and eventually consistent nature of the > S3 service. A S3 bucket is not a filesystem. S3 is not always immediately > read-your-writes. Object metadata can be temporarily inconsistent just after > new objects are stored. There can be a settling period to ride over. > Renaming/moving objects from one path to another are copy operations with > O(file) complexity and O(data) time followed by a series of deletes with > O(file) complexity. Failures at any point prior to completion will leave the > operation in an inconsistent state. The missing atomic rename semantic opens > opportunities for corruption and data loss, which may or may not be > repairable with HBCK. > Handling this at the HBase level could be done with a new multi-step > filesystem transaction framework. Call it StoreCommitTransaction. > SplitTransaction and MergeTransaction are well established cases where even > on HDFS we have non-atomic filesystem changes and are our implementation > template for the new work. In this new StoreCommitTransaction we'd be moving > flush and compaction temporaries out of the temporary directory into the > region store directory. On HDFS the implementation would be easy. We can rely > on the filesystem's atomic rename semantics. On S3 it would be work: First we > would build the list of objects to move, then copy each object into the > destination, and then finally delete all objects at the original path. We > must handle transient errors with retry strategies appropriate for the action > at hand. We must handle serious or permanent errors where the RS doesn't need > to be aborted with a rollback that cleans it all up. Finally, we must handle > permanent errors where the RS must be aborted with a rollback during region > open/recovery. Note that after all objects have been copied and we are > deleting obsolete source objects we must roll forward, not back. To support > recovery after an abort we must utilize the WAL to track transaction > progress. Put markers in for StoreCommitTransaction start and completion > state, with details of the store file(s) involved, so it can be rolled back > during region recovery at open. This will be significant work in HFile, > HStore, flusher, compactor, and HRegion. Wherever we use HDFS's rename now we > would substitute the running of this new multi-step filesystem transaction. > We need to determine this for certain, but I believe on S3 the PUT or > multipart upload of an object must complete before the object is visible, so > we don't have to worry about the case where an object is visible before fully > uploaded as part of normal operations. So an individual object copy will > either happen entirely and the target will then become visible, or it won't > and the target won't exist. > S3 has an optimization, PUT COPY > (https://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectCOPY.html), which > the AmazonClient embedded in S3A utilizes for moves. When designing the > StoreCommitTransaction be sure to allow for filesystem implementations that > leverage a server side copy operation. Doing a get-then-put should be > optional. (Not sure Hadoop has an interface that advertises this capability > yet; we can add one if not.) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20431) Store commit transaction for filesystems that do not support an atomic rename
[ https://issues.apache.org/jira/browse/HBASE-20431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454775#comment-16454775 ] Andrew Purtell edited comment on HBASE-20431 at 4/26/18 8:03 PM: - I think S3Guard may still be needed on AWS to ensure that once an object has become visible it remains visible, right? When enumerating a bucket we need to get back a list of committed objects aka "files" that always includes everything that has been committed. Sounds like Ceph RGW has saner semantics and since it is more likely we'd run HBase on Ceph than HBase on AWS, although both are in design scope, this is heartening. was (Author: apurtell): I think S3Guard is still needed on AWS to ensure that once an object has become visible it remains visible, right? When enumerating a bucket we need to get back a list of committed objects aka "files" that always includes everything that has been committed. Sounds like Ceph RGW has saner semantics and since it is more likely we'd run HBase on Ceph than HBase on AWS, although both are in design scope, this is heartening. > Store commit transaction for filesystems that do not support an atomic rename > - > > Key: HBASE-20431 > URL: https://issues.apache.org/jira/browse/HBASE-20431 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Priority: Major > > HBase expects the Hadoop filesystem implementation to support an atomic > rename() operation. HDFS does. The S3 backed filesystems do not. The > fundamental issue is the non-atomic and eventually consistent nature of the > S3 service. A S3 bucket is not a filesystem. S3 is not always immediately > read-your-writes. Object metadata can be temporarily inconsistent just after > new objects are stored. There can be a settling period to ride over. > Renaming/moving objects from one path to another are copy operations with > O(file) complexity and O(data) time followed by a series of deletes with > O(file) complexity. Failures at any point prior to completion will leave the > operation in an inconsistent state. The missing atomic rename semantic opens > opportunities for corruption and data loss, which may or may not be > repairable with HBCK. > Handling this at the HBase level could be done with a new multi-step > filesystem transaction framework. Call it StoreCommitTransaction. > SplitTransaction and MergeTransaction are well established cases where even > on HDFS we have non-atomic filesystem changes and are our implementation > template for the new work. In this new StoreCommitTransaction we'd be moving > flush and compaction temporaries out of the temporary directory into the > region store directory. On HDFS the implementation would be easy. We can rely > on the filesystem's atomic rename semantics. On S3 it would be work: First we > would build the list of objects to move, then copy each object into the > destination, and then finally delete all objects at the original path. We > must handle transient errors with retry strategies appropriate for the action > at hand. We must handle serious or permanent errors where the RS doesn't need > to be aborted with a rollback that cleans it all up. Finally, we must handle > permanent errors where the RS must be aborted with a rollback during region > open/recovery. Note that after all objects have been copied and we are > deleting obsolete source objects we must roll forward, not back. To support > recovery after an abort we must utilize the WAL to track transaction > progress. Put markers in for StoreCommitTransaction start and completion > state, with details of the store file(s) involved, so it can be rolled back > during region recovery at open. This will be significant work in HFile, > HStore, flusher, compactor, and HRegion. Wherever we use HDFS's rename now we > would substitute the running of this new multi-step filesystem transaction. > We need to determine this for certain, but I believe on S3 the PUT or > multipart upload of an object must complete before the object is visible, so > we don't have to worry about the case where an object is visible before fully > uploaded as part of normal operations. So an individual object copy will > either happen entirely and the target will then become visible, or it won't > and the target won't exist. > S3 has an optimization, PUT COPY > (https://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectCOPY.html), which > the AmazonClient embedded in S3A utilizes for moves. When designing the > StoreCommitTransaction be sure to allow for filesystem implementations that > leverage a server side copy operation. Doing a get-then-put should be > optional. (Not sure Hadoop has an interface that advertises this capability > yet; we
[jira] [Commented] (HBASE-20431) Store commit transaction for filesystems that do not support an atomic rename
[ https://issues.apache.org/jira/browse/HBASE-20431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454775#comment-16454775 ] Andrew Purtell commented on HBASE-20431: I think S3Guard is still needed on AWS to ensure that once an object has become visible it remains visible, right? Sounds like Ceph RGW has saner semantics and since it is more likely we'd run HBase on Ceph than HBase on AWS, although both are in design scope, this is heartening. > Store commit transaction for filesystems that do not support an atomic rename > - > > Key: HBASE-20431 > URL: https://issues.apache.org/jira/browse/HBASE-20431 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Priority: Major > > HBase expects the Hadoop filesystem implementation to support an atomic > rename() operation. HDFS does. The S3 backed filesystems do not. The > fundamental issue is the non-atomic and eventually consistent nature of the > S3 service. A S3 bucket is not a filesystem. S3 is not always immediately > read-your-writes. Object metadata can be temporarily inconsistent just after > new objects are stored. There can be a settling period to ride over. > Renaming/moving objects from one path to another are copy operations with > O(file) complexity and O(data) time followed by a series of deletes with > O(file) complexity. Failures at any point prior to completion will leave the > operation in an inconsistent state. The missing atomic rename semantic opens > opportunities for corruption and data loss, which may or may not be > repairable with HBCK. > Handling this at the HBase level could be done with a new multi-step > filesystem transaction framework. Call it StoreCommitTransaction. > SplitTransaction and MergeTransaction are well established cases where even > on HDFS we have non-atomic filesystem changes and are our implementation > template for the new work. In this new StoreCommitTransaction we'd be moving > flush and compaction temporaries out of the temporary directory into the > region store directory. On HDFS the implementation would be easy. We can rely > on the filesystem's atomic rename semantics. On S3 it would be work: First we > would build the list of objects to move, then copy each object into the > destination, and then finally delete all objects at the original path. We > must handle transient errors with retry strategies appropriate for the action > at hand. We must handle serious or permanent errors where the RS doesn't need > to be aborted with a rollback that cleans it all up. Finally, we must handle > permanent errors where the RS must be aborted with a rollback during region > open/recovery. Note that after all objects have been copied and we are > deleting obsolete source objects we must roll forward, not back. To support > recovery after an abort we must utilize the WAL to track transaction > progress. Put markers in for StoreCommitTransaction start and completion > state, with details of the store file(s) involved, so it can be rolled back > during region recovery at open. This will be significant work in HFile, > HStore, flusher, compactor, and HRegion. Wherever we use HDFS's rename now we > would substitute the running of this new multi-step filesystem transaction. > We need to determine this for certain, but I believe on S3 the PUT or > multipart upload of an object must complete before the object is visible, so > we don't have to worry about the case where an object is visible before fully > uploaded as part of normal operations. So an individual object copy will > either happen entirely and the target will then become visible, or it won't > and the target won't exist. > S3 has an optimization, PUT COPY > (https://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectCOPY.html), which > the AmazonClient embedded in S3A utilizes for moves. When designing the > StoreCommitTransaction be sure to allow for filesystem implementations that > leverage a server side copy operation. Doing a get-then-put should be > optional. (Not sure Hadoop has an interface that advertises this capability > yet; we can add one if not.) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20495) REST unit test fails with NoClassDefFoundError against hadoop3
[
https://issues.apache.org/jira/browse/HBASE-20495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454733#comment-16454733
]
Josh Elser commented on HBASE-20495:
{quote}So, maybe add exclusion in another JIRA if there is such need ?
{quote}
Ah, so we do have that same dependency coming in for hbase-thrift from Hadoop,
but it's not presently causing us any troubles (likely, because we're not
exercising those code paths in the same manner). Assuming a little bit since
the description doesn't the full stacktrace from TestDeleteRow.
Given that, +1 from me.
> REST unit test fails with NoClassDefFoundError against hadoop3
> --
>
> Key: HBASE-20495
> URL: https://issues.apache.org/jira/browse/HBASE-20495
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 20495.v1.txt
>
>
> The following was first observed in the test output of rest.TestDeleteRow
> against hadoop3:
> {code}
> java.lang.NoClassDefFoundError:
> com/sun/jersey/core/spi/factory/AbstractRuntimeDelegate
> Caused by: java.lang.ClassNotFoundException:
> com.sun.jersey.core.spi.factory.AbstractRuntimeDelegate
> {code}
> This was due to the following transitive dependency on jersey 1.19:
> {code}
> [INFO] +- org.apache.hbase:hbase-testing-util:jar:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-zookeeper:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-hadoop-compat:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-hadoop2-compat:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:3.0.0:compile
> [INFO] | | \-
> org.apache.hadoop:hadoop-mapreduce-client-common:jar:3.0.0:compile
> [INFO] | +- org.apache.hadoop:hadoop-hdfs:test-jar:tests:3.0.0:test
> [INFO] | | \- com.sun.jersey:jersey-server:jar:1.19:compile
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20433) HBase Export Snapshot utility does not close FileSystem instances
[ https://issues.apache.org/jira/browse/HBASE-20433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454717#comment-16454717 ] Sean Mackrory commented on HBASE-20433: --- I debugged a run of ExportSnapshot in connection with HADOOP-15392. In my case I see an FS instance get created for the snapshot directory (e.g. s3a://my-bucket/hbase-snapshots) and another FS instance get created for the specific snapshot (e.g. s3a://my-bucket/hbase-snapshots/snapshot-name-8015e164-781d-4187-afa1-bcc25541ba9e). The latter is the one where the FS instance is turned off. They also appear to get closed and the metrics system stopped when second one is closed too. There are ~100 files in the snapshot and I just got the 2 FS instances you'd expect, so I don't see how a 53k-file snapshot is resulting in 53k FS instances in the driver. > HBase Export Snapshot utility does not close FileSystem instances > - > > Key: HBASE-20433 > URL: https://issues.apache.org/jira/browse/HBASE-20433 > Project: HBase > Issue Type: Bug > Components: Client, fs, snapshots >Affects Versions: 1.2.6, 1.4.3 >Reporter: Voyta >Priority: Major > > It seems org.apache.hadoop.hbase.snapshot.ExportSnapshot disallows FileSystem > instance caching. > When verifySnapshot method is being run it calls often methods like > org.apache.hadoop.hbase.util.FSUtils#getRootDir that instantiate FileSystem > but never calls org.apache.hadoop.fs.FileSystem#close method. This behaviour > allows allocation of unwanted objects potentially causing memory leaks. > Related issue: https://issues.apache.org/jira/browse/HADOOP-15392 > > Expectation: > * HBase should properly release/close all objects, especially FileSystem > instances. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20332) shaded mapreduce module shouldn't include hadoop
[
https://issues.apache.org/jira/browse/HBASE-20332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454686#comment-16454686
]
Sean Busbey commented on HBASE-20332:
-
given the current status of checkstyle based check in HBASE-20478, is everyone
fine with me punting the check for old hbase apis to a follow-on that's blocked
on the checkstyle update needed for the "illegal classes" rule?
> shaded mapreduce module shouldn't include hadoop
>
>
> Key: HBASE-20332
> URL: https://issues.apache.org/jira/browse/HBASE-20332
> Project: HBase
> Issue Type: Sub-task
> Components: mapreduce, shading
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-20332.0.patch, HBASE-20332.1.WIP.patch,
> HBASE-20332.2.WIP.patch, HBASE-20332.3.patch
>
>
> AFAICT, we should just entirely skip including hadoop in our shaded mapreduce
> module
> 1) Folks expect to run yarn / mr apps via {{hadoop jar}} / {{yarn jar}}
> 2) those commands include all the needed Hadoop jars in your classpath by
> default (both client side and in the containers)
> 3) If you try to use "user classpath first" for your job as a workaround
> (e.g. for some library your application needs that hadoop provides) then our
> inclusion of *some but not all* hadoop classes then causes everything to fall
> over because of mixing rewritten and non-rewritten hadoop classes
> 4) if you don't use "user classpath first" then all of our
> non-relocated-but-still-shaded hadoop classes are ignored anyways so we're
> just wasting space
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20478) move import checks from hbaseanti to checkstyle
[ https://issues.apache.org/jira/browse/HBASE-20478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-20478: Status: In Progress (was: Patch Available) > move import checks from hbaseanti to checkstyle > --- > > Key: HBASE-20478 > URL: https://issues.apache.org/jira/browse/HBASE-20478 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Sean Busbey >Assignee: Mike Drob >Priority: Minor > Attachments: HBASE-20478.0.patch, HBASE-20478.1.patch, > HBASE-20478.2.patch, HBASE-20478.3.patch, HBASE-20478.WIP.2.patch, > HBASE-20478.WIP.2.patch, HBASE-20478.WIP.patch, HBASE-anti-check.patch > > > came up in discussion on HBASE-20332. our check of "don't do this" things in > the codebase doesn't log the specifics of complaints anywhere, which forces > those who want to follow up to reverse engineer the check. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20478) move import checks from hbaseanti to checkstyle
[ https://issues.apache.org/jira/browse/HBASE-20478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454675#comment-16454675 ] Mike Drob commented on HBASE-20478: --- I reverted the change for now until we figure out the proper checkstyle version > move import checks from hbaseanti to checkstyle > --- > > Key: HBASE-20478 > URL: https://issues.apache.org/jira/browse/HBASE-20478 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Sean Busbey >Assignee: Mike Drob >Priority: Minor > Attachments: HBASE-20478.0.patch, HBASE-20478.1.patch, > HBASE-20478.2.patch, HBASE-20478.3.patch, HBASE-20478.WIP.2.patch, > HBASE-20478.WIP.2.patch, HBASE-20478.WIP.patch, HBASE-anti-check.patch > > > came up in discussion on HBASE-20332. our check of "don't do this" things in > the codebase doesn't log the specifics of complaints anywhere, which forces > those who want to follow up to reverse engineer the check. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20478) move import checks from hbaseanti to checkstyle
[ https://issues.apache.org/jira/browse/HBASE-20478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454672#comment-16454672 ] Sean Busbey edited comment on HBASE-20478 at 4/26/18 6:35 PM: -- let's pick one and use it as an addendum. you prefer patch or review? was (Author: busbey): let's pick one and use it an addendum. you prefer patch or review? > move import checks from hbaseanti to checkstyle > --- > > Key: HBASE-20478 > URL: https://issues.apache.org/jira/browse/HBASE-20478 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Sean Busbey >Assignee: Mike Drob >Priority: Minor > Attachments: HBASE-20478.0.patch, HBASE-20478.1.patch, > HBASE-20478.2.patch, HBASE-20478.3.patch, HBASE-20478.WIP.2.patch, > HBASE-20478.WIP.2.patch, HBASE-20478.WIP.patch, HBASE-anti-check.patch > > > came up in discussion on HBASE-20332. our check of "don't do this" things in > the codebase doesn't log the specifics of complaints anywhere, which forces > those who want to follow up to reverse engineer the check. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20478) move import checks from hbaseanti to checkstyle
[ https://issues.apache.org/jira/browse/HBASE-20478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454672#comment-16454672 ] Sean Busbey commented on HBASE-20478: - let's pick one and use it an addendum. you prefer patch or review? > move import checks from hbaseanti to checkstyle > --- > > Key: HBASE-20478 > URL: https://issues.apache.org/jira/browse/HBASE-20478 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Sean Busbey >Assignee: Mike Drob >Priority: Minor > Attachments: HBASE-20478.0.patch, HBASE-20478.1.patch, > HBASE-20478.2.patch, HBASE-20478.3.patch, HBASE-20478.WIP.2.patch, > HBASE-20478.WIP.2.patch, HBASE-20478.WIP.patch, HBASE-anti-check.patch > > > came up in discussion on HBASE-20332. our check of "don't do this" things in > the codebase doesn't log the specifics of complaints anywhere, which forces > those who want to follow up to reverse engineer the check. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20478) move import checks from hbaseanti to checkstyle
[
https://issues.apache.org/jira/browse/HBASE-20478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454671#comment-16454671
]
Mike Drob commented on HBASE-20478:
---
you're right, running locally I get:
{noformat}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-checkstyle-plugin:2.17:checkstyle (default-cli)
on project hbase-server: An error has occurred in Checkstyle report
generation.: Failed during checkstyle configuration: cannot initialize module
TreeWalker - Property 'illegalClasses' in module IllegalImport does not exist,
please check the documentation -> [Help 1]
{noformat}
The illegalClasses element was added in Checkstyle 7.8 -
http://checkstyle.sourceforge.net/config_imports.html#IllegalImport
By default the checkstyle plugin uses Checkstyle 6.18 -
https://maven.apache.org/plugins/maven-checkstyle-plugin/
I think we can try upgrading it to a newer version -
https://maven.apache.org/plugins/maven-checkstyle-plugin/examples/upgrading-checkstyle.html
There's a lot of newer options out there -
http://central.maven.org/maven2/com/puppycrawl/tools/checkstyle/
> move import checks from hbaseanti to checkstyle
> ---
>
> Key: HBASE-20478
> URL: https://issues.apache.org/jira/browse/HBASE-20478
> Project: HBase
> Issue Type: Improvement
> Components: test
>Reporter: Sean Busbey
>Assignee: Mike Drob
>Priority: Minor
> Attachments: HBASE-20478.0.patch, HBASE-20478.1.patch,
> HBASE-20478.2.patch, HBASE-20478.3.patch, HBASE-20478.WIP.2.patch,
> HBASE-20478.WIP.2.patch, HBASE-20478.WIP.patch, HBASE-anti-check.patch
>
>
> came up in discussion on HBASE-20332. our check of "don't do this" things in
> the codebase doesn't log the specifics of complaints anywhere, which forces
> those who want to follow up to reverse engineer the check.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20478) move import checks from hbaseanti to checkstyle
[ https://issues.apache.org/jira/browse/HBASE-20478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454643#comment-16454643 ] Sean Busbey commented on HBASE-20478: - I pushed this to master, but in backporting ran into a problem where I can't make it fail. I think that's also true on master, so I might revert. here's a patch for QAbot that inserts each of the things we were checking for before htis patch. > move import checks from hbaseanti to checkstyle > --- > > Key: HBASE-20478 > URL: https://issues.apache.org/jira/browse/HBASE-20478 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Sean Busbey >Assignee: Mike Drob >Priority: Minor > Attachments: HBASE-20478.0.patch, HBASE-20478.1.patch, > HBASE-20478.2.patch, HBASE-20478.3.patch, HBASE-20478.WIP.2.patch, > HBASE-20478.WIP.2.patch, HBASE-20478.WIP.patch, HBASE-anti-check.patch > > > came up in discussion on HBASE-20332. our check of "don't do this" things in > the codebase doesn't log the specifics of complaints anywhere, which forces > those who want to follow up to reverse engineer the check. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20478) move import checks from hbaseanti to checkstyle
[ https://issues.apache.org/jira/browse/HBASE-20478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-20478: Attachment: HBASE-anti-check.patch > move import checks from hbaseanti to checkstyle > --- > > Key: HBASE-20478 > URL: https://issues.apache.org/jira/browse/HBASE-20478 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Sean Busbey >Assignee: Mike Drob >Priority: Minor > Attachments: HBASE-20478.0.patch, HBASE-20478.1.patch, > HBASE-20478.2.patch, HBASE-20478.3.patch, HBASE-20478.WIP.2.patch, > HBASE-20478.WIP.2.patch, HBASE-20478.WIP.patch, HBASE-anti-check.patch > > > came up in discussion on HBASE-20332. our check of "don't do this" things in > the codebase doesn't log the specifics of complaints anywhere, which forces > those who want to follow up to reverse engineer the check. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20492) UnassignProcedure is stuck in retry loop on region stuck in OPENING state
[
https://issues.apache.org/jira/browse/HBASE-20492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454636#comment-16454636
]
stack commented on HBASE-20492:
---
I just ran into it. Failed update to hbase:meta (because of broken patch which
made it so region was wonky) so region in OPENING state. Then a disable
table comes in which wants to close the region... but we are in OPENING state.
AMv2 keeps scheduling procedures unless told otherwise. The UnassignProcedure
will throw the UnexpectedStateException forever given region is in OPENING.
* Let me check why we dropped the region lock -- probably because of exception
on original assign.
* If UnexpectedStateException, fail the Unassign or whatever procedure?
> UnassignProcedure is stuck in retry loop on region stuck in OPENING state
> -
>
> Key: HBASE-20492
> URL: https://issues.apache.org/jira/browse/HBASE-20492
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Affects Versions: 2.0.0
>Reporter: Umesh Agashe
>Priority: Critical
> Fix For: 2.0.1
>
>
> UnassignProcedure gets stuck in a retry loop for a region stuck in OPENING
> state. From logs:
> {code:java}
> 2018-04-25 15:59:53,825 WARN
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure:
> Retryable error trying to transition: pid=142564,
> state=RUNNABLE:REGION_TRANSITION_DISPATCH; UnassignProcedure
> table=IntegrationTestBigLinkedList_20180331004141,
> region=bd2fb2c7d39236c9b9085f350358df7c,
> server=vb1122.halxg.cloudera.com,22101,1522626198450; rit=OPENING,
> location=vb1122.halxg.cloudera.com,22101,1522626198450
> org.apache.hadoop.hbase.exceptions.UnexpectedStateException: Expected
> [SPLITTING, SPLIT, MERGING, OPEN, CLOSING] so could move to CLOSING but
> current state=OPENING
> at
> org.apache.hadoop.hbase.master.assignment.RegionStates$RegionStateNode.transitionState(RegionStates.java:158)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.markRegionAsClosing(AssignmentManager.java:1514)
> at
> org.apache.hadoop.hbase.master.assignment.UnassignProcedure.updateTransition(UnassignProcedure.java:179)
> at
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:309)
> at
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:85)
> at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1458)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1227)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1738)
> 2018-04-25 15:59:53,892 WARN
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure:
> Retryable error trying to transition: pid=142564,
> state=RUNNABLE:REGION_TRANSITION_DISPATCH; UnassignProcedure
> table=IntegrationTestBigLinkedList_20180331004141,
> region=bd2fb2c7d39236c9b9085f350358df7c,
> server=vb1122.halxg.cloudera.com,22101,1522626198450; rit=OPENING,
> location=vb1122.halxg.cloudera.com,22101,1522626198450
> org.apache.hadoop.hbase.exceptions.UnexpectedStateException: Expected
> [SPLITTING, SPLIT, MERGING, OPEN, CLOSING] so could move to CLOSING but
> current state=OPENING
> at
> org.apache.hadoop.hbase.master.assignment.RegionStates$RegionStateNode.transitionState(RegionStates.java:158)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.markRegionAsClosing(AssignmentManager.java:1514)
> at
> org.apache.hadoop.hbase.master.assignment.UnassignProcedure.updateTransition(UnassignProcedure.java:179)
> at
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:309)
> at
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:85)
> at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1458)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1227)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1738){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18379) SnapshotManager#checkSnapshotSupport() should better handle malfunctioning hdfs snapshot
[
https://issues.apache.org/jira/browse/HBASE-18379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454632#comment-16454632
]
Ted Yu commented on HBASE-18379:
This issue was observed when running on top of Islon
> SnapshotManager#checkSnapshotSupport() should better handle malfunctioning
> hdfs snapshot
>
>
> Key: HBASE-18379
> URL: https://issues.apache.org/jira/browse/HBASE-18379
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Priority: Major
> Labels: snapshot
>
> The following was observed by a customer which prevented master from coming
> up:
> {code}
> 2017-07-13 13:25:07,898 FATAL [xyz:16000.activeMasterManager] master.HMaster:
> Failed to become active master
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: Daily_Snapshot_Apps_2017-xx
> at org.apache.hadoop.fs.Path.initialize(Path.java:205)
> at org.apache.hadoop.fs.Path.(Path.java:171)
> at org.apache.hadoop.fs.Path.(Path.java:93)
> at
> org.apache.hadoop.hdfs.protocol.HdfsFileStatus.getFullPath(HdfsFileStatus.java:230)
> at
> org.apache.hadoop.hdfs.protocol.HdfsFileStatus.makeQualified(HdfsFileStatus.java:263)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:911)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.access$600(DistributedFileSystem.java:113)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$21.doCall(DistributedFileSystem.java:966)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$21.doCall(DistributedFileSystem.java:962)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:962)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1534)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1574)
> at
> org.apache.hadoop.hbase.master.snapshot.SnapshotManager.getCompletedSnapshots(SnapshotManager.java:206)
> at
> org.apache.hadoop.hbase.master.snapshot.SnapshotManager.checkSnapshotSupport(SnapshotManager.java:1011)
> at
> org.apache.hadoop.hbase.master.snapshot.SnapshotManager.initialize(SnapshotManager.java:1070)
> at
> org.apache.hadoop.hbase.procedure.MasterProcedureManagerHost.initialize(MasterProcedureManagerHost.java:50)
> at
> org.apache.hadoop.hbase.master.HMaster.initializeZKBasedSystemTrackers(HMaster.java:667)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:732)
> at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:213)
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1863)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.net.URISyntaxException: Relative path in absolute URI:
> Daily_Snapshot_Apps_2017-xx
> at java.net.URI.checkPath(URI.java:1823)
> at java.net.URI.(URI.java:745)
> at org.apache.hadoop.fs.Path.initialize(Path.java:202)
> {code}
> Turns out the exception can be reproduced using hdfs command line accessing
> .snapshot directory.
> SnapshotManager#checkSnapshotSupport() should better handle malfunctioning
> hdfs snapshot so that master starts up.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19924) hbase rpc throttling does not work for multi() with request count rater.
[
https://issues.apache.org/jira/browse/HBASE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
huaxiang sun updated HBASE-19924:
-
Resolution: Fixed
Fix Version/s: 1.5.0
Status: Resolved (was: Patch Available)
pushed the patch to branch-1.
> hbase rpc throttling does not work for multi() with request count rater.
>
>
> Key: HBASE-19924
> URL: https://issues.apache.org/jira/browse/HBASE-19924
> Project: HBase
> Issue Type: Bug
> Components: rpc
>Affects Versions: 0.16.0, 1.2.6
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Major
> Fix For: 1.5.0, 2.0.0
>
> Attachments: HBASE-19924-branch-1-v001.patch,
> HBASE-19924-master-v001.patch, HBASE-19924-master-v002.patch,
> HBASE-19924-master-v002.patch, HBASE-19924-master-v002.patch,
> HBASE-19924-master-v002.patch
>
>
> Basically, rpc throttling does not work for request count based rater for
> multi. for the following code, when it calls limiter's checkQuota(),
> numWrites/numReads is lost.
> {code:java}
> @Override
> public void checkQuota(int numWrites, int numReads, int numScans) throws
> ThrottlingException {
> writeConsumed = estimateConsume(OperationType.MUTATE, numWrites, 100);
> readConsumed = estimateConsume(OperationType.GET, numReads, 100);
> readConsumed += estimateConsume(OperationType.SCAN, numScans, 1000);
> writeAvailable = Long.MAX_VALUE;
> readAvailable = Long.MAX_VALUE;
> for (final QuotaLimiter limiter : limiters) {
> if (limiter.isBypass()) continue;
> limiter.checkQuota(writeConsumed, readConsumed);
> readAvailable = Math.min(readAvailable, limiter.getReadAvailable());
> writeAvailable = Math.min(writeAvailable, limiter.getWriteAvailable());
> }
> for (final QuotaLimiter limiter : limiters) {
> limiter.grabQuota(writeConsumed, readConsumed);
> }
> }{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20495) REST unit test fails with NoClassDefFoundError against hadoop3
[
https://issues.apache.org/jira/browse/HBASE-20495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454596#comment-16454596
]
Ted Yu commented on HBASE-20495:
bq. switching some parts over to the hadoop 3 client facing modules that shade
things
Hmm. Can be done in another issue.
> REST unit test fails with NoClassDefFoundError against hadoop3
> --
>
> Key: HBASE-20495
> URL: https://issues.apache.org/jira/browse/HBASE-20495
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 20495.v1.txt
>
>
> The following was first observed in the test output of rest.TestDeleteRow
> against hadoop3:
> {code}
> java.lang.NoClassDefFoundError:
> com/sun/jersey/core/spi/factory/AbstractRuntimeDelegate
> Caused by: java.lang.ClassNotFoundException:
> com.sun.jersey.core.spi.factory.AbstractRuntimeDelegate
> {code}
> This was due to the following transitive dependency on jersey 1.19:
> {code}
> [INFO] +- org.apache.hbase:hbase-testing-util:jar:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-zookeeper:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-hadoop-compat:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-hadoop2-compat:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:3.0.0:compile
> [INFO] | | \-
> org.apache.hadoop:hadoop-mapreduce-client-common:jar:3.0.0:compile
> [INFO] | +- org.apache.hadoop:hadoop-hdfs:test-jar:tests:3.0.0:test
> [INFO] | | \- com.sun.jersey:jersey-server:jar:1.19:compile
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20495) REST unit test fails with NoClassDefFoundError against hadoop3
[
https://issues.apache.org/jira/browse/HBASE-20495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454590#comment-16454590
]
Sean Busbey commented on HBASE-20495:
-
we could take another stab at switching some parts over to the hadoop 3 client
facing modules that shade things. hbase-thrift / hbase-rest are decent
candidates maybe?
> REST unit test fails with NoClassDefFoundError against hadoop3
> --
>
> Key: HBASE-20495
> URL: https://issues.apache.org/jira/browse/HBASE-20495
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 20495.v1.txt
>
>
> The following was first observed in the test output of rest.TestDeleteRow
> against hadoop3:
> {code}
> java.lang.NoClassDefFoundError:
> com/sun/jersey/core/spi/factory/AbstractRuntimeDelegate
> Caused by: java.lang.ClassNotFoundException:
> com.sun.jersey.core.spi.factory.AbstractRuntimeDelegate
> {code}
> This was due to the following transitive dependency on jersey 1.19:
> {code}
> [INFO] +- org.apache.hbase:hbase-testing-util:jar:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-zookeeper:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-hadoop-compat:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-hadoop2-compat:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:3.0.0:compile
> [INFO] | | \-
> org.apache.hadoop:hadoop-mapreduce-client-common:jar:3.0.0:compile
> [INFO] | +- org.apache.hadoop:hadoop-hdfs:test-jar:tests:3.0.0:test
> [INFO] | | \- com.sun.jersey:jersey-server:jar:1.19:compile
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20495) REST unit test fails with NoClassDefFoundError against hadoop3
[
https://issues.apache.org/jira/browse/HBASE-20495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454582#comment-16454582
]
Ted Yu commented on HBASE-20495:
So it is from hbase-thrift module.
I checked the test result for thrift and thrift2 against hadoop 3.1 - every
test passes.
So, maybe add exclusion in another JIRA if there is such need ?
> REST unit test fails with NoClassDefFoundError against hadoop3
> --
>
> Key: HBASE-20495
> URL: https://issues.apache.org/jira/browse/HBASE-20495
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 20495.v1.txt
>
>
> The following was first observed in the test output of rest.TestDeleteRow
> against hadoop3:
> {code}
> java.lang.NoClassDefFoundError:
> com/sun/jersey/core/spi/factory/AbstractRuntimeDelegate
> Caused by: java.lang.ClassNotFoundException:
> com.sun.jersey.core.spi.factory.AbstractRuntimeDelegate
> {code}
> This was due to the following transitive dependency on jersey 1.19:
> {code}
> [INFO] +- org.apache.hbase:hbase-testing-util:jar:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-zookeeper:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-hadoop-compat:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-hadoop2-compat:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:3.0.0:compile
> [INFO] | | \-
> org.apache.hadoop:hadoop-mapreduce-client-common:jar:3.0.0:compile
> [INFO] | +- org.apache.hadoop:hadoop-hdfs:test-jar:tests:3.0.0:test
> [INFO] | | \- com.sun.jersey:jersey-server:jar:1.19:compile
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20474) Show non-RPC tasks on master/regionserver Web UI by default
[ https://issues.apache.org/jira/browse/HBASE-20474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454578#comment-16454578 ] Sahil Aggarwal commented on HBASE-20474: [~andrewcheng] Tested it, works fine. > Show non-RPC tasks on master/regionserver Web UI by default > > > Key: HBASE-20474 > URL: https://issues.apache.org/jira/browse/HBASE-20474 > Project: HBase > Issue Type: Improvement > Components: UI >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20474.master.001.patch > > > Now, when opening master or regionserver pages, all tasks will be displayed > on the page, however, but in most cases we will pay more attention to non-RPC > tasks. > In addition, if all tasks are displayed by default, a large number of pages > will be occupied. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20474) Show non-RPC tasks on master/regionserver Web UI by default
[ https://issues.apache.org/jira/browse/HBASE-20474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Aggarwal updated HBASE-20474: --- Attachment: (was: HBASE-20474.master.002.patch) > Show non-RPC tasks on master/regionserver Web UI by default > > > Key: HBASE-20474 > URL: https://issues.apache.org/jira/browse/HBASE-20474 > Project: HBase > Issue Type: Improvement > Components: UI >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20474.master.001.patch > > > Now, when opening master or regionserver pages, all tasks will be displayed > on the page, however, but in most cases we will pay more attention to non-RPC > tasks. > In addition, if all tasks are displayed by default, a large number of pages > will be occupied. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20474) Show non-RPC tasks on master/regionserver Web UI by default
[ https://issues.apache.org/jira/browse/HBASE-20474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Aggarwal updated HBASE-20474: --- Attachment: HBASE-20474.master.002.patch > Show non-RPC tasks on master/regionserver Web UI by default > > > Key: HBASE-20474 > URL: https://issues.apache.org/jira/browse/HBASE-20474 > Project: HBase > Issue Type: Improvement > Components: UI >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20474.master.001.patch, > HBASE-20474.master.002.patch > > > Now, when opening master or regionserver pages, all tasks will be displayed > on the page, however, but in most cases we will pay more attention to non-RPC > tasks. > In addition, if all tasks are displayed by default, a large number of pages > will be occupied. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20495) REST unit test fails with NoClassDefFoundError against hadoop3
[
https://issues.apache.org/jira/browse/HBASE-20495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454568#comment-16454568
]
Josh Elser commented on HBASE-20495:
{quote}{quote}do we need the hadoop-hdfs:test dependency at all?
{quote}
If hadoop-hdfs:test is excluded, you would get the following:
{quote}
Ah, I was thinking that we had a better dependency for that now. I guess we
have to switch over to the new minicluster jars to get that working.
{quote}Yes. This fix is needed - same exception was observed running against
3.1.0
{quote}
Ok, great.
{quote}Which module is the above observed ?
There is only one occurrence of jersey-server jar in hbase-rest module
dependency tree.
{quote}
{noformat}
[INFO] ---< org.apache.hbase:hbase-thrift >
[INFO] Building Apache HBase - Thrift 2.0.0 [23/39]
[INFO] [ jar ]-
[INFO]
[INFO] --- maven-dependency-plugin:3.0.1:tree (default-cli) @ hbase-thrift ---
[INFO] org.apache.hbase:hbase-thrift:jar:2.0.0
[INFO] +- org.apache.hadoop:hadoop-minicluster:jar:3.0.0:compile
[INFO] | +- org.apache.hadoop:hadoop-common:test-jar:tests:3.0.0:compile
[INFO] | | +- com.sun.jersey:jersey-servlet:jar:1.19:compile
[INFO] | | +- com.sun.jersey:jersey-json:jar:1.19:compile
[INFO] | | | +- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:compile
[INFO] | | | +- org.codehaus.jackson:jackson-jaxrs:jar:1.9.2:compile
[INFO] | | | \- org.codehaus.jackson:jackson-xc:jar:1.9.2:compile
[INFO] | | \- com.sun.jersey:jersey-server:jar:1.19:compile
{noformat}
This was from HEAD of branch-2.0 using {{mvn dependency:tree
-Dhadoop.profile=3.0}}
> REST unit test fails with NoClassDefFoundError against hadoop3
> --
>
> Key: HBASE-20495
> URL: https://issues.apache.org/jira/browse/HBASE-20495
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 20495.v1.txt
>
>
> The following was first observed in the test output of rest.TestDeleteRow
> against hadoop3:
> {code}
> java.lang.NoClassDefFoundError:
> com/sun/jersey/core/spi/factory/AbstractRuntimeDelegate
> Caused by: java.lang.ClassNotFoundException:
> com.sun.jersey.core.spi.factory.AbstractRuntimeDelegate
> {code}
> This was due to the following transitive dependency on jersey 1.19:
> {code}
> [INFO] +- org.apache.hbase:hbase-testing-util:jar:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-zookeeper:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-hadoop-compat:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-hadoop2-compat:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:3.0.0:compile
> [INFO] | | \-
> org.apache.hadoop:hadoop-mapreduce-client-common:jar:3.0.0:compile
> [INFO] | +- org.apache.hadoop:hadoop-hdfs:test-jar:tests:3.0.0:test
> [INFO] | | \- com.sun.jersey:jersey-server:jar:1.19:compile
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20495) REST unit test fails with NoClassDefFoundError against hadoop3
[
https://issues.apache.org/jira/browse/HBASE-20495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454541#comment-16454541
]
Hadoop QA commented on HBASE-20495:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
31s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
1s{color} | {color:blue} The patch file was not named according to hbase's
naming conventions. Please see
https://yetus.apache.org/documentation/0.7.0/precommit-patchnames for
instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
48s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
47s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
14m 38s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
1s{color} | {color:green} hbase-rest in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
9s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 39m 2s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20495 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12920828/20495.v1.txt |
| Optional Tests | asflicense javac javadoc unit shadedjars hadoopcheck
xml compile |
| uname | Linux 693c13292e8d 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 90d103d6d5 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_162 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/12667/testReport/ |
| Max. process+thread count | 2321 (vs. ulimit of 1) |
| modules | C: hbase-rest U: hbase-rest |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/12667/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> REST unit test fails with NoClassDefFoundError against hadoop3
> --
>
> Key: HBASE-20495
> URL: https://issues.apache.org/jira/browse/HBASE-20495
[jira] [Commented] (HBASE-20369) Document incompatibilities between HBase 1.1.2 and HBase 2.0
[
https://issues.apache.org/jira/browse/HBASE-20369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454539#comment-16454539
]
Sean Busbey commented on HBASE-20369:
-
{code}
+[[General information]]
+=== General information
+
+* You can perform a rolling upgrade from HBase 1.2 to HBase 2.0 without any
downtime on a secured cluster. If you are on 1.x has to first upgrade (rolling
upgrade supported) to latest version in 1.x and then to 2.0.
+* If you are an administrator, you cannot manage a 2.0 cluster with 1.x client
version.
+* HBase replication from 2.0 to 1.x and 1.x to 2.0 works as expected without
any interruption.
+* Lot of changes to InterfaceAudience and Private classes and many of them are
removed in HBase 2.0.
+
{code}
Our current upgrade section expressly calls out that rolling upgrade from 1.x
to 2.0 doesn't work:
http://hbase.apache.org/book.html#upgrade2.0.rolling.upgrades
Is there new testing somewhere to indicate otherwise?
bq. Lot of changes to InterfaceAudience and Private classes and many of them
are removed in HBase 2.0.
Does this mean to say "changes to InterfaceAudience.Private classes" ? If so,
worth sticking a "downstream users are reminded that these classes are internal
to the project and must not be relied upon."
> Document incompatibilities between HBase 1.1.2 and HBase 2.0
>
>
> Key: HBASE-20369
> URL: https://issues.apache.org/jira/browse/HBASE-20369
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Thiriguna Bharat Rao
>Assignee: Thiriguna Bharat Rao
>Priority: Critical
> Labels: patch
> Attachments: HBASE-20369.patch, HBASE-20369_v1.patch,
> HBASE-20369_v2.patch, HBASE-20369_v3.patch, HBASE-20369_v4.patch, book.adoc
>
>
> Hi,
> I compiled a draft document for the HBase incompatibilities from the raw
> source content that was available in HBase Beta 1 site. Can someone please
> review and provide a feedback or share your comments on this document?
> Appreciate your support and time.
>
> Best Regards,
> Triguna
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20495) REST unit test fails with NoClassDefFoundError against hadoop3
[
https://issues.apache.org/jira/browse/HBASE-20495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454513#comment-16454513
]
Ted Yu commented on HBASE-20495:
bq. do we need the hadoop-hdfs:test dependency at all?
If hadoop-hdfs:test is excluded, you would get the following:
{code}
[INFO] Running org.apache.hadoop.hbase.rest.TestDeleteRow
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.362 s
<<< FAILURE! - in org.apache.hadoop.hbase.rest.TestDeleteRow
[ERROR] org.apache.hadoop.hbase.rest.TestDeleteRow Time elapsed: 1.361 s <<<
ERROR!
java.lang.NoClassDefFoundError: org/apache/hadoop/hdfs/MiniDFSCluster
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.hdfs.MiniDFSCluster
{code}
> REST unit test fails with NoClassDefFoundError against hadoop3
> --
>
> Key: HBASE-20495
> URL: https://issues.apache.org/jira/browse/HBASE-20495
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 20495.v1.txt
>
>
> The following was first observed in the test output of rest.TestDeleteRow
> against hadoop3:
> {code}
> java.lang.NoClassDefFoundError:
> com/sun/jersey/core/spi/factory/AbstractRuntimeDelegate
> Caused by: java.lang.ClassNotFoundException:
> com.sun.jersey.core.spi.factory.AbstractRuntimeDelegate
> {code}
> This was due to the following transitive dependency on jersey 1.19:
> {code}
> [INFO] +- org.apache.hbase:hbase-testing-util:jar:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-zookeeper:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-hadoop-compat:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-hadoop2-compat:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:3.0.0:compile
> [INFO] | | \-
> org.apache.hadoop:hadoop-mapreduce-client-common:jar:3.0.0:compile
> [INFO] | +- org.apache.hadoop:hadoop-hdfs:test-jar:tests:3.0.0:test
> [INFO] | | \- com.sun.jersey:jersey-server:jar:1.19:compile
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20495) REST unit test fails with NoClassDefFoundError against hadoop3
[
https://issues.apache.org/jira/browse/HBASE-20495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454497#comment-16454497
]
Ted Yu commented on HBASE-20495:
bq. Do we need the same change with Hadoop 3.0.2 and/or 3.1.0
Yes. This fix is needed - same exception was observed running against 3.1.0
bq. I also see another place where this jersey-server jar gets on the
dependencyTree
Which module is the above observed ?
There is only one occurrence of jersey-server jar in hbase-rest module
dependency tree.
> REST unit test fails with NoClassDefFoundError against hadoop3
> --
>
> Key: HBASE-20495
> URL: https://issues.apache.org/jira/browse/HBASE-20495
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 20495.v1.txt
>
>
> The following was first observed in the test output of rest.TestDeleteRow
> against hadoop3:
> {code}
> java.lang.NoClassDefFoundError:
> com/sun/jersey/core/spi/factory/AbstractRuntimeDelegate
> Caused by: java.lang.ClassNotFoundException:
> com.sun.jersey.core.spi.factory.AbstractRuntimeDelegate
> {code}
> This was due to the following transitive dependency on jersey 1.19:
> {code}
> [INFO] +- org.apache.hbase:hbase-testing-util:jar:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-zookeeper:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-hadoop-compat:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-hadoop2-compat:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:3.0.0:compile
> [INFO] | | \-
> org.apache.hadoop:hadoop-mapreduce-client-common:jar:3.0.0:compile
> [INFO] | +- org.apache.hadoop:hadoop-hdfs:test-jar:tests:3.0.0:test
> [INFO] | | \- com.sun.jersey:jersey-server:jar:1.19:compile
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20495) REST unit test fails with NoClassDefFoundError against hadoop3
[
https://issues.apache.org/jira/browse/HBASE-20495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454477#comment-16454477
]
Josh Elser commented on HBASE-20495:
Do we need the same change with Hadoop 3.0.2 and/or 3.1.0? I recall seeing at
least one issue in dependency management that Hadoop had fixed over 3.0.0.
Better yet, do we need the hadoop-hdfs:test dependency at all? That is kind of
a smell to me outright.
{noformat}
[INFO] +- org.apache.hadoop:hadoop-minicluster:jar:3.0.0:compile
[INFO] | +- org.apache.hadoop:hadoop-common:test-jar:tests:3.0.0:compile
[INFO] | | +- com.sun.jersey:jersey-servlet:jar:1.19:compile
[INFO] | | +- com.sun.jersey:jersey-json:jar:1.19:compile
[INFO] | | | +- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:compile
[INFO] | | | +- org.codehaus.jackson:jackson-jaxrs:jar:1.9.2:compile
[INFO] | | | \- org.codehaus.jackson:jackson-xc:jar:1.9.2:compile
[INFO] | | \- com.sun.jersey:jersey-server:jar:1.19:compile{noformat}
I also see another place where this jersey-server jar gets on the
dependencyTree. Should it be excluded there as well?
> REST unit test fails with NoClassDefFoundError against hadoop3
> --
>
> Key: HBASE-20495
> URL: https://issues.apache.org/jira/browse/HBASE-20495
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 20495.v1.txt
>
>
> The following was first observed in the test output of rest.TestDeleteRow
> against hadoop3:
> {code}
> java.lang.NoClassDefFoundError:
> com/sun/jersey/core/spi/factory/AbstractRuntimeDelegate
> Caused by: java.lang.ClassNotFoundException:
> com.sun.jersey.core.spi.factory.AbstractRuntimeDelegate
> {code}
> This was due to the following transitive dependency on jersey 1.19:
> {code}
> [INFO] +- org.apache.hbase:hbase-testing-util:jar:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-zookeeper:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-hadoop-compat:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-hadoop2-compat:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:3.0.0:compile
> [INFO] | | \-
> org.apache.hadoop:hadoop-mapreduce-client-common:jar:3.0.0:compile
> [INFO] | +- org.apache.hadoop:hadoop-hdfs:test-jar:tests:3.0.0:test
> [INFO] | | \- com.sun.jersey:jersey-server:jar:1.19:compile
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20495) REST unit test fails with NoClassDefFoundError against hadoop3
[
https://issues.apache.org/jira/browse/HBASE-20495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454466#comment-16454466
]
Ted Yu commented on HBASE-20495:
Ran TestDeleteRow with patch locally against both hadoop2 and hadoop3 - passed.
> REST unit test fails with NoClassDefFoundError against hadoop3
> --
>
> Key: HBASE-20495
> URL: https://issues.apache.org/jira/browse/HBASE-20495
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 20495.v1.txt
>
>
> The following was first observed in the test output of rest.TestDeleteRow
> against hadoop3:
> {code}
> java.lang.NoClassDefFoundError:
> com/sun/jersey/core/spi/factory/AbstractRuntimeDelegate
> Caused by: java.lang.ClassNotFoundException:
> com.sun.jersey.core.spi.factory.AbstractRuntimeDelegate
> {code}
> This was due to the following transitive dependency on jersey 1.19:
> {code}
> [INFO] +- org.apache.hbase:hbase-testing-util:jar:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-zookeeper:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-hadoop-compat:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-hadoop2-compat:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:3.0.0:compile
> [INFO] | | \-
> org.apache.hadoop:hadoop-mapreduce-client-common:jar:3.0.0:compile
> [INFO] | +- org.apache.hadoop:hadoop-hdfs:test-jar:tests:3.0.0:test
> [INFO] | | \- com.sun.jersey:jersey-server:jar:1.19:compile
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20369) Document incompatibilities between HBase 1.1.2 and HBase 2.0
[
https://issues.apache.org/jira/browse/HBASE-20369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454469#comment-16454469
]
Hadoop QA commented on HBASE-20369:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
31s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
35s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 4m
47s{color} | {color:blue} branch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
1s{color} | {color:red} The patch 4 line(s) with tabs. {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 4m
50s{color} | {color:blue} patch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
10s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 15m 9s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20369 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12920824/HBASE-20369_v4.patch |
| Optional Tests | asflicense refguide |
| uname | Linux fff3008a3d0f 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 90d103d6d5 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| refguide |
https://builds.apache.org/job/PreCommit-HBASE-Build/12666/artifact/patchprocess/branch-site/book.html
|
| whitespace |
https://builds.apache.org/job/PreCommit-HBASE-Build/12666/artifact/patchprocess/whitespace-tabs.txt
|
| refguide |
https://builds.apache.org/job/PreCommit-HBASE-Build/12666/artifact/patchprocess/patch-site/book.html
|
| Max. process+thread count | 83 (vs. ulimit of 1) |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/12666/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Document incompatibilities between HBase 1.1.2 and HBase 2.0
>
>
> Key: HBASE-20369
> URL: https://issues.apache.org/jira/browse/HBASE-20369
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Thiriguna Bharat Rao
>Assignee: Thiriguna Bharat Rao
>Priority: Critical
> Labels: patch
> Attachments: HBASE-20369.patch, HBASE-20369_v1.patch,
> HBASE-20369_v2.patch, HBASE-20369_v3.patch, HBASE-20369_v4.patch, book.adoc
>
>
> Hi,
> I compiled a draft document for the HBase incompatibilities from the raw
> source content that was available in HBase Beta 1 site. Can someone please
> review and provide a feedback or share your comments on this document?
> Appreciate your support and time.
>
> Best Regards,
> Triguna
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20495) REST unit test fails with NoClassDefFoundError against hadoop3
[
https://issues.apache.org/jira/browse/HBASE-20495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-20495:
---
Status: Patch Available (was: Open)
> REST unit test fails with NoClassDefFoundError against hadoop3
> --
>
> Key: HBASE-20495
> URL: https://issues.apache.org/jira/browse/HBASE-20495
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 20495.v1.txt
>
>
> The following was first observed in the test output of rest.TestDeleteRow
> against hadoop3:
> {code}
> java.lang.NoClassDefFoundError:
> com/sun/jersey/core/spi/factory/AbstractRuntimeDelegate
> Caused by: java.lang.ClassNotFoundException:
> com.sun.jersey.core.spi.factory.AbstractRuntimeDelegate
> {code}
> This was due to the following transitive dependency on jersey 1.19:
> {code}
> [INFO] +- org.apache.hbase:hbase-testing-util:jar:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-zookeeper:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-hadoop-compat:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-hadoop2-compat:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:3.0.0:compile
> [INFO] | | \-
> org.apache.hadoop:hadoop-mapreduce-client-common:jar:3.0.0:compile
> [INFO] | +- org.apache.hadoop:hadoop-hdfs:test-jar:tests:3.0.0:test
> [INFO] | | \- com.sun.jersey:jersey-server:jar:1.19:compile
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20495) REST unit test fails with NoClassDefFoundError against hadoop3
[
https://issues.apache.org/jira/browse/HBASE-20495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-20495:
---
Attachment: 20495.v1.txt
> REST unit test fails with NoClassDefFoundError against hadoop3
> --
>
> Key: HBASE-20495
> URL: https://issues.apache.org/jira/browse/HBASE-20495
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 20495.v1.txt
>
>
> The following was first observed in the test output of rest.TestDeleteRow
> against hadoop3:
> {code}
> java.lang.NoClassDefFoundError:
> com/sun/jersey/core/spi/factory/AbstractRuntimeDelegate
> Caused by: java.lang.ClassNotFoundException:
> com.sun.jersey.core.spi.factory.AbstractRuntimeDelegate
> {code}
> This was due to the following transitive dependency on jersey 1.19:
> {code}
> [INFO] +- org.apache.hbase:hbase-testing-util:jar:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-zookeeper:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-hadoop-compat:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-hadoop2-compat:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:3.0.0:compile
> [INFO] | | \-
> org.apache.hadoop:hadoop-mapreduce-client-common:jar:3.0.0:compile
> [INFO] | +- org.apache.hadoop:hadoop-hdfs:test-jar:tests:3.0.0:test
> [INFO] | | \- com.sun.jersey:jersey-server:jar:1.19:compile
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-20495) REST unit test fails with NoClassDefFoundError against hadoop3
Ted Yu created HBASE-20495:
--
Summary: REST unit test fails with NoClassDefFoundError against
hadoop3
Key: HBASE-20495
URL: https://issues.apache.org/jira/browse/HBASE-20495
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
The following was first observed in the test output of rest.TestDeleteRow
against hadoop3:
{code}
java.lang.NoClassDefFoundError:
com/sun/jersey/core/spi/factory/AbstractRuntimeDelegate
Caused by: java.lang.ClassNotFoundException:
com.sun.jersey.core.spi.factory.AbstractRuntimeDelegate
{code}
This was due to the following transitive dependency on jersey 1.19:
{code}
[INFO] +- org.apache.hbase:hbase-testing-util:jar:2.0.0.3.0.0.0-SNAPSHOT:test
[INFO] | +-
org.apache.hbase:hbase-zookeeper:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
[INFO] | +-
org.apache.hbase:hbase-hadoop-compat:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
[INFO] | +-
org.apache.hbase:hbase-hadoop2-compat:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
[INFO] | +-
org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:3.0.0:compile
[INFO] | | \-
org.apache.hadoop:hadoop-mapreduce-client-common:jar:3.0.0:compile
[INFO] | +- org.apache.hadoop:hadoop-hdfs:test-jar:tests:3.0.0:test
[INFO] | | \- com.sun.jersey:jersey-server:jar:1.19:compile
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20369) Document incompatibilities between HBase 1.1.2 and HBase 2.0
[ https://issues.apache.org/jira/browse/HBASE-20369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454443#comment-16454443 ] Thiriguna Bharat Rao commented on HBASE-20369: -- The name of the new patch file: HBASE-20369_v4.patch > Document incompatibilities between HBase 1.1.2 and HBase 2.0 > > > Key: HBASE-20369 > URL: https://issues.apache.org/jira/browse/HBASE-20369 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Thiriguna Bharat Rao >Assignee: Thiriguna Bharat Rao >Priority: Critical > Labels: patch > Attachments: HBASE-20369.patch, HBASE-20369_v1.patch, > HBASE-20369_v2.patch, HBASE-20369_v3.patch, HBASE-20369_v4.patch, book.adoc > > > Hi, > I compiled a draft document for the HBase incompatibilities from the raw > source content that was available in HBase Beta 1 site. Can someone please > review and provide a feedback or share your comments on this document? > Appreciate your support and time. > > Best Regards, > Triguna -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20369) Document incompatibilities between HBase 1.1.2 and HBase 2.0
[ https://issues.apache.org/jira/browse/HBASE-20369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454438#comment-16454438 ] Thiriguna Bharat Rao commented on HBASE-20369: -- [~busbey] Many thanks for the feedback. I fixed most of the formatting issues, please review it and let me know your comments. Appreciate your support and time. Best, Triguna > Document incompatibilities between HBase 1.1.2 and HBase 2.0 > > > Key: HBASE-20369 > URL: https://issues.apache.org/jira/browse/HBASE-20369 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Thiriguna Bharat Rao >Assignee: Thiriguna Bharat Rao >Priority: Critical > Labels: patch > Attachments: HBASE-20369.patch, HBASE-20369_v1.patch, > HBASE-20369_v2.patch, HBASE-20369_v3.patch, HBASE-20369_v4.patch, book.adoc > > > Hi, > I compiled a draft document for the HBase incompatibilities from the raw > source content that was available in HBase Beta 1 site. Can someone please > review and provide a feedback or share your comments on this document? > Appreciate your support and time. > > Best Regards, > Triguna -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20369) Document incompatibilities between HBase 1.1.2 and HBase 2.0
[ https://issues.apache.org/jira/browse/HBASE-20369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiriguna Bharat Rao updated HBASE-20369: - Attachment: HBASE-20369_v4.patch > Document incompatibilities between HBase 1.1.2 and HBase 2.0 > > > Key: HBASE-20369 > URL: https://issues.apache.org/jira/browse/HBASE-20369 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Thiriguna Bharat Rao >Assignee: Thiriguna Bharat Rao >Priority: Critical > Labels: patch > Attachments: HBASE-20369.patch, HBASE-20369_v1.patch, > HBASE-20369_v2.patch, HBASE-20369_v3.patch, HBASE-20369_v4.patch, book.adoc > > > Hi, > I compiled a draft document for the HBase incompatibilities from the raw > source content that was available in HBase Beta 1 site. Can someone please > review and provide a feedback or share your comments on this document? > Appreciate your support and time. > > Best Regards, > Triguna -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20484) Remove the unnecessary autoboxing in FilterListBase
[
https://issues.apache.org/jira/browse/HBASE-20484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454419#comment-16454419
]
Hudson commented on HBASE-20484:
Results for branch branch-2
[build #662 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/662/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/662//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/662//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/662//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Remove the unnecessary autoboxing in FilterListBase
> ---
>
> Key: HBASE-20484
> URL: https://issues.apache.org/jira/browse/HBASE-20484
> Project: HBase
> Issue Type: Improvement
>Reporter: Chia-Ping Tsai
>Assignee: Guangxu Cheng
>Priority: Trivial
> Labels: beginner, beginners
> Fix For: 3.0.0, 2.1.0, 2.0.1
>
> Attachments: HBASE-20484.master.001.patch
>
>
> {code:java}
> protected static boolean checkAndGetReversed(List rowFilters, boolean
> defaultValue) {
> if (rowFilters.isEmpty()) {
> return defaultValue;
> }
> Boolean retValue = rowFilters.get(0).isReversed(); // this one
> boolean allEqual =
> rowFilters.stream().map(Filter::isReversed).allMatch(retValue::equals);
> if (!allEqual) {
> throw new IllegalArgumentException("Filters in the list must have the
> same reversed flag");
> }
> return retValue;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20476) Fix the flaky TestReplicationSmallTests unit test
[
https://issues.apache.org/jira/browse/HBASE-20476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454391#comment-16454391
]
Hadoop QA commented on HBASE-20476:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
54s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
48s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
14m 34s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}108m
22s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}153m 40s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20476 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12920789/HBASE-20476.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux defcbce84eec 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 2ab731824f |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_162 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/12665/testReport/ |
| Max. process+thread count | 4496 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/12665/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Fix the flaky
[jira] [Commented] (HBASE-19543) Abstract a replication storage interface to extract the zk specific code
[ https://issues.apache.org/jira/browse/HBASE-19543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454387#comment-16454387 ] Mike Drob commented on HBASE-19543: --- [~Apache9] - any further thoughts on this? > Is there a better way to get this same information? > Abstract a replication storage interface to extract the zk specific code > > > Key: HBASE-19543 > URL: https://issues.apache.org/jira/browse/HBASE-19543 > Project: HBase > Issue Type: Sub-task > Components: proc-v2, Replication >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.1.0 > > Attachments: HBASE-19543-HBASE-19397-v1.patch, > HBASE-19543-HBASE-19397-v2.patch, HBASE-19543-HBASE-19397-v3.patch, > HBASE-19543-HBASE-19397.patch, HBASE-19543-HBASE-19397.patch > > > For now, we will do sanity checks at the same time when updating replication > peer. But this is not a safe way for procedure based replication peer > modification. > For the old zk watcher way, the only thing is updating the data on zk, so if > the data is updated and then we crashes, there is no problem. > For the new procedure way, we need to trigger refresh by ourselves after > updating zk. If we crashes after the updating and before we record the state > change of the procedure, we may fail with IllegalArgumentException when we > execute the procedure next time since the data on zk has already been updated. > So the current way is to do sanity checks in PRE_PEER_MODIFICATION state, and > in UPDATE_STORAGE state we will not do sanity checks any more, just > update(overwrite) the peer storage. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20478) move import checks from hbaseanti to checkstyle
[ https://issues.apache.org/jira/browse/HBASE-20478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454364#comment-16454364 ] Sean Busbey commented on HBASE-20478: - yeah that matches my understanding. I don't know why we got the second QA run. but the first looks good and the second was the node going dead talking with jenkins. > move import checks from hbaseanti to checkstyle > --- > > Key: HBASE-20478 > URL: https://issues.apache.org/jira/browse/HBASE-20478 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Sean Busbey >Assignee: Mike Drob >Priority: Minor > Attachments: HBASE-20478.0.patch, HBASE-20478.1.patch, > HBASE-20478.2.patch, HBASE-20478.3.patch, HBASE-20478.WIP.2.patch, > HBASE-20478.WIP.2.patch, HBASE-20478.WIP.patch > > > came up in discussion on HBASE-20332. our check of "don't do this" things in > the codebase doesn't log the specifics of complaints anywhere, which forces > those who want to follow up to reverse engineer the check. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20478) move import checks from hbaseanti to checkstyle
[ https://issues.apache.org/jira/browse/HBASE-20478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-20478: Priority: Minor (was: Major) > move import checks from hbaseanti to checkstyle > --- > > Key: HBASE-20478 > URL: https://issues.apache.org/jira/browse/HBASE-20478 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Sean Busbey >Assignee: Mike Drob >Priority: Minor > Attachments: HBASE-20478.0.patch, HBASE-20478.1.patch, > HBASE-20478.2.patch, HBASE-20478.3.patch, HBASE-20478.WIP.2.patch, > HBASE-20478.WIP.2.patch, HBASE-20478.WIP.patch > > > came up in discussion on HBASE-20332. our check of "don't do this" things in > the codebase doesn't log the specifics of complaints anywhere, which forces > those who want to follow up to reverse engineer the check. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20478) move import checks from hbaseanti to checkstyle
[ https://issues.apache.org/jira/browse/HBASE-20478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454328#comment-16454328 ] Mike Drob commented on HBASE-20478: --- Oh, I thought we were waiting on QA to run, but it looks like it did, and then it triggered again and that run is what blew up? No objections to merging now. > move import checks from hbaseanti to checkstyle > --- > > Key: HBASE-20478 > URL: https://issues.apache.org/jira/browse/HBASE-20478 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Sean Busbey >Assignee: Mike Drob >Priority: Major > Attachments: HBASE-20478.0.patch, HBASE-20478.1.patch, > HBASE-20478.2.patch, HBASE-20478.3.patch, HBASE-20478.WIP.2.patch, > HBASE-20478.WIP.2.patch, HBASE-20478.WIP.patch > > > came up in discussion on HBASE-20332. our check of "don't do this" things in > the codebase doesn't log the specifics of complaints anywhere, which forces > those who want to follow up to reverse engineer the check. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20478) move import checks from hbaseanti to checkstyle
[ https://issues.apache.org/jira/browse/HBASE-20478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454308#comment-16454308 ] Sean Busbey commented on HBASE-20478: - I'd like to merge this now. any objections? > move import checks from hbaseanti to checkstyle > --- > > Key: HBASE-20478 > URL: https://issues.apache.org/jira/browse/HBASE-20478 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Sean Busbey >Assignee: Mike Drob >Priority: Major > Attachments: HBASE-20478.0.patch, HBASE-20478.1.patch, > HBASE-20478.2.patch, HBASE-20478.3.patch, HBASE-20478.WIP.2.patch, > HBASE-20478.WIP.2.patch, HBASE-20478.WIP.patch > > > came up in discussion on HBASE-20332. our check of "don't do this" things in > the codebase doesn't log the specifics of complaints anywhere, which forces > those who want to follow up to reverse engineer the check. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20475) Fix the flaky TestReplicationDroppedTables unit test.
[
https://issues.apache.org/jira/browse/HBASE-20475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454297#comment-16454297
]
Zheng Hu commented on HBASE-20475:
--
Found the cause why the test to be flaky again, and it happened randomly ...
see the following log:
{code}
2018-04-25 18:08:24,432 INFO [regionserver/asf915:0.logRoller]
wal.AbstractFSWAL(671): Rolled WAL
/user/jenkins/test-data/be2e424e-e2f6-4ac1-91f1-d33621a46da3/WALs/asf915.gq1.ygridcore.net,42474,1524679689830/asf915.gq1.ygridcore.net%2C42
474%2C1524679689830.1524679692092 with entries=13, filesize=3.55 KB; new WAL
/user/jenkins/test-data/be2e424e-e2f6-4ac1-91f1-d33621a46da3/WALs/asf915.gq1.ygridcore.net,42474,1524679689830/asf915.gq1.ygridcore.net%2C42474%2C1524679689830.15
24679704418
..
2018-04-25 18:08:36,957 INFO [ReplicationExecutor-0]
replication.ZKReplicationQueueStorage(387): Atomically moving
asf915.gq1.ygridcore.net,42474,1524679689830/2's WALs to
asf915.gq1.ygridcore.net,34222,1524679706894
2018-04-25 18:08:36,957 DEBUG
[RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=36265]
ipc.AbstractRpcClient(200):
Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@396b2d58,
compressor=null, tcpKeepAlive=true, tcpNoDelay=tru
e, connectTO=1, readTO=2, writeTO=6, minIdleTimeBeforeClose=12,
maxRetries=0, fallbackAllowed=true, bind address=null
2018-04-25 18:08:36,959 DEBUG [ReplicationExecutor-0]
replication.ZKReplicationQueueStorage(414): Creating
asf915.gq1.ygridcore.net%2C42474%2C1524679689830.1524679704418 with data
PBUF\x08\x9F\x1A
2018-04-25 18:08:36,961 INFO [ReplicationExecutor-0]
replication.ZKReplicationQueueStorage(426): Atomically moved
asf915.gq1.ygridcore.net,42474,1524679689830/2's WALs to
asf915.gq1.ygridcore.net,34222,1524679706894
{code}
Step.1 Atomically moving -> it means that we begin to claimQueue from the
deadRS to destination RS.
Step.2 at 2018-04-25 18:08:24,432 , the rs start to roll the WAL, and at
2018-04-25 18:08:36,959 , it created the WAL.
Step.3 Atomically moved ... -> the NodeFailoverWorker finished to
cliamQueue from deadRS to dest RS, but exclude the WAL
asf915.gq1.ygridcore.net%2C42474%2C1524679689830.1524679704418, because had no
ZNODE yet.
So when our RecoveredReplicationSourceShipper try to ship the edits and
setWALPosition, it found that the znode did not exist, and the RS crashed
finally. (It's strange here: the new rs had been transferring queue, but the
dead RS was still creating the new WAL...)
{code}
2018-04-25 18:08:39,107 DEBUG
[ReplicationExecutor-0.replicationSource,2-asf915.gq1.ygridcore.net,42474,1524679689830.replicationSource.wal-reader.asf915.gq1.ygridcore.net%2C42474%2C1524679689830,2-asf915.gq1.ygridcore.net,42474,1524679689
830] regionserver.WALEntryStream(250): Reached the end of log
hdfs://localhost:43322/user/jenkins/test-data/be2e424e-e2f6-4ac1-91f1-d33621a46da3/oldWALs/asf915.gq1.ygridcore.net%2C42474%2C1524679689830.1524679704418
2018-04-25 18:08:39,109 DEBUG
[ReplicationExecutor-0.replicationSource,2-asf915.gq1.ygridcore.net,42474,1524679689830.replicationSource.shipperasf915.gq1.ygridcore.net%2C42474%2C1524679689830,2-asf915.gq1.ygridcore.net,42474,1524679689830]
replication.ReplicationQueueInfo(110): Found dead
servers:[asf915.gq1.ygridcore.net,42474,1524679689830]
2018-04-25 18:08:39,118 ERROR
[ReplicationExecutor-0.replicationSource,2-asf915.gq1.ygridcore.net,42474,1524679689830.replicationSource.shipperasf915.gq1.ygridcore.net%2C42474%2C1524679689830,2-asf915.gq1.ygridcore.net,42474,1524679689830]
helpers.MarkerIgnoringBase(159): * ABORTING region server
asf915.gq1.ygridcore.net,34222,1524679706894: Failed to operate on replication
queue *
org.apache.hadoop.hbase.replication.ReplicationException: Failed to set log
position (serverName=asf915.gq1.ygridcore.net,34222,1524679706894,
queueId=2-asf915.gq1.ygridcore.net,42474,1524679689830,
fileName=asf915.gq1.ygridcore.net%2C4247
4%2C1524679689830.1524679704418, position=3632)
at
org.apache.hadoop.hbase.replication.ZKReplicationQueueStorage.setWALPosition(ZKReplicationQueueStorage.java:256)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.lambda$logPositionAndCleanOldLogs$7(ReplicationSourceManager.java:488)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.abortWhenFail(ReplicationSourceManager.java:455)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.logPositionAndCleanOldLogs(ReplicationSourceManager.java:488)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.updateLogPosition(ReplicationSourceShipper.java:231)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.shipEdits(ReplicationSourceShipper.java:133)
at
[jira] [Updated] (HBASE-20467) Precommit personality should only run checkstyle once if we're going to run it at the root.
[
https://issues.apache.org/jira/browse/HBASE-20467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-20467:
Resolution: Fixed
Status: Resolved (was: Patch Available)
pushed to master branch. Thanks!
> Precommit personality should only run checkstyle once if we're going to run
> it at the root.
> ---
>
> Key: HBASE-20467
> URL: https://issues.apache.org/jira/browse/HBASE-20467
> Project: HBase
> Issue Type: Bug
> Components: community, test
>Reporter: Sean Busbey
>Assignee: Nihal Jain
>Priority: Minor
> Labels: beginner
> Fix For: 3.0.0
>
> Attachments: HBASE-20467.master.001.patch
>
>
> shows up v egregiously on HBASE-20332, but I've seen in a few places where
> we'll do checkstyle at top level as well on individual modules.
> eg.:
> {code}
> +1checkstyle 0m 25s hbase-common: The patch generated 0 new + 88
> unchanged - 1 fixed = 88 total (was 89)
> +1checkstyle 0m 12s The patch hbase-hadoop2-compat passed checkstyle
> +1checkstyle 0m 30s The patch hbase-client passed checkstyle
> +1checkstyle 0m 11s The patch hbase-replication passed checkstyle
> +1checkstyle 1m 15s The patch hbase-server passed checkstyle
> +1checkstyle 0m 17s The patch hbase-mapreduce passed checkstyle
> +1checkstyle 0m 8s The patch hbase-testing-util passed checkstyle
> +1checkstyle 0m 28s The patch hbase-thrift passed checkstyle
> +1checkstyle 0m 11s The patch hbase-rsgroup passed checkstyle
> +1checkstyle 0m 10s The patch hbase-shell passed checkstyle
> +1checkstyle 0m 12s The patch hbase-endpoint passed checkstyle
> +1checkstyle 0m 14s The patch hbase-backup passed checkstyle
> +1checkstyle 0m 21s The patch hbase-it passed checkstyle
> +1checkstyle 0m 14s The patch hbase-examples passed checkstyle
> +1checkstyle 0m 16s The patch hbase-rest passed checkstyle
> +1checkstyle 0m 9s The patch hbase-external-blockcache passed
> checkstyle
> +1checkstyle 0m 10s The patch hbase-shaded passed checkstyle
> +1checkstyle 0m 8s The patch hbase-shaded-mapreduce passed
> checkstyle
> +1checkstyle 0m 8s The patch hbase-shaded-check-invariants passed
> checkstyle
> +1checkstyle 0m 9s The patch
> hbase-shaded-with-hadoop-check-invariants passed checkstyle
> +1checkstyle 2m 38s root: The patch generated 0 new + 429 unchanged
> - 1 fixed = 429 total (was 430)
> {code}
> we should use the same shortcut we do with other modules when root is present
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20426) Give up replicating anything in S state
[
https://issues.apache.org/jira/browse/HBASE-20426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454260#comment-16454260
]
Hadoop QA commented on HBASE-20426:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} HBASE-19064 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
4s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
10s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
24s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
14s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
50s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} HBASE-19064 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 3m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} The patch hbase-protocol-shaded passed checkstyle
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
13s{color} | {color:green} hbase-server: The patch generated 0 new + 4
unchanged - 1 fixed = 4 total (was 5) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
18s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 33s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
1m 42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
34s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}170m 31s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
38s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}235m 25s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.TestClientOperationTimeout |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20426 |
| JIRA Patch URL |
[jira] [Commented] (HBASE-20475) Fix the flaky TestReplicationDroppedTables unit test.
[ https://issues.apache.org/jira/browse/HBASE-20475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454240#comment-16454240 ] Duo Zhang commented on HBASE-20475: --- Please open an issue to fix it... > Fix the flaky TestReplicationDroppedTables unit test. > - > > Key: HBASE-20475 > URL: https://issues.apache.org/jira/browse/HBASE-20475 > Project: HBase > Issue Type: Bug >Affects Versions: 2.1.0 >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 2.1.0 > > Attachments: HBASE-20475-addendum.patch, HBASE-20475.patch > > > See > https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20475) Fix the flaky TestReplicationDroppedTables unit test.
[
https://issues.apache.org/jira/browse/HBASE-20475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454213#comment-16454213
]
Zheng Hu commented on HBASE-20475:
--
Found the above bug, because I notice the log :
{code}
2018-04-25 18:08:39,091 DEBUG
[ReplicationExecutor-0.replicationSource,2-asf915.gq1.ygridcore.net,42474,1524679689830]
zookeeper.ZKUtil(614): regionserver:34222-0x162fdfda59e0021,
quorum=localhost:60019, baseZNode=/1 Unable to get data of znode
/1/replication/rs/asf915.gq1.ygridcore.net,42474,1524679689830/2-asf915.gq1.ygridcore.net,42474,1524679689830/asf915.gq1.ygridcore.net%2C42474%2C1524679689830.1524679704418
because node does not exist (not an error)
2018-04-25 18:08:39,091 WARN
[ReplicationExecutor-0.replicationSource,2-asf915.gq1.ygridcore.net,42474,1524679689830]
replication.ZKReplicationQueueStorage(377): Failed parse log position
(serverName=asf915.gq1.ygridcore.net,42474,1524679689830,
queueId=2-asf915.gq1.ygridcore.net,42474,1524679689830,
fileName=asf915.gq1.ygridcore.net%2C42474%2C1524679689830.1524679704418)
{code}
> Fix the flaky TestReplicationDroppedTables unit test.
> -
>
> Key: HBASE-20475
> URL: https://issues.apache.org/jira/browse/HBASE-20475
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20475-addendum.patch, HBASE-20475.patch
>
>
> See
> https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20475) Fix the flaky TestReplicationDroppedTables unit test.
[
https://issues.apache.org/jira/browse/HBASE-20475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454202#comment-16454202
]
Zheng Hu commented on HBASE-20475:
--
Found an unrelated bug in
RecoveredReplicationSourceShipper#getRecoveredQueueStartPos()
{code}
private long getRecoveredQueueStartPos() {
long startPosition = 0;
String peerClusterZnode = source.getQueueId();
try {
startPosition =
this.replicationQueues.getWALPosition(source.getServerWALsBelongTo(),
peerClusterZnode, this.queue.peek().getName());
if (LOG.isTraceEnabled()) {
LOG.trace("Recovered queue started with log " + this.queue.peek() + "
at position " +
startPosition);
}
} catch (ReplicationException e) {
terminate("Couldn't get the position of this recovered queue " +
peerClusterZnode, e);
}
return startPosition;
}
{code}
When we start run to RecoveredReplicationSourceShipper, all WALs of dead server
has been pushed into the new RS's queue, So it will always return -1 for
following method, because the path does not exist any more .
{code}
startPosition =
this.replicationQueues.getWALPosition(source.getServerWALsBelongTo(),
peerClusterZnode, this.queue.peek().getName());
{code}
> Fix the flaky TestReplicationDroppedTables unit test.
> -
>
> Key: HBASE-20475
> URL: https://issues.apache.org/jira/browse/HBASE-20475
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20475-addendum.patch, HBASE-20475.patch
>
>
> See
> https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20359) why Comparator extends comparable?
[ https://issues.apache.org/jira/browse/HBASE-20359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454199#comment-16454199 ] Mike Drob commented on HBASE-20359: --- This is also an error-prone check, yes we should improve this. > why Comparator extends comparable? > -- > > Key: HBASE-20359 > URL: https://issues.apache.org/jira/browse/HBASE-20359 > Project: HBase > Issue Type: Improvement >Reporter: Nick.han >Priority: Minor > > Comparable is an interface,object implements this interface can be compare > with some other object, Comparator is a tool for compare object,the tool > itself can be compare is useless, so why a tool implement the comparable > interface?Do we have to improve this? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20483) [PERFORMANCE] Flushing is 2x slower in hbase2.
[ https://issues.apache.org/jira/browse/HBASE-20483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454187#comment-16454187 ] stack commented on HBASE-20483: --- Attached little perpetual flushing programs so could study flush performance and compare to 1.2. > [PERFORMANCE] Flushing is 2x slower in hbase2. > -- > > Key: HBASE-20483 > URL: https://issues.apache.org/jira/browse/HBASE-20483 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: stack >Assignee: stack >Priority: Major > Attachments: PerpetualFlushingProgram.patch, > PerpetualFlushingProgram1.2.patch, SnapshotSegmentScanner_2.0.0.patch > > > When flush is slow, memstores backup and go slower. See "1.2.7 vs. 2.0.0 > Flush History" > https://docs.google.com/spreadsheets/d/1sihTxb4aCplR3Rr_GGXkPlwhMIm-CbB9j_5339AS0Zc/edit#gid=1016758826 > for compare. First noted by [~anoop.hbase] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20359) why Comparator extends comparable?
[ https://issues.apache.org/jira/browse/HBASE-20359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454188#comment-16454188 ] maoling commented on HBASE-20359: - [~Nick.han] can you give some examples? > why Comparator extends comparable? > -- > > Key: HBASE-20359 > URL: https://issues.apache.org/jira/browse/HBASE-20359 > Project: HBase > Issue Type: Improvement >Reporter: Nick.han >Priority: Minor > > Comparable is an interface,object implements this interface can be compare > with some other object, Comparator is a tool for compare object,the tool > itself can be compare is useless, so why a tool implement the comparable > interface?Do we have to improve this? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20483) [PERFORMANCE] Flushing is 2x slower in hbase2.
[ https://issues.apache.org/jira/browse/HBASE-20483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20483: -- Attachment: PerpetualFlushingProgram1.2.patch > [PERFORMANCE] Flushing is 2x slower in hbase2. > -- > > Key: HBASE-20483 > URL: https://issues.apache.org/jira/browse/HBASE-20483 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: stack >Assignee: stack >Priority: Major > Attachments: PerpetualFlushingProgram.patch, > PerpetualFlushingProgram1.2.patch, SnapshotSegmentScanner_2.0.0.patch > > > When flush is slow, memstores backup and go slower. See "1.2.7 vs. 2.0.0 > Flush History" > https://docs.google.com/spreadsheets/d/1sihTxb4aCplR3Rr_GGXkPlwhMIm-CbB9j_5339AS0Zc/edit#gid=1016758826 > for compare. First noted by [~anoop.hbase] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20483) [PERFORMANCE] Flushing is 2x slower in hbase2.
[ https://issues.apache.org/jira/browse/HBASE-20483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20483: -- Attachment: PerpetualFlushingProgram.patch > [PERFORMANCE] Flushing is 2x slower in hbase2. > -- > > Key: HBASE-20483 > URL: https://issues.apache.org/jira/browse/HBASE-20483 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: stack >Assignee: stack >Priority: Major > Attachments: PerpetualFlushingProgram.patch, > SnapshotSegmentScanner_2.0.0.patch > > > When flush is slow, memstores backup and go slower. See "1.2.7 vs. 2.0.0 > Flush History" > https://docs.google.com/spreadsheets/d/1sihTxb4aCplR3Rr_GGXkPlwhMIm-CbB9j_5339AS0Zc/edit#gid=1016758826 > for compare. First noted by [~anoop.hbase] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20483) [PERFORMANCE] Flushing is 2x slower in hbase2.
[ https://issues.apache.org/jira/browse/HBASE-20483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454010#comment-16454010 ] stack commented on HBASE-20483: --- Attached [~anoop.hbase]'s resort to ColelctionBackedScanner patch. > [PERFORMANCE] Flushing is 2x slower in hbase2. > -- > > Key: HBASE-20483 > URL: https://issues.apache.org/jira/browse/HBASE-20483 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: stack >Assignee: stack >Priority: Major > Attachments: SnapshotSegmentScanner_2.0.0.patch > > > When flush is slow, memstores backup and go slower. See "1.2.7 vs. 2.0.0 > Flush History" > https://docs.google.com/spreadsheets/d/1sihTxb4aCplR3Rr_GGXkPlwhMIm-CbB9j_5339AS0Zc/edit#gid=1016758826 > for compare. First noted by [~anoop.hbase] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20483) [PERFORMANCE] Flushing is 2x slower in hbase2.
[ https://issues.apache.org/jira/browse/HBASE-20483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20483: -- Attachment: SnapshotSegmentScanner_2.0.0.patch > [PERFORMANCE] Flushing is 2x slower in hbase2. > -- > > Key: HBASE-20483 > URL: https://issues.apache.org/jira/browse/HBASE-20483 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: stack >Assignee: stack >Priority: Major > Attachments: SnapshotSegmentScanner_2.0.0.patch > > > When flush is slow, memstores backup and go slower. See "1.2.7 vs. 2.0.0 > Flush History" > https://docs.google.com/spreadsheets/d/1sihTxb4aCplR3Rr_GGXkPlwhMIm-CbB9j_5339AS0Zc/edit#gid=1016758826 > for compare. First noted by [~anoop.hbase] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20457) Return immediately for a scan rpc call when we want to switch from pread to stream
[
https://issues.apache.org/jira/browse/HBASE-20457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16453982#comment-16453982
]
Hadoop QA commented on HBASE-20457:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
27s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
56s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 3s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
8s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}108m
12s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
38s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}172m 2s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20457 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12920765/HBASE-20457-v1.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux d84f41dbf227 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / f39ecac488 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_162 |
| findbugs |
[jira] [Commented] (HBASE-20482) Print a link to the ref guide chapter for the shell during startup
[ https://issues.apache.org/jira/browse/HBASE-20482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16453966#comment-16453966 ] Sean Busbey commented on HBASE-20482: - should the link be version specific? that is, the patch to master correctly points at the top-level reference guide but when backporting to e.g. branch-2.0 we'd link to http://hbase.apache.org/2.0/book.html > Print a link to the ref guide chapter for the shell during startup > -- > > Key: HBASE-20482 > URL: https://issues.apache.org/jira/browse/HBASE-20482 > Project: HBase > Issue Type: Task > Components: documentation, shell >Reporter: Sakthi >Assignee: Sakthi >Priority: Minor > Attachments: hbase-20482.master.001.patch, > hbase-20482.master.002.patch, hbase-20482.master.003.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20476) Fix the flaky TestReplicationSmallTests unit test
[ https://issues.apache.org/jira/browse/HBASE-20476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20476: -- Status: Patch Available (was: Open) A simple patch to increase the retry numbers. And the more important thing is that, set the sleep interval for SerialReplicationChecker.waitUntilCanPush to make the test run fast. The default value is 10s, I set it to 100ms. > Fix the flaky TestReplicationSmallTests unit test > - > > Key: HBASE-20476 > URL: https://issues.apache.org/jira/browse/HBASE-20476 > Project: HBase > Issue Type: Bug >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Attachments: HBASE-20476.patch > > > See > https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20476) Fix the flaky TestReplicationSmallTests unit test
[ https://issues.apache.org/jira/browse/HBASE-20476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20476: -- Attachment: HBASE-20476.patch > Fix the flaky TestReplicationSmallTests unit test > - > > Key: HBASE-20476 > URL: https://issues.apache.org/jira/browse/HBASE-20476 > Project: HBase > Issue Type: Bug >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Attachments: HBASE-20476.patch > > > See > https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20040) Master UI should include "Cluster Key" needed to use the cluster as a replication sink
[
https://issues.apache.org/jira/browse/HBASE-20040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16453959#comment-16453959
]
Sean Busbey commented on HBASE-20040:
-
patch doesn't work on branch-1 due to HBASE-16690. Would you mind making a
branch-1 patch [~jatsakthi]?
> Master UI should include "Cluster Key" needed to use the cluster as a
> replication sink
> --
>
> Key: HBASE-20040
> URL: https://issues.apache.org/jira/browse/HBASE-20040
> Project: HBase
> Issue Type: Improvement
> Components: Replication, Usability
>Reporter: Sean Busbey
>Assignee: Sakthi
>Priority: Minor
> Labels: beginner
> Fix For: 3.0.0, 2.1.0
>
> Attachments: hbase-20040.master.001.patch,
> hbase-20040.master.002.patch
>
>
> The ref guide defines a "Cluster Key" needed to add an hbase cluster as a
> replication peer
> {quote}
> CLUSTER_KEY: composed using the following template, with appropriate
> place-holders:
> {code}hbase.zookeeper.quorum:hbase.zookeeper.property.clientPort:zookeeper.znode.parent{code}
> {quote}
> the Master UI has all of the pieces displayed currently, but it should
> include a single field that operators can copy/paste.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
