[jira] [Commented] (HBASE-18375) The pool chunks from ChunkCreator are deallocated while in pool because there is no reference to them
[ https://issues.apache.org/jira/browse/HBASE-18375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226293#comment-16226293 ] Anoop Sam John commented on HBASE-18375: bq.FAILURE: Integrated in Jenkins build HBase-2.0 #467 (See https://builds.apache.org/job/HBase-2.0/467/) Ya seems this is already there in branch-2. Around 7th Sep. So which version? Alpha-3? [~stack]? > The pool chunks from ChunkCreator are deallocated while in pool because there > is no reference to them > - > > Key: HBASE-18375 > URL: https://issues.apache.org/jira/browse/HBASE-18375 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-alpha-1 >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky >Priority: Critical > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-18375-V01.patch, HBASE-18375-V02-branch-2.patch, > HBASE-18375-V02.patch, HBASE-18375-V03.patch, HBASE-18375-V04.patch, > HBASE-18375-V05.patch, HBASE-18375-V06.patch, HBASE-18375-V07.patch, > HBASE-18375-V08.patch, HBASE-18375-V09.patch, HBASE-18375-V10.patch, > HBASE-18375-V11.patch, HBASE-18375-branch-2.patch > > > Because MSLAB list of chunks was changed to list of chunk IDs, the chunks > returned back to pool can be deallocated by JVM because there is no reference > to them. The solution is to protect pool chunks from GC by the strong map of > ChunkCreator introduced by HBASE-18010. Will prepare the patch today. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18602) rsgroup cleanup unassign code
[
https://issues.apache.org/jira/browse/HBASE-18602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226287#comment-16226287
]
Hudson commented on HBASE-18602:
FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3975 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/3975/])
HBASE-18602 rsgroup cleanup unassign code (chia7712: rev
e5111b860891b2984f879e259c76aecf06bf97ba)
* (edit)
hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupBasedLoadBalancer.java
> rsgroup cleanup unassign code
> -
>
> Key: HBASE-18602
> URL: https://issues.apache.org/jira/browse/HBASE-18602
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Wang, Xinglong
>Assignee: Wang, Xinglong
>Priority: Minor
> Fix For: 2.0.0, 1.4.0, 1.5.0
>
> Attachments: HBASE-18602-master-v1.patch,
> HBASE-18602-master-v2.patch, HBASE-18602-master-v3.patch,
> HBASE-18602-master-v3.patch, HBASE-18602-master-v3.patch,
> HBASE-18602-master-v4.patch
>
>
> While walking through rsgroup code, I found that variable misplacedRegions
> has never been added any element into. This makes the unassign region code is
> not functional. And according to my test, it is actually unnecessary to do
> that.
> RSGroupBasedLoadBalancer.java
> {code:java}
> private Map correctAssignments(
>Map existingAssignments)
> throws HBaseIOException{
> Map correctAssignments = new TreeMap<>();
> List misplacedRegions = new LinkedList<>();
> correctAssignments.put(LoadBalancer.BOGUS_SERVER_NAME, new
> LinkedList<>());
> for (Map.Entry assignments :
> existingAssignments.entrySet()){
> ServerName sName = assignments.getKey();
> correctAssignments.put(sName, new LinkedList<>());
> List regions = assignments.getValue();
> for (HRegionInfo region : regions) {
> RSGroupInfo info = null;
> try {
> info = rsGroupInfoManager.getRSGroup(
> rsGroupInfoManager.getRSGroupOfTable(region.getTable()));
> } catch (IOException exp) {
> LOG.debug("RSGroup information null for region of table " +
> region.getTable(),
> exp);
> }
> if ((info == null) || (!info.containsServer(sName.getAddress( {
> correctAssignments.get(LoadBalancer.BOGUS_SERVER_NAME).add(region);
> } else {
> correctAssignments.get(sName).add(region);
> }
> }
> }
> //TODO bulk unassign?
> //unassign misplaced regions, so that they are assigned to correct groups.
> for(HRegionInfo info: misplacedRegions) {
> try {
> this.masterServices.getAssignmentManager().unassign(info);
> } catch (IOException e) {
> throw new HBaseIOException(e);
> }
> }
> return correctAssignments;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19125) TestReplicator is flaky
[ https://issues.apache.org/jira/browse/HBASE-19125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226288#comment-16226288 ] Hudson commented on HBASE-19125: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3975 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3975/]) HBASE-19125 TestReplicator is flaky (apurtell: rev 14368c432a6f0fe41f9e627ed53d3317d0853212) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicator.java > TestReplicator is flaky > --- > > Key: HBASE-19125 > URL: https://issues.apache.org/jira/browse/HBASE-19125 > Project: HBase > Issue Type: Bug >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0 > > > TestReplicator fails now and again. I had a look at the test. This is > something I contributed a while back but looking at it again it needs a > different approach. I'm going to disable it for now until this issue is > resolved. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18870) Hbase Backup should set the details to MR job name
[ https://issues.apache.org/jira/browse/HBASE-18870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226286#comment-16226286 ] Hudson commented on HBASE-18870: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3975 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3975/]) HBASE-18870 Hbase Backup should set the details to MR job name (Vishal (tedyu: rev 6712f8f632e6ece032ee45f679db9a42a69f805f) * (edit) hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/BackupRestoreConstants.java * (edit) hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/snapshot/ExportSnapshot.java * (edit) hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/RestoreTablesClient.java * (edit) hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/IncrementalTableBackupClient.java * (edit) hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/FullTableBackupClient.java > Hbase Backup should set the details to MR job name > -- > > Key: HBASE-18870 > URL: https://issues.apache.org/jira/browse/HBASE-18870 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: Vishal Khandelwal >Assignee: Vishal Khandelwal >Priority: Minor > Labels: backup > Fix For: 2.0.0-alpha-4 > > Attachments: 18870.v3.txt, HBASE-18870.branch-2.v1.patch, > HBASE-18870.branch-2.v2.patch > > > For Incremental and Full Backups job names should be set correctly. It would > make the debug and monitoring better. > Current : > incremental -> WALPlayer_1506332227619 > ExportSnapshot-snapshot_1506323760362_default_t1 > Proposed : > incremental -> Backup_Incremental__ > Full -> Backups_Full__ -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19129) TestChoreService is flaky (branch-1 / branch-1.4)
[ https://issues.apache.org/jira/browse/HBASE-19129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226289#comment-16226289 ] Hudson commented on HBASE-19129: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3975 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3975/]) HBASE-19129 TestChoreService is flaky (apurtell: rev 98c01bd4cfadc1253b1e7a60594c63167c299098) * (edit) hbase-common/src/test/java/org/apache/hadoop/hbase/TestChoreService.java > TestChoreService is flaky (branch-1 / branch-1.4) > - > > Key: HBASE-19129 > URL: https://issues.apache.org/jira/browse/HBASE-19129 > Project: HBase > Issue Type: Bug >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Trivial > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0 > > Attachments: HBASE-19129.patch > > > Rare flake. > [ERROR] Tests run: 17, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: > 21.788 s <<< FAILURE! - in org.apache.hadoop.hbase.TestChoreService > [ERROR] testForceTrigger(org.apache.hadoop.hbase.TestChoreService) Time > elapsed: 1.044 s <<< FAILURE! > java.lang.AssertionError > at > org.apache.hadoop.hbase.TestChoreService.testForceTrigger(TestChoreService.java:381) > Increase the delta to compensate for environmental variance. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19092) Make Tag IA.LimitedPrivate and expose for CPs
[ https://issues.apache.org/jira/browse/HBASE-19092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226279#comment-16226279 ] stack commented on HBASE-19092: --- Yes. Was thinking getCellBuilder. They'd maybe need a tags builder too? Or can the CellBuilder build Tags? > Make Tag IA.LimitedPrivate and expose for CPs > - > > Key: HBASE-19092 > URL: https://issues.apache.org/jira/browse/HBASE-19092 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-19092-branch-2.patch, > HBASE-19092_001-branch-2.patch > > > We need to make tags as LimitedPrivate as some use cases are trying to use > tags like timeline server. The same topic was discussed in dev@ and also in > HBASE-18995. > Shall we target this for beta1 - cc [~saint@gmail.com]. > So once we do this all related Util methods and APIs should also move to > LimitedPrivate Util classes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19092) Make Tag IA.LimitedPrivate and expose for CPs
[ https://issues.apache.org/jira/browse/HBASE-19092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226274#comment-16226274 ] ramkrishna.s.vasudevan commented on HBASE-19092: Ok seeing your comment now. Will read it and be back. The last one from my side was posted before seeing yours. > Make Tag IA.LimitedPrivate and expose for CPs > - > > Key: HBASE-19092 > URL: https://issues.apache.org/jira/browse/HBASE-19092 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-19092-branch-2.patch, > HBASE-19092_001-branch-2.patch > > > We need to make tags as LimitedPrivate as some use cases are trying to use > tags like timeline server. The same topic was discussed in dev@ and also in > HBASE-18995. > Shall we target this for beta1 - cc [~saint@gmail.com]. > So once we do this all related Util methods and APIs should also move to > LimitedPrivate Util classes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18995) Move methods that are for internal usage from CellUtil to Private util class
[ https://issues.apache.org/jira/browse/HBASE-18995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226275#comment-16226275 ] stack commented on HBASE-18995: --- bq. I think createFirstOnRow is the only one that is missed? Yes. I see too why you'd want to hide the others Crunch is using. We need to help Crunch get up on the factory and builders. It is fair to call it client-side code (MR/Spark and misc). Thanks. > Move methods that are for internal usage from CellUtil to Private util class > > > Key: HBASE-18995 > URL: https://issues.apache.org/jira/browse/HBASE-18995 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-alpha-3 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18995-branch-2.002.patch, > HBASE-18995-branch-2.patch, HBASE-18995-branch-2_1.patch, > HBASE-18995-branch-2_1.patch, HBASE-18995-branch-2_1.patch, > HBASE-18995-branch-2_1.patch, HBASE-18995-branch-2_2.patch, > HBASE-18995_002-branch-2.patch, HBASE-18995_002-branch-2.patch, > HBASE-18995_003-branch-2.patch, HBASE-18995_1.patch, HBASE-18995_2.patch, > HBASE-18995_2.patch > > > This was brought up long time back. We need to move some of the public APIs > from CellUtil to internal private Util class because they are used in some > internal flow and does not make sense to have it in a @public exposed Util > class. > The topic again came in HBASE-18945 RB comments also. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19092) Make Tag IA.LimitedPrivate and expose for CPs
[ https://issues.apache.org/jira/browse/HBASE-19092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226273#comment-16226273 ] ramkrishna.s.vasudevan commented on HBASE-19092: bq.Could even filter it via the CP Env to remove the SequenceId method Or throw exception if they try to mess w/ it. This part am not very clear. The recent CP Env changes are helping to have richer API for Core CPs and they expose OnlineRegions, and other RegionServerServices. So you mean we need to do something like that ? > Make Tag IA.LimitedPrivate and expose for CPs > - > > Key: HBASE-19092 > URL: https://issues.apache.org/jira/browse/HBASE-19092 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-19092-branch-2.patch, > HBASE-19092_001-branch-2.patch > > > We need to make tags as LimitedPrivate as some use cases are trying to use > tags like timeline server. The same topic was discussed in dev@ and also in > HBASE-18995. > Shall we target this for beta1 - cc [~saint@gmail.com]. > So once we do this all related Util methods and APIs should also move to > LimitedPrivate Util classes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19092) Make Tag IA.LimitedPrivate and expose for CPs
[ https://issues.apache.org/jira/browse/HBASE-19092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226269#comment-16226269 ] stack commented on HBASE-19092: --- bq. Which means for CPs we don't give them a chance to build a tag. We have to. Timeline Server needs it. bq. I think making ECB LimitedPrivate is not right IMHO. What do you think? I think we are too late for alpha-4. There is too much to do in here still. We should just punt this to beta-1. Also Tags by nature (server-side-only) requires ECB access I'd say. ECB is an Interface. Making it LP seems fine. And you have a factory. Can't you just add getCellBuilder(type) to RegionCoprocessorEnvironment? It'd return a CellBuilder that the CP could use. It would be all LP Interfaces? Then no need of the LP*Util stuff? Can I have Tag builder, something that will make simple Tag [] to give to the CellBuilder? There is none in the patch. Can we have a smaller patch? What you reckon? Thanks [~ram_krish]. > Make Tag IA.LimitedPrivate and expose for CPs > - > > Key: HBASE-19092 > URL: https://issues.apache.org/jira/browse/HBASE-19092 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-19092-branch-2.patch, > HBASE-19092_001-branch-2.patch > > > We need to make tags as LimitedPrivate as some use cases are trying to use > tags like timeline server. The same topic was discussed in dev@ and also in > HBASE-18995. > Shall we target this for beta1 - cc [~saint@gmail.com]. > So once we do this all related Util methods and APIs should also move to > LimitedPrivate Util classes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18995) Move methods that are for internal usage from CellUtil to Private util class
[ https://issues.apache.org/jira/browse/HBASE-18995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226264#comment-16226264 ] ramkrishna.s.vasudevan commented on HBASE-18995: bq. I see three methods in CellUtil with Private annotation. Any reason they were not moved? I see they are @Deprecated. Is that because they have been moved already? I left it because they are still exposed in 2.0. You mean they are left over in 3.0 also? If so it is my mistake. I have to remove them. bq.Are there methods from PrivateCellUtil you want to expose to Coprocessors? As said in the other JIRA. We need to know what others can we expose. We can definitely move some more to the LPCellUtil which needs to be exposed but for now I can see Tag related APIs. But coming to createfirstOnRow(), createLastOnRow() - that is something internal. Why should that be in CellUtil? In one of the review comments it was discussed we need not expose them. I am fine if there is an usage but again there are lot of variants which are not worth exposing we need to see which are really useful and which are internally used. In the list of APIs used by Crunch I think createFirstOnRow is the only one that is missed? All other APIs are still available in Public CellUtil. So this project Crunch is it a CP based project or is it a client side code? bq.(static methods on interfaces is one of the benefits of us being on jdk8 now: ref) I like this idea. How ever I think what needs to go as static methods in Cell interface is still questionable because there are cases like this createFirstOnRow() which goes to IA.Private but Cell does not have it and still it is not helping them. > Move methods that are for internal usage from CellUtil to Private util class > > > Key: HBASE-18995 > URL: https://issues.apache.org/jira/browse/HBASE-18995 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-alpha-3 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18995-branch-2.002.patch, > HBASE-18995-branch-2.patch, HBASE-18995-branch-2_1.patch, > HBASE-18995-branch-2_1.patch, HBASE-18995-branch-2_1.patch, > HBASE-18995-branch-2_1.patch, HBASE-18995-branch-2_2.patch, > HBASE-18995_002-branch-2.patch, HBASE-18995_002-branch-2.patch, > HBASE-18995_003-branch-2.patch, HBASE-18995_1.patch, HBASE-18995_2.patch, > HBASE-18995_2.patch > > > This was brought up long time back. We need to move some of the public APIs > from CellUtil to internal private Util class because they are used in some > internal flow and does not make sense to have it in a @public exposed Util > class. > The topic again came in HBASE-18945 RB comments also. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19092) Make Tag IA.LimitedPrivate and expose for CPs
[ https://issues.apache.org/jira/browse/HBASE-19092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226254#comment-16226254 ] ramkrishna.s.vasudevan commented on HBASE-19092: bq.At this stage I'd say just let them have ECB. Could even filter it via the CP Env to remove the SequenceId method Or throw exception if they try to mess w/ it. Ok. So i wont add any new builder and will leave ECB as private for now. Which means for CPs we don't give them a chance to build a tag. For now they can read the tags? Is that fine? I think making ECB LimitedPrivate is not right IMHO. What do you think? > Make Tag IA.LimitedPrivate and expose for CPs > - > > Key: HBASE-19092 > URL: https://issues.apache.org/jira/browse/HBASE-19092 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-19092-branch-2.patch, > HBASE-19092_001-branch-2.patch > > > We need to make tags as LimitedPrivate as some use cases are trying to use > tags like timeline server. The same topic was discussed in dev@ and also in > HBASE-18995. > Shall we target this for beta1 - cc [~saint@gmail.com]. > So once we do this all related Util methods and APIs should also move to > LimitedPrivate Util classes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19118) Use SaslUtil to set Sasl.QOP in 'Thrift'
[
https://issues.apache.org/jira/browse/HBASE-19118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226251#comment-16226251
]
Ted Yu commented on HBASE-19118:
lgtm
> Use SaslUtil to set Sasl.QOP in 'Thrift'
>
>
> Key: HBASE-19118
> URL: https://issues.apache.org/jira/browse/HBASE-19118
> Project: HBase
> Issue Type: Bug
> Components: Thrift
>Affects Versions: 1.2.6
>Reporter: Reid Chan
>Assignee: Reid Chan
> Attachments: HBASE-19118.master.001.patch,
> HBASE-19118.master.002.patch, HBASE-19118.master.003.patch
>
>
> In [Configure the Thrift
> Gateway|http://hbase.apache.org/book.html#security.client.thrift], it says
> "set the property {{hbase.thrift.security.qop}} to one of the following three
> values: {{privacy}}, {{integrity}}, {{authentication}}", which would lead to
> failure of starting up a thrift server.
> In fact, the value of {{hbase.thrift.security.qop}} should be {{auth}},
> {{auth-int}}, {{auth-conf}}, according to the documentation of {{Sasl.QOP}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19114) Split out o.a.h.h.zookeeper from hbase-server and hbase-client
[
https://issues.apache.org/jira/browse/HBASE-19114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226243#comment-16226243
]
Hadoop QA commented on HBASE-19114:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m
42s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 61 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 6m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 11m
43s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
20s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
24s{color} | {color:red} hbase-common: The patch generated 11 new + 4 unchanged
- 0 fixed = 15 total (was 4) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
15s{color} | {color:red} hbase-zookeeper: The patch generated 207 new + 0
unchanged - 0 fixed = 207 total (was 0) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
49s{color} | {color:red} root: The patch generated 322 new + 2122 unchanged -
268 fixed = 2444 total (was 2390) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} The patch hbase-assembly passed checkstyle {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
37s{color} | {color:red} hbase-client: The patch generated 11 new + 640
unchanged - 171 fixed = 651 total (was 811) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} hbase-endpoint: The patch generated 0 new + 2
unchanged - 1 fixed = 2 total (was 3) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} The patch hbase-examples passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} The patch hbase-it passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} The patch hbase-mapreduce passed checkstyle {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
12s{color} | {color:red} hbase-replication: The patch generated 6 new + 21
unchanged - 2 fixed = 27 total (was 23) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
11s{color} | {color:red} hbase-rsgroup: The patch generated 2 new + 8 unchanged
- 0 fixed = 10 total (was 8) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
27s{color} | {color:red} hbase-server: The patch generated 85 new + 1430
unchanged - 94 fixed = 1515 total (was 1524) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} |
[jira] [Commented] (HBASE-18233) We shouldn't wait for readlock in doMiniBatchMutation in case of deadlock
[ https://issues.apache.org/jira/browse/HBASE-18233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226241#comment-16226241 ] Sean Busbey commented on HBASE-18233: - I started another build after bumping the timeout to 7 hours: https://builds.apache.org/job/PreCommit-HBASE-Build/9528/ > We shouldn't wait for readlock in doMiniBatchMutation in case of deadlock > - > > Key: HBASE-18233 > URL: https://issues.apache.org/jira/browse/HBASE-18233 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.7 >Reporter: Allan Yang >Assignee: Allan Yang >Priority: Blocker > Fix For: 1.3.2, 1.2.7 > > Attachments: HBASE-18233-branch-1.2.patch, > HBASE-18233-branch-1.2.v2.patch, HBASE-18233-branch-1.2.v3.patch, > HBASE-18233-branch-1.2.v4 (1).patch, HBASE-18233-branch-1.2.v4 (1).patch, > HBASE-18233-branch-1.2.v4 (2).patch, HBASE-18233-branch-1.2.v4.patch, > HBASE-18233-branch-1.2.v4.patch, HBASE-18233-branch-1.2.v4.patch, > HBASE-18233-branch-1.2.v4.patch, HBASE-18233-branch-1.2.v4.patch, > HBASE-18233-branch-1.2.v4.patch, HBASE-18233-branch-1.2.v4.patch > > > Please refer to the discuss in HBASE-18144 > https://issues.apache.org/jira/browse/HBASE-18144?focusedCommentId=16051701=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16051701 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19118) Use SaslUtil to set Sasl.QOP in 'Thrift'
[
https://issues.apache.org/jira/browse/HBASE-19118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226228#comment-16226228
]
Reid Chan commented on HBASE-19118:
---
Yes, successfully.
> Use SaslUtil to set Sasl.QOP in 'Thrift'
>
>
> Key: HBASE-19118
> URL: https://issues.apache.org/jira/browse/HBASE-19118
> Project: HBase
> Issue Type: Bug
> Components: Thrift
>Affects Versions: 1.2.6
>Reporter: Reid Chan
>Assignee: Reid Chan
> Attachments: HBASE-19118.master.001.patch,
> HBASE-19118.master.002.patch, HBASE-19118.master.003.patch
>
>
> In [Configure the Thrift
> Gateway|http://hbase.apache.org/book.html#security.client.thrift], it says
> "set the property {{hbase.thrift.security.qop}} to one of the following three
> values: {{privacy}}, {{integrity}}, {{authentication}}", which would lead to
> failure of starting up a thrift server.
> In fact, the value of {{hbase.thrift.security.qop}} should be {{auth}},
> {{auth-int}}, {{auth-conf}}, according to the documentation of {{Sasl.QOP}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19118) Use SaslUtil to set Sasl.QOP in 'Thrift'
[
https://issues.apache.org/jira/browse/HBASE-19118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226221#comment-16226221
]
Reid Chan commented on HBASE-19118:
---
Without this patch, only one of these values: auth/auth-int/auth-conf would
bring up thrift server successfully. Otherwise:
{code}
Exception in thread "main" java.io.IOException: Invalid
hbase.thrift.security.qop: authentication, it must be 'auth', 'auth-int', or
'auth-conf'
at
org.apache.hadoop.hbase.thrift.ThriftServerRunner.(ThriftServerRunner.java:333)
at
org.apache.hadoop.hbase.thrift.ThriftServer.doMain(ThriftServer.java:95)
at
org.apache.hadoop.hbase.thrift.ThriftServer.main(ThriftServer.java:240)
{code}
With this patch, "auth/auth-int/auth-conf" or
"authentication/integrity/privacy" works the same.
> Use SaslUtil to set Sasl.QOP in 'Thrift'
>
>
> Key: HBASE-19118
> URL: https://issues.apache.org/jira/browse/HBASE-19118
> Project: HBase
> Issue Type: Bug
> Components: Thrift
>Affects Versions: 1.2.6
>Reporter: Reid Chan
>Assignee: Reid Chan
> Attachments: HBASE-19118.master.001.patch,
> HBASE-19118.master.002.patch, HBASE-19118.master.003.patch
>
>
> In [Configure the Thrift
> Gateway|http://hbase.apache.org/book.html#security.client.thrift], it says
> "set the property {{hbase.thrift.security.qop}} to one of the following three
> values: {{privacy}}, {{integrity}}, {{authentication}}", which would lead to
> failure of starting up a thrift server.
> In fact, the value of {{hbase.thrift.security.qop}} should be {{auth}},
> {{auth-int}}, {{auth-conf}}, according to the documentation of {{Sasl.QOP}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19118) Use SaslUtil to set Sasl.QOP in 'Thrift'
[
https://issues.apache.org/jira/browse/HBASE-19118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226211#comment-16226211
]
Josh Elser commented on HBASE-19118:
Ahh, sorry about that. I see that on second glance. I was mislead by the
following :)
{noformat}
+ if (qop != QualityOfProtection.AUTHENTICATION &&
+ qop != QualityOfProtection.INTEGRITY &&
+ qop != QualityOfProtection.PRIVACY) {
{noformat}
bq. I'm not sure there's a unit test for kerberized thrift server, but i'm
sure there's an example DemoClient in hbase-example which i'm using as a test.
A shame if we don't have one :(. You've tested by hand with the old and new
properties successfully?
> Use SaslUtil to set Sasl.QOP in 'Thrift'
>
>
> Key: HBASE-19118
> URL: https://issues.apache.org/jira/browse/HBASE-19118
> Project: HBase
> Issue Type: Bug
> Components: Thrift
>Affects Versions: 1.2.6
>Reporter: Reid Chan
>Assignee: Reid Chan
> Attachments: HBASE-19118.master.001.patch,
> HBASE-19118.master.002.patch, HBASE-19118.master.003.patch
>
>
> In [Configure the Thrift
> Gateway|http://hbase.apache.org/book.html#security.client.thrift], it says
> "set the property {{hbase.thrift.security.qop}} to one of the following three
> values: {{privacy}}, {{integrity}}, {{authentication}}", which would lead to
> failure of starting up a thrift server.
> In fact, the value of {{hbase.thrift.security.qop}} should be {{auth}},
> {{auth-int}}, {{auth-conf}}, according to the documentation of {{Sasl.QOP}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18995) Move methods that are for internal usage from CellUtil to Private util class
[ https://issues.apache.org/jira/browse/HBASE-18995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226208#comment-16226208 ] Sean Busbey commented on HBASE-18995: - Moving stuff into PrivateCellUtil is a bad idea, IMHO. Method level annotations work fine. If someone is going to ignore a method being IA.Private, they're also going to ignore a class being IA.Private. We already hide them from our downstream facing javadocs. If we need more, we can invest the time in making an enforcer rule that folks can opt-into. If we want to make some CellUtil methods "more public" why not just move them to be static methods on the Cell interface and mark all of CellUtil as deprecated / going IA.Private in HBase 3? (static methods on interfaces is one of the benefits of us being on jdk8 now: [ref|https://docs.oracle.com/javase/tutorial/java/IandI/defaultmethods.html#static]) This has the nice side effect of starting to undo all of the FooUtil classes that litter our codebase. > Move methods that are for internal usage from CellUtil to Private util class > > > Key: HBASE-18995 > URL: https://issues.apache.org/jira/browse/HBASE-18995 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-alpha-3 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18995-branch-2.002.patch, > HBASE-18995-branch-2.patch, HBASE-18995-branch-2_1.patch, > HBASE-18995-branch-2_1.patch, HBASE-18995-branch-2_1.patch, > HBASE-18995-branch-2_1.patch, HBASE-18995-branch-2_2.patch, > HBASE-18995_002-branch-2.patch, HBASE-18995_002-branch-2.patch, > HBASE-18995_003-branch-2.patch, HBASE-18995_1.patch, HBASE-18995_2.patch, > HBASE-18995_2.patch > > > This was brought up long time back. We need to move some of the public APIs > from CellUtil to internal private Util class because they are used in some > internal flow and does not make sense to have it in a @public exposed Util > class. > The topic again came in HBASE-18945 RB comments also. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19102) TestZooKeeperMainServer fails with KeeperException$ConnectionLossException
[ https://issues.apache.org/jira/browse/HBASE-19102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19102: -- Attachment: HBASE-19102.master.002.patch > TestZooKeeperMainServer fails with KeeperException$ConnectionLossException > -- > > Key: HBASE-19102 > URL: https://issues.apache.org/jira/browse/HBASE-19102 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Attachments: HBASE-19102.master.001.patch, > HBASE-19102.master.002.patch > > > I'm trying to run test suite on a local machine. I never get to the second > part because I fail on below test with below exception near every time (and > an ipv6 test... will do that next). > 1 > --- > 2 Test set: org.apache.hadoop.hbase.zookeeper.TestZooKeeperMainServer > 3 > --- > 4 Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 16.161 s > <<< FAILURE! - in org.apache.hadoop.hbase.zookeeper.TestZooKeeperMainServer > 5 > testCommandLineWorks(org.apache.hadoop.hbase.zookeeper.TestZooKeeperMainServer) > Time elapsed: 15.848 s <<< ERROR! > 6 org.apache.zookeeper.KeeperException$ConnectionLossException: > KeeperErrorCode = ConnectionLoss for /testCommandLineWorks > 7 at > org.apache.hadoop.hbase.zookeeper.TestZooKeeperMainServer.testCommandLineWorks(TestZooKeeperMainServer.java:81) > Looks like running the command before we are connected causes the above -- we > pause 15 seconds and then throw the above. If I wait until connected before > proceding, stuff seems to work reliably. I don't have access to the watcher > on connections since we override the zk main class... so this seems only > avenue available at mo (This zk main thing is all a hack around zk main > because it had bugs ... but I think we have to keep the hack because folks > use different versions of zk. My $workplace defaults to something that is > years old, 3.4.5 for instance). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19118) Use SaslUtil to set Sasl.QOP in 'Thrift'
[
https://issues.apache.org/jira/browse/HBASE-19118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226206#comment-16226206
]
Reid Chan commented on HBASE-19118:
---
Thanks for reply! [~elserj]
In {{SaslUtil}}
{code}
public boolean matches(String stringQop) {
if (saslQop.equals(stringQop)) {
LOG.warn("Use authentication/integrity/privacy as value for rpc
protection "
+ "configurations instead of auth/auth-int/auth-conf.");
return true;
}
return name().equalsIgnoreCase(stringQop);
}
{code}
So, it will not break up BC.
I'm not sure there's a unit test for kerberized thrift server, but i'm sure
there's an example {{DemoClient}} in hbase-example which i'm using as a test.
> Use SaslUtil to set Sasl.QOP in 'Thrift'
>
>
> Key: HBASE-19118
> URL: https://issues.apache.org/jira/browse/HBASE-19118
> Project: HBase
> Issue Type: Bug
> Components: Thrift
>Affects Versions: 1.2.6
>Reporter: Reid Chan
>Assignee: Reid Chan
> Attachments: HBASE-19118.master.001.patch,
> HBASE-19118.master.002.patch, HBASE-19118.master.003.patch
>
>
> In [Configure the Thrift
> Gateway|http://hbase.apache.org/book.html#security.client.thrift], it says
> "set the property {{hbase.thrift.security.qop}} to one of the following three
> values: {{privacy}}, {{integrity}}, {{authentication}}", which would lead to
> failure of starting up a thrift server.
> In fact, the value of {{hbase.thrift.security.qop}} should be {{auth}},
> {{auth-int}}, {{auth-conf}}, according to the documentation of {{Sasl.QOP}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18770) Remove bypass method in ObserverContext and implement the 'bypass' logic case by case
[
https://issues.apache.org/jira/browse/HBASE-18770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226205#comment-16226205
]
Hadoop QA commented on HBASE-18770:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 4m
38s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
51s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
45s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
14s{color} | {color:red} hbase-server: The patch generated 73 new + 1005
unchanged - 75 fixed = 1078 total (was 1080) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
45s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
58m 2s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
46s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}133m 5s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
19s{color} | {color:green} hbase-endpoint in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
43s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}233m 32s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-18770 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12894870/HBASE-18770.master.011.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux a237e620c77c 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[jira] [Comment Edited] (HBASE-18770) Remove bypass method in ObserverContext and implement the 'bypass' logic case by case
[ https://issues.apache.org/jira/browse/HBASE-18770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226195#comment-16226195 ] stack edited comment on HBASE-18770 at 10/31/17 3:47 AM: - On a read-only WALKey, I think I'll skip. It is yet another object creation on the write-path or if I were to make WALKey into an Interface of read-only methods, it is a bunch of work given WALKey is serialized to HFiles up to this. This WALKey as-is we want to take internal for hbase3 at least and probably redo since its so fat. WALKey is already IA.Private. A CP could try setting compression context, original sequence id (needed at WAL Replay time), and replication scope can be edited as the WAL Key is replicated to different peers. I added a warning on each CP method that WALKey should not be modified [~chia7712] Let me know if this ok. I could file a follow-on and schedule it for beta-1. was (Author: stack): On a read-only WALKey, I think I'll skip. It is yet another object creation on the write-path or if I were to make WALKey into an Interface of read-only methods, it is a bunch of work given WALKey is serialized to HFiles up to this. This WALKey as-is we want to take internal for hbase3 at least and probably redo since its so fat. WALKey is already IA.Private. A CP could try setting compression context, original sequence id (needed at WAL Replay time), and replication scope can be edited as the WAL Key is replicated to different peers. I added a warning on each CP method that WALKey should not be modified. > Remove bypass method in ObserverContext and implement the 'bypass' logic case > by case > - > > Key: HBASE-18770 > URL: https://issues.apache.org/jira/browse/HBASE-18770 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Assignee: stack >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18770.master.001.patch, > HBASE-18770.master.002.patch, HBASE-18770.master.003.patch, > HBASE-18770.master.004.patch, HBASE-18770.master.005.patch, > HBASE-18770.master.006.patch, HBASE-18770.master.007.patch, > HBASE-18770.master.008.patch, HBASE-18770.master.009.patch, > HBASE-18770.master.010.patch, HBASE-18770.master.011.patch > > > http://search-hadoop.com/m/HBase/YGbbXd0RDCIHSC1 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19118) Use SaslUtil to set Sasl.QOP in 'Thrift'
[
https://issues.apache.org/jira/browse/HBASE-19118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226200#comment-16226200
]
Josh Elser commented on HBASE-19118:
[~reidchan], while I agree with your choice in making the Thrift QoP properties
the same as the RPC server's QoP properties, could you fix this in a backwards
compatible way? It would be a shame to break someone with an functioning
configuration for the sake of a bug-fix.
We don't have any tests that set up a Thrift server with security?
> Use SaslUtil to set Sasl.QOP in 'Thrift'
>
>
> Key: HBASE-19118
> URL: https://issues.apache.org/jira/browse/HBASE-19118
> Project: HBase
> Issue Type: Bug
> Components: Thrift
>Affects Versions: 1.2.6
>Reporter: Reid Chan
>Assignee: Reid Chan
> Attachments: HBASE-19118.master.001.patch,
> HBASE-19118.master.002.patch, HBASE-19118.master.003.patch
>
>
> In [Configure the Thrift
> Gateway|http://hbase.apache.org/book.html#security.client.thrift], it says
> "set the property {{hbase.thrift.security.qop}} to one of the following three
> values: {{privacy}}, {{integrity}}, {{authentication}}", which would lead to
> failure of starting up a thrift server.
> In fact, the value of {{hbase.thrift.security.qop}} should be {{auth}},
> {{auth-int}}, {{auth-conf}}, according to the documentation of {{Sasl.QOP}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18770) Remove bypass method in ObserverContext and implement the 'bypass' logic case by case
[ https://issues.apache.org/jira/browse/HBASE-18770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226195#comment-16226195 ] stack commented on HBASE-18770: --- On a read-only WALKey, I think I'll skip. It is yet another object creation on the write-path or if I were to make WALKey into an Interface of read-only methods, it is a bunch of work given WALKey is serialized to HFiles up to this. This WALKey as-is we want to take internal for hbase3 at least and probably redo since its so fat. WALKey is already IA.Private. A CP could try setting compression context, original sequence id (needed at WAL Replay time), and replication scope can be edited as the WAL Key is replicated to different peers. I added a warning on each CP method that WALKey should not be modified. > Remove bypass method in ObserverContext and implement the 'bypass' logic case > by case > - > > Key: HBASE-18770 > URL: https://issues.apache.org/jira/browse/HBASE-18770 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Assignee: stack >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18770.master.001.patch, > HBASE-18770.master.002.patch, HBASE-18770.master.003.patch, > HBASE-18770.master.004.patch, HBASE-18770.master.005.patch, > HBASE-18770.master.006.patch, HBASE-18770.master.007.patch, > HBASE-18770.master.008.patch, HBASE-18770.master.009.patch, > HBASE-18770.master.010.patch, HBASE-18770.master.011.patch > > > http://search-hadoop.com/m/HBase/YGbbXd0RDCIHSC1 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18925) Need updated mockito for using java optional
[
https://issues.apache.org/jira/browse/HBASE-18925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226189#comment-16226189
]
Hadoop QA commented on HBASE-18925:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m
0s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 56 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
37s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 6m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 11m
3s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m
21s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
27s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
27s{color} | {color:red} hbase-client: The patch generated 3 new + 18 unchanged
- 1 fixed = 21 total (was 19) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
12s{color} | {color:red} hbase-http: The patch generated 1 new + 57 unchanged -
0 fixed = 58 total (was 57) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
9s{color} | {color:red} hbase-server: The patch generated 21 new + 442
unchanged - 21 fixed = 463 total (was 463) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
17s{color} | {color:red} hbase-mapreduce: The patch generated 1 new + 102
unchanged - 0 fixed = 103 total (was 102) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
23s{color} | {color:red} root: The patch generated 26 new + 644 unchanged - 22
fixed = 670 total (was 666) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
15s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
51s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
48m 9s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m
46s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}108m 14s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}215m 45s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests
[jira] [Commented] (HBASE-19125) TestReplicator is flaky
[ https://issues.apache.org/jira/browse/HBASE-19125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226182#comment-16226182 ] Hudson commented on HBASE-19125: FAILURE: Integrated in Jenkins build HBase-1.5 #124 (See [https://builds.apache.org/job/HBase-1.5/124/]) HBASE-19125 TestReplicator is flaky (apurtell: rev 826e11a15d8ab01f9d47f808128d9c9074259a05) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicator.java > TestReplicator is flaky > --- > > Key: HBASE-19125 > URL: https://issues.apache.org/jira/browse/HBASE-19125 > Project: HBase > Issue Type: Bug >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0 > > > TestReplicator fails now and again. I had a look at the test. This is > something I contributed a while back but looking at it again it needs a > different approach. I'm going to disable it for now until this issue is > resolved. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19129) TestChoreService is flaky (branch-1 / branch-1.4)
[ https://issues.apache.org/jira/browse/HBASE-19129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226183#comment-16226183 ] Hudson commented on HBASE-19129: FAILURE: Integrated in Jenkins build HBase-1.5 #124 (See [https://builds.apache.org/job/HBase-1.5/124/]) HBASE-19129 TestChoreService is flaky (apurtell: rev d099263e13546403da4d9ccf689a652be2505b24) * (edit) hbase-common/src/test/java/org/apache/hadoop/hbase/TestChoreService.java > TestChoreService is flaky (branch-1 / branch-1.4) > - > > Key: HBASE-19129 > URL: https://issues.apache.org/jira/browse/HBASE-19129 > Project: HBase > Issue Type: Bug >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Trivial > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0 > > Attachments: HBASE-19129.patch > > > Rare flake. > [ERROR] Tests run: 17, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: > 21.788 s <<< FAILURE! - in org.apache.hadoop.hbase.TestChoreService > [ERROR] testForceTrigger(org.apache.hadoop.hbase.TestChoreService) Time > elapsed: 1.044 s <<< FAILURE! > java.lang.AssertionError > at > org.apache.hadoop.hbase.TestChoreService.testForceTrigger(TestChoreService.java:381) > Increase the delta to compensate for environmental variance. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19125) TestReplicator is flaky
[ https://issues.apache.org/jira/browse/HBASE-19125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226176#comment-16226176 ] Hudson commented on HBASE-19125: SUCCESS: Integrated in Jenkins build HBase-1.4 #980 (See [https://builds.apache.org/job/HBase-1.4/980/]) HBASE-19125 TestReplicator is flaky (apurtell: rev fd899125a10476dc97d7eb7ee7b2fe4e629735dc) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicator.java > TestReplicator is flaky > --- > > Key: HBASE-19125 > URL: https://issues.apache.org/jira/browse/HBASE-19125 > Project: HBase > Issue Type: Bug >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0 > > > TestReplicator fails now and again. I had a look at the test. This is > something I contributed a while back but looking at it again it needs a > different approach. I'm going to disable it for now until this issue is > resolved. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18770) Remove bypass method in ObserverContext and implement the 'bypass' logic case by case
[ https://issues.apache.org/jira/browse/HBASE-18770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226170#comment-16226170 ] stack commented on HBASE-18770: --- bq. If CP bypass, we skip the real operation so it don't need update metrics (I mean the real read/write metric)? Presumption is that the CP does the append work itself so we update metricsRegionServer.updateAppend. What is the read/write metric [~zghaobac]? Thanks. > Remove bypass method in ObserverContext and implement the 'bypass' logic case > by case > - > > Key: HBASE-18770 > URL: https://issues.apache.org/jira/browse/HBASE-18770 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Assignee: stack >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18770.master.001.patch, > HBASE-18770.master.002.patch, HBASE-18770.master.003.patch, > HBASE-18770.master.004.patch, HBASE-18770.master.005.patch, > HBASE-18770.master.006.patch, HBASE-18770.master.007.patch, > HBASE-18770.master.008.patch, HBASE-18770.master.009.patch, > HBASE-18770.master.010.patch, HBASE-18770.master.011.patch > > > http://search-hadoop.com/m/HBase/YGbbXd0RDCIHSC1 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18950) Remove Optional parameters in AsyncAdmin interface
[ https://issues.apache.org/jira/browse/HBASE-18950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-18950: --- Attachment: HBASE-18950.master.010.patch > Remove Optional parameters in AsyncAdmin interface > -- > > Key: HBASE-18950 > URL: https://issues.apache.org/jira/browse/HBASE-18950 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Duo Zhang >Assignee: Guanghao Zhang >Priority: Blocker > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-18950.master.001.patch, > HBASE-18950.master.002.patch, HBASE-18950.master.003.patch, > HBASE-18950.master.004.patch, HBASE-18950.master.005.patch, > HBASE-18950.master.005.patch, HBASE-18950.master.006.patch, > HBASE-18950.master.007.patch, HBASE-18950.master.008.patch, > HBASE-18950.master.008.patch, HBASE-18950.master.009.patch, > HBASE-18950.master.010.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18870) Hbase Backup should set the details to MR job name
[ https://issues.apache.org/jira/browse/HBASE-18870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-18870: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: (was: 2.0.0) 2.0.0-alpha-4 Status: Resolved (was: Patch Available) Thanks for the patch, Vishal. > Hbase Backup should set the details to MR job name > -- > > Key: HBASE-18870 > URL: https://issues.apache.org/jira/browse/HBASE-18870 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: Vishal Khandelwal >Assignee: Vishal Khandelwal >Priority: Minor > Labels: backup > Fix For: 2.0.0-alpha-4 > > Attachments: 18870.v3.txt, HBASE-18870.branch-2.v1.patch, > HBASE-18870.branch-2.v2.patch > > > For Incremental and Full Backups job names should be set correctly. It would > make the debug and monitoring better. > Current : > incremental -> WALPlayer_1506332227619 > ExportSnapshot-snapshot_1506323760362_default_t1 > Proposed : > incremental -> Backup_Incremental__ > Full -> Backups_Full__ -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19087) Log an Optional's value/null instead of Optional[value]/Optional.empty.
[
https://issues.apache.org/jira/browse/HBASE-19087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226161#comment-16226161
]
Guangxu Cheng commented on HBASE-19087:
---
[~appy] Thanks. Thanks all for reviewing.:)
> Log an Optional's value/null instead of Optional[value]/Optional.empty.
> ---
>
> Key: HBASE-19087
> URL: https://issues.apache.org/jira/browse/HBASE-19087
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha-4
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-19087.master.001.patch,
> HBASE-19087.master.001.patch, HBASE-19087.master.001.patch
>
>
> After HBASE-18878, the audit log contains redundant characters "Optional"
> which leading to unreadable
> {code:java}
> 2017-10-19 19:51:08,054 INFO
> [RpcServer.default.FPBQ.Fifo.handler=49,queue=4,port=8081] master.HMaster:
> Client=Optional[username]/Optional[/locahost] disable tablename
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-16417) In-Memory MemStore Policy for Flattening and Compactions
[ https://issues.apache.org/jira/browse/HBASE-16417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-16417: -- Attachment: HBASE-16417.17.patch > In-Memory MemStore Policy for Flattening and Compactions > > > Key: HBASE-16417 > URL: https://issues.apache.org/jira/browse/HBASE-16417 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Eshcar Hillel > Fix For: 3.0.0 > > Attachments: HBASE-16417 - Adaptive Compaction Policy - 20171001.pdf, > HBASE-16417 - parameter tuning - 20171001.pdf, HBASE-16417-V01.patch, > HBASE-16417-benchmarkresults-20161101.pdf, > HBASE-16417-benchmarkresults-20161110.pdf, > HBASE-16417-benchmarkresults-20161123.pdf, > HBASE-16417-benchmarkresults-20161205.pdf, > HBASE-16417-benchmarkresults-20170309.pdf, > HBASE-16417-benchmarkresults-20170317.pdf, HBASE-16417.01.patch, > HBASE-16417.02.patch, HBASE-16417.03.patch, HBASE-16417.04.patch, > HBASE-16417.05.patch, HBASE-16417.06.patch, HBASE-16417.07.patch, > HBASE-16417.07.patch, HBASE-16417.08.patch, HBASE-16417.09.patch, > HBASE-16417.10.patch, HBASE-16417.11.patch, HBASE-16417.12.patch, > HBASE-16417.13.patch, HBASE-16417.14.patch, HBASE-16417.15.patch, > HBASE-16417.16.patch, HBASE-16417.17.patch, HBASE-16417.17.patch, > HBASE-16417.17.patch, HBASE-16417.17.patch > > > This Jira explores the performance of different memstore compaction policies. > It presents the result of write-only workload evaluation as well as read > performance in read-write workloads. > We investigate several settings of hardware (SSD, HDD), key distribution > (Zipf, uniform), with multiple settings of the system, and compare measures > like write throughput, read latency, write volume, total gc time, etc. > The submitted patch sets some system properties at the values yielding > optimal performance. In addition we suggest a new Adaptive memstore > compaction policy that shows good tradeoffs between write throughput and > write volume. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18232) Add variable size chunks to the MSLAB
[
https://issues.apache.org/jira/browse/HBASE-18232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226152#comment-16226152
]
Hadoop QA commented on HBASE-18232:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m
23s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
49s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} hbase-server: The patch generated 0 new + 60
unchanged - 1 fixed = 60 total (was 61) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
36s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
45m 31s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 97m
53s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}164m 53s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-18232 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12894860/HBASE-18232-V10.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 6985eb787ed3 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7
11:05:26 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 14368c432a |
| Default Java | 1.8.0_141 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/9523/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/9523/console |
| Powered by | Apache Yetus 0.5.0 http://yetus.apache.org |
This message was automatically generated.
> Add variable size chunks to the MSLAB
> -
>
> Key: HBASE-18232
> URL: https://issues.apache.org/jira/browse/HBASE-18232
> Project: HBase
>
[jira] [Commented] (HBASE-18770) Remove bypass method in ObserverContext and implement the 'bypass' logic case by case
[ https://issues.apache.org/jira/browse/HBASE-18770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226126#comment-16226126 ] Guanghao Zhang commented on HBASE-18770: bq. preAppend Metrics were being updated whether CP bypass or not If CP bypass, we skip the real operation so it don't need update metrics (I mean the real read/write metric)? Maybe we should split the metric to read read/write metric and metric for append operation, then only update metric for append op if CP bypass. > Remove bypass method in ObserverContext and implement the 'bypass' logic case > by case > - > > Key: HBASE-18770 > URL: https://issues.apache.org/jira/browse/HBASE-18770 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Assignee: stack >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18770.master.001.patch, > HBASE-18770.master.002.patch, HBASE-18770.master.003.patch, > HBASE-18770.master.004.patch, HBASE-18770.master.005.patch, > HBASE-18770.master.006.patch, HBASE-18770.master.007.patch, > HBASE-18770.master.008.patch, HBASE-18770.master.009.patch, > HBASE-18770.master.010.patch, HBASE-18770.master.011.patch > > > http://search-hadoop.com/m/HBase/YGbbXd0RDCIHSC1 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19118) Use SaslUtil to set Sasl.QOP in 'Thrift'
[
https://issues.apache.org/jira/browse/HBASE-19118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226121#comment-16226121
]
Reid Chan commented on HBASE-19118:
---
ping [~tedyu] [~chia7712], would you have time to give a review?
> Use SaslUtil to set Sasl.QOP in 'Thrift'
>
>
> Key: HBASE-19118
> URL: https://issues.apache.org/jira/browse/HBASE-19118
> Project: HBase
> Issue Type: Bug
> Components: Thrift
>Affects Versions: 1.2.6
>Reporter: Reid Chan
>Assignee: Reid Chan
> Attachments: HBASE-19118.master.001.patch,
> HBASE-19118.master.002.patch, HBASE-19118.master.003.patch
>
>
> In [Configure the Thrift
> Gateway|http://hbase.apache.org/book.html#security.client.thrift], it says
> "set the property {{hbase.thrift.security.qop}} to one of the following three
> values: {{privacy}}, {{integrity}}, {{authentication}}", which would lead to
> failure of starting up a thrift server.
> In fact, the value of {{hbase.thrift.security.qop}} should be {{auth}},
> {{auth-int}}, {{auth-conf}}, according to the documentation of {{Sasl.QOP}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19087) Log an Optional's value/null instead of Optional[value]/Optional.empty.
[
https://issues.apache.org/jira/browse/HBASE-19087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-19087:
-
Resolution: Fixed
Fix Version/s: (was: 2.0.0-beta-1)
2.0.0-alpha-4
Status: Resolved (was: Patch Available)
> Log an Optional's value/null instead of Optional[value]/Optional.empty.
> ---
>
> Key: HBASE-19087
> URL: https://issues.apache.org/jira/browse/HBASE-19087
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha-4
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-19087.master.001.patch,
> HBASE-19087.master.001.patch, HBASE-19087.master.001.patch
>
>
> After HBASE-18878, the audit log contains redundant characters "Optional"
> which leading to unreadable
> {code:java}
> 2017-10-19 19:51:08,054 INFO
> [RpcServer.default.FPBQ.Fifo.handler=49,queue=4,port=8081] master.HMaster:
> Client=Optional[username]/Optional[/locahost] disable tablename
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-16653) Backport HBASE-11393 to all branches which support namespace
[ https://issues.apache.org/jira/browse/HBASE-16653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-16653: --- Release Note: During HBASE-11393, we have done two things: 1. unify tableCFs with peerConfig 2. Fix ns not support issue for replication. This issue is to backport it to branch-1 How to rolling update if the replication peer have old table-cfs string config? Due to we modify proto object of ReplicationPeerConfig (add tableCFs field), so when we do rolling update, we have to update original ReplicationPeerConfig data on ZK firstly. 1. Make sure the master have the permission to modify replication peer znode. 2. Disable the replication peer. 3. Rolling update master first. The master will copy the table-cfs config from old table-cfs znode and add it to the new proto object of ReplicationPeerConfig. 4. Rolling update regionservers. 5. Enable the replication peer. If you can't change the replication peer znode permission, you can use the TableCFsUpdater tool to copy the table-cfs config. 1. Disable the replication peer. 2. bin/hbase org.apache.hadoop.hbase.replication.master.TableCFsUpdater update 3. Rolling update master and regionservers. 4. Enable the replication peer. Notes: We have to wait for the rolling update completed if we want change one replication peer's table-cfs config. The old client can change the table-cfs znode directly by append_peer_tableCFs or remove_peer_tableCFs. Because the new server will read table-cfs config from replication peer znode. So it doesn't change the replication behavior. The old client can't change the replication peer znode directly if the peer has table-cfs config. Because the old client will wirte a new PB object without table-cfs field to zk directly. Then the new server can't read table-cfs config, the replication behavior will be wrong. was: During HBASE-11393, we have done two things: 1. unify tableCFs with peerConfig 2. Fix ns not support issue for replication. This issue is to backport it to branch-1 How to rolling update if the replication peer have old table-cfs string config? Due to we modify proto object of ReplicationPeerConfig (add tableCFs field), so when we do rolling update, we have to update original ReplicationPeerConfig data on ZK firstly. 1. Make sure the master have the permission to modify replication peer znode. 2. Disable the replication peer. 3. Rolling update master first. The master will copy the table-cfs config from old table-cfs znode and add it to the new proto object of ReplicationPeerConfig. 4. Rolling update regionservers. 5. Enable the replication peer. If you can't change the replication peer znode permission, you can use the TableCFsUpdater tool to copy the table-cfs config. 1. Disable the replication peer. 2. bin/hbase org.apache.hadoop.hbase.replication.master.TableCFsUpdater update 3. Rolling update master and regionservers. 4. Enable the replication peer. Notes: We have to wait for the rolling update completed if we want change one replication peer's table-cfs config. The old client change the table-cfs znode directly not the replication peer znode. And the new server will read table-cfs config from replication peer znode. So it is ok for old client to change the old table-cfs znode directly but it will not work. > Backport HBASE-11393 to all branches which support namespace > > > Key: HBASE-16653 > URL: https://issues.apache.org/jira/browse/HBASE-16653 > Project: HBase > Issue Type: Bug >Affects Versions: 1.0.4, 1.4.0, 1.3.1, 0.98.22, 1.1.7, 1.2.4 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Fix For: 1.4.0 > > Attachments: HBASE-16653-branch-1-v1.patch, > HBASE-16653-branch-1-v2.patch, HBASE-16653-branch-1-v3.patch, > HBASE-16653-branch-1-v4.patch, HBASE-16653-branch-1-v4.patch, > HBASE-16653-branch-1-v5.patch > > > As HBASE-11386 mentioned, the parse code about replication table-cfs config > will be wrong when table name contains namespace and we can only config the > default namespace's tables in the peer. It is a bug for all branches which > support namespace. HBASE-11393 resolved this by use a pb object but it was > only merged to master branch. Other branches still have this problem. I > thought we should fix this bug in all branches which support namespace. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19099) Evaluate the remaining API compatibility concerns between branch-1.3 and branch-1.4 / branch-1
[ https://issues.apache.org/jira/browse/HBASE-19099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226105#comment-16226105 ] Guanghao Zhang commented on HBASE-19099: bq. make sure the release note on HBASE-16653 explains the issue and provides the upgrade instructions like in the above comment. Update the release note of HBASE-16653. > Evaluate the remaining API compatibility concerns between branch-1.3 and > branch-1.4 / branch-1 > -- > > Key: HBASE-19099 > URL: https://issues.apache.org/jira/browse/HBASE-19099 > Project: HBase > Issue Type: Task >Affects Versions: 1.4.0, 1.5.0 >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Blocker > Fix For: 1.4.0, 1.5.0 > > > For discussion, here are the remaining compatibility concerns you will > encounter upon moving up from 1.3 to 1.4: > The below all relate to source level compatibility. Recompilation of a client > program may be terminated with the message "class C is not abstract and does > not override abstract method M in TYPE". According to our compatibility > guidelines we are allowed to add methods to interfaces, so these are ALLOWED. > Please let me know if you disagree. > package org.apache.hadoop.hbase.client > interface Admin > * Abstract method clearDeadServers ( List ) has been added to > this interface. > * Abstract method cloneSnapshot ( String, TableName, boolean ) has been added > to this interface. > * Abstract method isCleanerChoreEnabled ( ) has been added to this interface. > * Abstract method isMasterInMaintenanceMode ( ) has been added to this > interface. > * Abstract method listDeadServers ( ) has been added to this interface. > * Abstract method restoreSnapshot ( String, boolean, boolean ) has been added > to this interface. > * Abstract method runCleanerChore ( ) has been added to this interface. > * Abstract method setCleanerChoreRunning ( boolean ) has been added to this > interface. > package org.apache.hadoop.hbase.client > interface ResultScanner > * Abstract method renewLease ( ) has been added to this interface. > * Abstract method getScanMetrics ( ) has been added to this interface. > package org.apache.hadoop.hbase.client > interface Table > * Abstract method getReadRpcTimeout ( ) has been added to this interface. > * Abstract method getWriteRpcTimeout ( ) has been added to this interface. > * Abstract method setReadRpcTimeout ( int ) has been added to this interface. > * Abstract method setWriteRpcTimeout ( int ) has been added to this interface. > package org.apache.hadoop.hbase.replication > interface ReplicationPeer > * Abstract method getPeerBandwidth ( ) has been added to this interface. > * Abstract method trackPeerConfigChanges ( ReplicationPeerConfigListener ) > has been added to this interface. > package org.apache.hadoop.hbase.coprocessor > interface MasterCoprocessorEnvironment > * Abstract method getMetricRegistryForMaster ( ) has been added to this > interface. > package org.apache.hadoop.hbase.coprocessor > interface MasterObserver > * Abstract method postAddRSGroup ( > ObserverContext, String ) has been added to > this interface. > * Abstract method postBalanceRSGroup ( > ObserverContext, String, boolean ) has been > added to this interface. > * Abstract method postClearDeadServers ( > ObserverContext ) has been added to this > interface. > * Abstract method postListDeadServers ( > ObserverContext ) has been added to this > interface. > * Abstract method postMoveServers ( > ObserverContext, Set, String ) has > been added to this interface. > * Abstract method postMoveServersAndTables ( > ObserverContext, Set, Set, > String ) has been added to this interface. > * Abstract method postMoveTables ( > ObserverContext, Set, String ) has > been added to this interface. > * Abstract method postRemoveRSGroup ( > ObserverContext, String ) has been added to > this interface. > * Abstract method preAddRSGroup ( > ObserverContext, String ) has been added to > this interface. > * Abstract method preBalanceRSGroup ( > ObserverContext, String ) has been added to > this interface. > * Abstract method preClearDeadServers ( > ObserverContext ) has been added to this > interface. > * Abstract method preListDeadServers ( > ObserverContext ) has been added to this > interface. > * Abstract method preMoveServers ( > ObserverContext, Set, String ) has > been added to this interface. > * Abstract method preMoveServersAndTables ( > ObserverContext, Set, Set, > String ) has been added to this interface. > * Abstract method preMoveTables ( > ObserverContext, Set, String ) has > been added to this interface. > * Abstract method preRemoveRSGroup ( > ObserverContext, String ) has been added to > this
[jira] [Resolved] (HBASE-18952) Remove the Optional(Int) usage in the constructors of HStore
[ https://issues.apache.org/jira/browse/HBASE-18952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-18952. --- Resolution: Implemented Already implemented by HBASE-19033. > Remove the Optional(Int) usage in the constructors of HStore > > > Key: HBASE-18952 > URL: https://issues.apache.org/jira/browse/HBASE-18952 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: Duo Zhang > Fix For: 2.0.0 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19087) Logs an Optional's value/null instead of Optional[value]/Optional.empty.
[
https://issues.apache.org/jira/browse/HBASE-19087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-19087:
-
Summary: Logs an Optional's value/null instead of
Optional[value]/Optional.empty. (was: Logs an Optional's value/null instead of
"Optional[value]/Optional.empty.)
> Logs an Optional's value/null instead of Optional[value]/Optional.empty.
>
>
> Key: HBASE-19087
> URL: https://issues.apache.org/jira/browse/HBASE-19087
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha-4
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19087.master.001.patch,
> HBASE-19087.master.001.patch, HBASE-19087.master.001.patch
>
>
> After HBASE-18878, the audit log contains redundant characters "Optional"
> which leading to unreadable
> {code:java}
> 2017-10-19 19:51:08,054 INFO
> [RpcServer.default.FPBQ.Fifo.handler=49,queue=4,port=8081] master.HMaster:
> Client=Optional[username]/Optional[/locahost] disable tablename
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19087) Log an Optional's value/null instead of Optional[value]/Optional.empty.
[
https://issues.apache.org/jira/browse/HBASE-19087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-19087:
-
Summary: Log an Optional's value/null instead of
Optional[value]/Optional.empty. (was: Logs an Optional's value/null instead of
Optional[value]/Optional.empty.)
> Log an Optional's value/null instead of Optional[value]/Optional.empty.
> ---
>
> Key: HBASE-19087
> URL: https://issues.apache.org/jira/browse/HBASE-19087
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha-4
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19087.master.001.patch,
> HBASE-19087.master.001.patch, HBASE-19087.master.001.patch
>
>
> After HBASE-18878, the audit log contains redundant characters "Optional"
> which leading to unreadable
> {code:java}
> 2017-10-19 19:51:08,054 INFO
> [RpcServer.default.FPBQ.Fifo.handler=49,queue=4,port=8081] master.HMaster:
> Client=Optional[username]/Optional[/locahost] disable tablename
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19087) Logs an Optional's value/null instead of "Optional[value]/Optional.empty.
[
https://issues.apache.org/jira/browse/HBASE-19087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-19087:
-
Summary: Logs an Optional's value/null instead of
"Optional[value]/Optional.empty. (was: Remove redundant characters from the
audit log)
> Logs an Optional's value/null instead of "Optional[value]/Optional.empty.
> -
>
> Key: HBASE-19087
> URL: https://issues.apache.org/jira/browse/HBASE-19087
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha-4
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19087.master.001.patch,
> HBASE-19087.master.001.patch, HBASE-19087.master.001.patch
>
>
> After HBASE-18878, the audit log contains redundant characters "Optional"
> which leading to unreadable
> {code:java}
> 2017-10-19 19:51:08,054 INFO
> [RpcServer.default.FPBQ.Fifo.handler=49,queue=4,port=8081] master.HMaster:
> Client=Optional[username]/Optional[/locahost] disable tablename
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-16653) Backport HBASE-11393 to all branches which support namespace
[ https://issues.apache.org/jira/browse/HBASE-16653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-16653: --- Release Note: During HBASE-11393, we have done two things: 1. unify tableCFs with peerConfig 2. Fix ns not support issue for replication. This issue is to backport it to branch-1 How to rolling update if the replication peer have old table-cfs string config? Due to we modify proto object of ReplicationPeerConfig (add tableCFs field), so when we do rolling update, we have to update original ReplicationPeerConfig data on ZK firstly. 1. Make sure the master have the permission to modify replication peer znode. 2. Disable the replication peer. 3. Rolling update master first. The master will copy the table-cfs config from old table-cfs znode and add it to the new proto object of ReplicationPeerConfig. 4. Rolling update regionservers. 5. Enable the replication peer. If you can't change the replication peer znode permission, you can use the TableCFsUpdater tool to copy the table-cfs config. 1. Disable the replication peer. 2. bin/hbase org.apache.hadoop.hbase.replication.master.TableCFsUpdater update 3. Rolling update master and regionservers. 4. Enable the replication peer. Notes: We have to wait for the rolling update completed if we want change one replication peer's table-cfs config. The old client change the table-cfs znode directly not the replication peer znode. And the new server will read table-cfs config from replication peer znode. So it is ok for old client to change the old table-cfs znode directly but it will not work. was: During HBASE-11393, we have done two things: 1. unify tableCFs with peerConfig 2. Fix ns not support issue for replication. This issue is to backport it to branch-1 How to rolling update if the replication peer have old table-cfs string config? Due to we modify proto object of ReplicationPeerConfig (add tableCFs field), so when we do rolling update, we have to update original ReplicationPeerConfig data on ZK firstly. 1. Make sure the master have the permission to modify replication peer znode. 2. Disable the replication peer. 3. Rolling update master first. The master will update the table-cfs config from string to PB. 4. Rolling update regionservers. 5. Enable the replication peer. Notes: Due to we modify proto object of ReplicationPeerConfig (add tableCFs field), so when we do rolling update, we have to update original ReplicationPeerConfig data on ZK firstly. This means during rolling update, if one peer with namespace added, this peer replication on old regionserver could not work. We have to wait for the rolling update completed. > Backport HBASE-11393 to all branches which support namespace > > > Key: HBASE-16653 > URL: https://issues.apache.org/jira/browse/HBASE-16653 > Project: HBase > Issue Type: Bug >Affects Versions: 1.0.4, 1.4.0, 1.3.1, 0.98.22, 1.1.7, 1.2.4 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Fix For: 1.4.0 > > Attachments: HBASE-16653-branch-1-v1.patch, > HBASE-16653-branch-1-v2.patch, HBASE-16653-branch-1-v3.patch, > HBASE-16653-branch-1-v4.patch, HBASE-16653-branch-1-v4.patch, > HBASE-16653-branch-1-v5.patch > > > As HBASE-11386 mentioned, the parse code about replication table-cfs config > will be wrong when table name contains namespace and we can only config the > default namespace's tables in the peer. It is a bug for all branches which > support namespace. HBASE-11393 resolved this by use a pb object but it was > only merged to master branch. Other branches still have this problem. I > thought we should fix this bug in all branches which support namespace. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19087) Remove redundant characters from the audit log
[
https://issues.apache.org/jira/browse/HBASE-19087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226102#comment-16226102
]
Appy commented on HBASE-19087:
--
lgtm.
Thanks for the patch [~andrewcheng]. Committed to master and branch-2 (since
HBASE-18878 only made that far back).
Reworded the commit message to explicitly mention which redundant characters
were being removed.
> Remove redundant characters from the audit log
> --
>
> Key: HBASE-19087
> URL: https://issues.apache.org/jira/browse/HBASE-19087
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha-4
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19087.master.001.patch,
> HBASE-19087.master.001.patch, HBASE-19087.master.001.patch
>
>
> After HBASE-18878, the audit log contains redundant characters "Optional"
> which leading to unreadable
> {code:java}
> 2017-10-19 19:51:08,054 INFO
> [RpcServer.default.FPBQ.Fifo.handler=49,queue=4,port=8081] master.HMaster:
> Client=Optional[username]/Optional[/locahost] disable tablename
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18972) Use Builder pattern to remove nullable parameters for coprocessor methods in RawAsyncTable interface
[ https://issues.apache.org/jira/browse/HBASE-18972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226101#comment-16226101 ] Duo Zhang commented on HBASE-18972: --- For specify the inclusive/exclusive of startKey/endKey, I think both approach are OK. You can say that it is only a single parameter(the start/stop point of a scan) for a scan but only need two variables to represent, or you can treat them as two separated parameters which have relations. Both OK. But I have already used the (row, inclusive) pattern in Scan, you can see the withStartRow and withStopRow in Scan. And they are also introduced in branch-1.x so I think we'd better keep it in that way. For other suggestions, seems good. I will upload a new patch to address them. Thanks. > Use Builder pattern to remove nullable parameters for coprocessor methods in > RawAsyncTable interface > > > Key: HBASE-18972 > URL: https://issues.apache.org/jira/browse/HBASE-18972 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Blocker > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-18972-v1.patch, HBASE-18972.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-16653) Backport HBASE-11393 to all branches which support namespace
[ https://issues.apache.org/jira/browse/HBASE-16653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-16653: --- Release Note: During HBASE-11393, we have done two things: 1. unify tableCFs with peerConfig 2. Fix ns not support issue for replication. This issue is to backport it to branch-1 How to rolling update if the replication peer have old table-cfs string config? Due to we modify proto object of ReplicationPeerConfig (add tableCFs field), so when we do rolling update, we have to update original ReplicationPeerConfig data on ZK firstly. 1. Make sure the master have the permission to modify replication peer znode. 2. Disable the replication peer. 3. Rolling update master first. The master will update the table-cfs config from string to PB. 4. Rolling update regionservers. 5. Enable the replication peer. Notes: Due to we modify proto object of ReplicationPeerConfig (add tableCFs field), so when we do rolling update, we have to update original ReplicationPeerConfig data on ZK firstly. This means during rolling update, if one peer with namespace added, this peer replication on old regionserver could not work. We have to wait for the rolling update completed. was: During HBASE-11393, we have done two things: 1. unify tableCFs with peerConfig 2. Fix ns not support issue for replication. This issue is to backport it to branch-1 Notes: Due to we modify proto object of ReplicationPeerConfig (add tableCFs field), so when we do rolling update, we have to update original ReplicationPeerConfig data on ZK firstly. This means during rolling update, if one peer with namespace added, this peer replication on old regionserver could not work. We have to wait for the rolling update completed. > Backport HBASE-11393 to all branches which support namespace > > > Key: HBASE-16653 > URL: https://issues.apache.org/jira/browse/HBASE-16653 > Project: HBase > Issue Type: Bug >Affects Versions: 1.0.4, 1.4.0, 1.3.1, 0.98.22, 1.1.7, 1.2.4 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Fix For: 1.4.0 > > Attachments: HBASE-16653-branch-1-v1.patch, > HBASE-16653-branch-1-v2.patch, HBASE-16653-branch-1-v3.patch, > HBASE-16653-branch-1-v4.patch, HBASE-16653-branch-1-v4.patch, > HBASE-16653-branch-1-v5.patch > > > As HBASE-11386 mentioned, the parse code about replication table-cfs config > will be wrong when table name contains namespace and we can only config the > default namespace's tables in the peer. It is a bug for all branches which > support namespace. HBASE-11393 resolved this by use a pb object but it was > only merged to master branch. Other branches still have this problem. I > thought we should fix this bug in all branches which support namespace. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18995) Move methods that are for internal usage from CellUtil to Private util class
[
https://issues.apache.org/jira/browse/HBASE-18995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226094#comment-16226094
]
stack commented on HBASE-18995:
---
The use case [~mdrob] ran into was CellUtil.createFirstOnRow. Its not in
CellUtil anymore [~ram_krish]
Here are other uses of CellUtil by Crunch:
{code}
./crunch-hbase/src/it/java/org/apache/crunch/io/hbase/HFileTargetIT.java:
Cell cell = CellUtil.createCell(Bytes.toBytes(w), Bytes.toBytes(c));
./crunch-hbase/src/main/java/org/apache/crunch/io/hbase/HFileOutputFormatForCrunch.java:
InetSocketAddress preferredNodeForRow =
regionLocationTable.getPreferredNodeForRow(CellUtil.cloneRow(cell));
./crunch-hbase/src/main/java/org/apache/crunch/io/hbase/HFileUtils.java:
return ByteBuffer.wrap(CellUtil.cloneRow(input));
./crunch-hbase/src/main/java/org/apache/crunch/io/hbase/HFileUtils.java:
LOG.debug("split row: " + Bytes.toString(CellUtil.cloneRow(kv)));
./crunch-hbase/src/main/java/org/apache/crunch/io/hbase/HFileUtils.java:
emitter.emit(ByteBuffer.wrap(CellUtil.cloneRow(foundCell)));
./crunch-spark/src/it/java/org/apache/crunch/SparkHFileTargetIT.java:
Cell cell = CellUtil.createCell(Bytes.toBytes(w), Bytes.toBytes(c));
{code}
> Move methods that are for internal usage from CellUtil to Private util class
>
>
> Key: HBASE-18995
> URL: https://issues.apache.org/jira/browse/HBASE-18995
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.0.0-alpha-3
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-18995-branch-2.002.patch,
> HBASE-18995-branch-2.patch, HBASE-18995-branch-2_1.patch,
> HBASE-18995-branch-2_1.patch, HBASE-18995-branch-2_1.patch,
> HBASE-18995-branch-2_1.patch, HBASE-18995-branch-2_2.patch,
> HBASE-18995_002-branch-2.patch, HBASE-18995_002-branch-2.patch,
> HBASE-18995_003-branch-2.patch, HBASE-18995_1.patch, HBASE-18995_2.patch,
> HBASE-18995_2.patch
>
>
> This was brought up long time back. We need to move some of the public APIs
> from CellUtil to internal private Util class because they are used in some
> internal flow and does not make sense to have it in a @public exposed Util
> class.
> The topic again came in HBASE-18945 RB comments also.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19036) Add action in Chaos Monkey to restart Active Namenode
[
https://issues.apache.org/jira/browse/HBASE-19036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226093#comment-16226093
]
Hadoop QA commented on HBASE-19036:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 7 new or modified test
files. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
12s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
5s{color} | {color:green} branch-1 passed with JDK v1.8.0_141 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
30s{color} | {color:green} branch-1 passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
34s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
24s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
11s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
16s{color} | {color:green} branch-1 passed with JDK v1.8.0_141 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} branch-1 passed with JDK v1.7.0_151 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
45s{color} | {color:green} the patch passed with JDK v1.8.0_141 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
23s{color} | {color:green} the patch passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
50s{color} | {color:red} hbase-server: The patch generated 5 new + 27 unchanged
- 0 fixed = 32 total (was 27) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 546 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
17s{color} | {color:red} The patch 384 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
5s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
38m 11s{color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3.
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed with JDK v1.8.0_141 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} the patch passed with JDK v1.7.0_151 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}765m 50s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
[jira] [Commented] (HBASE-18770) Remove bypass method in ObserverContext and implement the 'bypass' logic case by case
[ https://issues.apache.org/jira/browse/HBASE-18770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226091#comment-16226091 ] stack commented on HBASE-18770: --- And thank you for the @deprecated [~chia7712] Let me do that too > Remove bypass method in ObserverContext and implement the 'bypass' logic case > by case > - > > Key: HBASE-18770 > URL: https://issues.apache.org/jira/browse/HBASE-18770 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Assignee: stack >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18770.master.001.patch, > HBASE-18770.master.002.patch, HBASE-18770.master.003.patch, > HBASE-18770.master.004.patch, HBASE-18770.master.005.patch, > HBASE-18770.master.006.patch, HBASE-18770.master.007.patch, > HBASE-18770.master.008.patch, HBASE-18770.master.009.patch, > HBASE-18770.master.010.patch, HBASE-18770.master.011.patch > > > http://search-hadoop.com/m/HBase/YGbbXd0RDCIHSC1 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19099) Evaluate the remaining API compatibility concerns between branch-1.3 and branch-1.4 / branch-1
[ https://issues.apache.org/jira/browse/HBASE-19099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226087#comment-16226087 ] Guanghao Zhang commented on HBASE-19099: [~apurtell] Thanks for you explanation. Let's resolve this in HBASE-18626. > Evaluate the remaining API compatibility concerns between branch-1.3 and > branch-1.4 / branch-1 > -- > > Key: HBASE-19099 > URL: https://issues.apache.org/jira/browse/HBASE-19099 > Project: HBase > Issue Type: Task >Affects Versions: 1.4.0, 1.5.0 >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Blocker > Fix For: 1.4.0, 1.5.0 > > > For discussion, here are the remaining compatibility concerns you will > encounter upon moving up from 1.3 to 1.4: > The below all relate to source level compatibility. Recompilation of a client > program may be terminated with the message "class C is not abstract and does > not override abstract method M in TYPE". According to our compatibility > guidelines we are allowed to add methods to interfaces, so these are ALLOWED. > Please let me know if you disagree. > package org.apache.hadoop.hbase.client > interface Admin > * Abstract method clearDeadServers ( List ) has been added to > this interface. > * Abstract method cloneSnapshot ( String, TableName, boolean ) has been added > to this interface. > * Abstract method isCleanerChoreEnabled ( ) has been added to this interface. > * Abstract method isMasterInMaintenanceMode ( ) has been added to this > interface. > * Abstract method listDeadServers ( ) has been added to this interface. > * Abstract method restoreSnapshot ( String, boolean, boolean ) has been added > to this interface. > * Abstract method runCleanerChore ( ) has been added to this interface. > * Abstract method setCleanerChoreRunning ( boolean ) has been added to this > interface. > package org.apache.hadoop.hbase.client > interface ResultScanner > * Abstract method renewLease ( ) has been added to this interface. > * Abstract method getScanMetrics ( ) has been added to this interface. > package org.apache.hadoop.hbase.client > interface Table > * Abstract method getReadRpcTimeout ( ) has been added to this interface. > * Abstract method getWriteRpcTimeout ( ) has been added to this interface. > * Abstract method setReadRpcTimeout ( int ) has been added to this interface. > * Abstract method setWriteRpcTimeout ( int ) has been added to this interface. > package org.apache.hadoop.hbase.replication > interface ReplicationPeer > * Abstract method getPeerBandwidth ( ) has been added to this interface. > * Abstract method trackPeerConfigChanges ( ReplicationPeerConfigListener ) > has been added to this interface. > package org.apache.hadoop.hbase.coprocessor > interface MasterCoprocessorEnvironment > * Abstract method getMetricRegistryForMaster ( ) has been added to this > interface. > package org.apache.hadoop.hbase.coprocessor > interface MasterObserver > * Abstract method postAddRSGroup ( > ObserverContext, String ) has been added to > this interface. > * Abstract method postBalanceRSGroup ( > ObserverContext, String, boolean ) has been > added to this interface. > * Abstract method postClearDeadServers ( > ObserverContext ) has been added to this > interface. > * Abstract method postListDeadServers ( > ObserverContext ) has been added to this > interface. > * Abstract method postMoveServers ( > ObserverContext, Set, String ) has > been added to this interface. > * Abstract method postMoveServersAndTables ( > ObserverContext, Set, Set, > String ) has been added to this interface. > * Abstract method postMoveTables ( > ObserverContext, Set, String ) has > been added to this interface. > * Abstract method postRemoveRSGroup ( > ObserverContext, String ) has been added to > this interface. > * Abstract method preAddRSGroup ( > ObserverContext, String ) has been added to > this interface. > * Abstract method preBalanceRSGroup ( > ObserverContext, String ) has been added to > this interface. > * Abstract method preClearDeadServers ( > ObserverContext ) has been added to this > interface. > * Abstract method preListDeadServers ( > ObserverContext ) has been added to this > interface. > * Abstract method preMoveServers ( > ObserverContext, Set, String ) has > been added to this interface. > * Abstract method preMoveServersAndTables ( > ObserverContext, Set, Set, > String ) has been added to this interface. > * Abstract method preMoveTables ( > ObserverContext, Set, String ) has > been added to this interface. > * Abstract method preRemoveRSGroup ( > ObserverContext, String ) has been added to > this interface. > package org.apache.hadoop.hbase.coprocessor > interface
[jira] [Commented] (HBASE-18770) Remove bypass method in ObserverContext and implement the 'bypass' logic case by case
[ https://issues.apache.org/jira/browse/HBASE-18770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226086#comment-16226086 ] stack commented on HBASE-18770: --- I like this idea. Let me do it. > Remove bypass method in ObserverContext and implement the 'bypass' logic case > by case > - > > Key: HBASE-18770 > URL: https://issues.apache.org/jira/browse/HBASE-18770 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Assignee: stack >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18770.master.001.patch, > HBASE-18770.master.002.patch, HBASE-18770.master.003.patch, > HBASE-18770.master.004.patch, HBASE-18770.master.005.patch, > HBASE-18770.master.006.patch, HBASE-18770.master.007.patch, > HBASE-18770.master.008.patch, HBASE-18770.master.009.patch, > HBASE-18770.master.010.patch, HBASE-18770.master.011.patch > > > http://search-hadoop.com/m/HBase/YGbbXd0RDCIHSC1 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18770) Remove bypass method in ObserverContext and implement the 'bypass' logic case by case
[
https://issues.apache.org/jira/browse/HBASE-18770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226079#comment-16226079
]
Chia-Ping Tsai commented on HBASE-18770:
{quote}
@Deprecated public boolean preWALRestore(final RegionInfo info, final WALKey
logKey, final WALEdit logEdit) throws IOException {
@Deprecated public void postWALRestore(final RegionInfo info, final WALKey
logKey, final WALEdit logEdit) throws IOException {
{quote}
What about introducing a immutable {{WALKey}}? CP users should not modify the
{{WALKey}} but it is OK to retrieve information from {{WALKey}} .
> Remove bypass method in ObserverContext and implement the 'bypass' logic case
> by case
> -
>
> Key: HBASE-18770
> URL: https://issues.apache.org/jira/browse/HBASE-18770
> Project: HBase
> Issue Type: Sub-task
> Components: Coprocessors
>Reporter: Duo Zhang
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-18770.master.001.patch,
> HBASE-18770.master.002.patch, HBASE-18770.master.003.patch,
> HBASE-18770.master.004.patch, HBASE-18770.master.005.patch,
> HBASE-18770.master.006.patch, HBASE-18770.master.007.patch,
> HBASE-18770.master.008.patch, HBASE-18770.master.009.patch,
> HBASE-18770.master.010.patch, HBASE-18770.master.011.patch
>
>
> http://search-hadoop.com/m/HBase/YGbbXd0RDCIHSC1
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18972) Use Builder pattern to remove nullable parameters for coprocessor methods in RawAsyncTable interface
[
https://issues.apache.org/jira/browse/HBASE-18972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226078#comment-16226078
]
Appy commented on HBASE-18972:
--
So we have 2 nullable/non-mandatory params here - startKey, endKey.
How about default values for startKeyInclusive and endKeyInclusive? Maybe true
and false respectively to match our default way of writing keyranges?
So we have 7 params, 3 mandatory (stubMaker, callable and callback) and 4
optional.
Nice! Builder pattern is perfect here! And with that execute() fn, it has
perfect 'fluent' finish.
Some change suggestions:
- Reading from(..) and to(..) in AsyncAggregationClient, i had to go back and
see what they were. Let's rename them to {from/to}RowKey to make it explicit?
- Also, since we are already using builder patterns, there's no point merging
setters for different params. Let's add {start/end}Key{Inclusive/Exclusive}()
functions.
- It was weird seeing those checkNotNull preconditions in execute(). Since
those three (stubMaker, callable and callback) are mandatory params, they
should go in builder's constructor. (That's the common pattern - mandatory
params through constructor and optional params by setters; exception being,
when there are many mandatory params but that's not the case here).
-
{quote}
\* TODO: For now the only difference between this interface and \{@link
AsyncTable} is the scan
\* method. The \{@link RawScanResultConsumer} exposes the implementation
details of a scan(heartbeat)
\* so it is not suitable for a normal user. If it is still the only difference
after we implement
\* most features of AsyncTable, we can think about merge these two interfaces.
\* @since 2.0.0
{quote}
ref:
https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncTable.java#L42
So i guess this won't be happening anymore?
> Use Builder pattern to remove nullable parameters for coprocessor methods in
> RawAsyncTable interface
>
>
> Key: HBASE-18972
> URL: https://issues.apache.org/jira/browse/HBASE-18972
> Project: HBase
> Issue Type: Sub-task
> Components: Client
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Blocker
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-18972-v1.patch, HBASE-18972.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (HBASE-19129) TestChoreService is flaky (branch-1 / branch-1.4)
[ https://issues.apache.org/jira/browse/HBASE-19129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-19129. Resolution: Fixed Pushed trivial patch to branch-1.4 and up > TestChoreService is flaky (branch-1 / branch-1.4) > - > > Key: HBASE-19129 > URL: https://issues.apache.org/jira/browse/HBASE-19129 > Project: HBase > Issue Type: Bug >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Trivial > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0 > > Attachments: HBASE-19129.patch > > > Rare flake. > [ERROR] Tests run: 17, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: > 21.788 s <<< FAILURE! - in org.apache.hadoop.hbase.TestChoreService > [ERROR] testForceTrigger(org.apache.hadoop.hbase.TestChoreService) Time > elapsed: 1.044 s <<< FAILURE! > java.lang.AssertionError > at > org.apache.hadoop.hbase.TestChoreService.testForceTrigger(TestChoreService.java:381) > Increase the delta to compensate for environmental variance. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18770) Remove bypass method in ObserverContext and implement the 'bypass' logic case by case
[
https://issues.apache.org/jira/browse/HBASE-18770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226052#comment-16226052
]
Chia-Ping Tsai commented on HBASE-18770:
{code}
@Deprecated public boolean prePrepareTimeStampForDeleteVersion(final Mutation
mutation, final Cell kv, final byte[] byteNow, final Get get) throws
IOException {
{code}
The {{RegionObserver#prePrepareTimeStampForDeleteVersion}} isn't annotated
Deprecated. Please fix it also.
> Remove bypass method in ObserverContext and implement the 'bypass' logic case
> by case
> -
>
> Key: HBASE-18770
> URL: https://issues.apache.org/jira/browse/HBASE-18770
> Project: HBase
> Issue Type: Sub-task
> Components: Coprocessors
>Reporter: Duo Zhang
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-18770.master.001.patch,
> HBASE-18770.master.002.patch, HBASE-18770.master.003.patch,
> HBASE-18770.master.004.patch, HBASE-18770.master.005.patch,
> HBASE-18770.master.006.patch, HBASE-18770.master.007.patch,
> HBASE-18770.master.008.patch, HBASE-18770.master.009.patch,
> HBASE-18770.master.010.patch, HBASE-18770.master.011.patch
>
>
> http://search-hadoop.com/m/HBase/YGbbXd0RDCIHSC1
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18995) Move methods that are for internal usage from CellUtil to Private util class
[ https://issues.apache.org/jira/browse/HBASE-18995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226036#comment-16226036 ] stack commented on HBASE-18995: --- [~ram_krish] I see three methods in CellUtil with Private annotation. Any reason they were not moved? I see they are @Deprecated. Is that because they have been moved already? Are there methods from PrivateCellUtil you want to expose to Coprocessors? > Move methods that are for internal usage from CellUtil to Private util class > > > Key: HBASE-18995 > URL: https://issues.apache.org/jira/browse/HBASE-18995 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-alpha-3 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18995-branch-2.002.patch, > HBASE-18995-branch-2.patch, HBASE-18995-branch-2_1.patch, > HBASE-18995-branch-2_1.patch, HBASE-18995-branch-2_1.patch, > HBASE-18995-branch-2_1.patch, HBASE-18995-branch-2_2.patch, > HBASE-18995_002-branch-2.patch, HBASE-18995_002-branch-2.patch, > HBASE-18995_003-branch-2.patch, HBASE-18995_1.patch, HBASE-18995_2.patch, > HBASE-18995_2.patch > > > This was brought up long time back. We need to move some of the public APIs > from CellUtil to internal private Util class because they are used in some > internal flow and does not make sense to have it in a @public exposed Util > class. > The topic again came in HBASE-18945 RB comments also. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19114) Split out o.a.h.h.zookeeper from hbase-server and hbase-client
[ https://issues.apache.org/jira/browse/HBASE-19114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19114: - Attachment: HBASE-19114.master.003.patch > Split out o.a.h.h.zookeeper from hbase-server and hbase-client > -- > > Key: HBASE-19114 > URL: https://issues.apache.org/jira/browse/HBASE-19114 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Appy > Attachments: HBASE-19114.master.001.patch, > HBASE-19114.master.002.patch, HBASE-19114.master.003.patch > > > Changes so far: > - Moved DrainingServerTracker and RegionServerTracker to > hbase-server:o.a.h.h.master. > - Move Abortable to hbase-common. Since it's IA.Private and independent of > anything, moving it to hbase-common which is at bottom of the dependency tree > is better. > - Moved RecoveringRegionWatcher to hbase-server:o.a.h.h.regionserver > - Moved SplitOrMergeTracker to oahh.master (because it depends on a PB) > - Moving hbase-client:oahh.zookeeper.* to hbase-zookeeper module. We want to > keep hbase-zookeeper very independent and hence at lowest levels in our > dependency tree. > - ZKUtil is a huge tangle since it's linked to almost everything in > \[hbase-client/]oahh.zookeeper. And pulling it down requires some basic proto > functions (mergeFrom, PBmagic, etc). So what i did was: >** Pulled down common and basic protobuf functions (which only depend on > com.google.protobuf.\*) to hbase-common so other code depending on them can > be pulled down if possible/wanted in future. This will help future dependency > untangling too. These are ProtobufMagic and ProtobufHelpers. > ** Didn't move any hbase-specific PB stuff to hbase-common. We can't pull > things into hbase-common which add dependency between it and > hbase-protobuf/hbase-shaded-protobuf since we very recently untangled them. > - DEFAULT_REPLICA_ID is used in many places in ZK. Declared a new constant in > HConstants (since it's in hbase-common) and using it in hbase-zookeeper. > RegionInfo.DEFAULT_REPLICA_ID also takes its value from it to avoid case > where two values can become different. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18995) Move methods that are for internal usage from CellUtil to Private util class
[ https://issues.apache.org/jira/browse/HBASE-18995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226026#comment-16226026 ] stack commented on HBASE-18995: --- [~mdrob] Annotating on the method-level is unusual in the codebase. Usual is to annotate on the class. Some methods remain annotated Private in CellUtil even after this change but I think the sense is that this is a failing to be fixed. We are having trouble with CellUtil and TagUtil and a profusion of classes colored by access. Annotation by method might help. Background. Much of the alpha-4 project (and before this) is about shutting down access to internals either removing private classes as parameters to Coprocessors or sorting classes by private and public portions (usually returning an Interface that has the public methods but internally we'll use the implementation that allows access to all methods or populating Coprocessor Environments/Context with methods that have been properly santized). This CellUtil (and TagUtil) are a little difficult. Their static methods generally so options are less. A long running project has been about undoing dependence on our internal Cell representation so we can change it one day. There are Factories and Builders if you want to make Cells. We want to steer folks this direction. CellUtil is peppered with methods that are for nought but internal devs and a particular context. Hiding them from non-devs saves them trouble should the format change. > Move methods that are for internal usage from CellUtil to Private util class > > > Key: HBASE-18995 > URL: https://issues.apache.org/jira/browse/HBASE-18995 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-alpha-3 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18995-branch-2.002.patch, > HBASE-18995-branch-2.patch, HBASE-18995-branch-2_1.patch, > HBASE-18995-branch-2_1.patch, HBASE-18995-branch-2_1.patch, > HBASE-18995-branch-2_1.patch, HBASE-18995-branch-2_2.patch, > HBASE-18995_002-branch-2.patch, HBASE-18995_002-branch-2.patch, > HBASE-18995_003-branch-2.patch, HBASE-18995_1.patch, HBASE-18995_2.patch, > HBASE-18995_2.patch > > > This was brought up long time back. We need to move some of the public APIs > from CellUtil to internal private Util class because they are used in some > internal flow and does not make sense to have it in a @public exposed Util > class. > The topic again came in HBASE-18945 RB comments also. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19129) TestChoreService is flaky (branch-1 / branch-1.4)
[ https://issues.apache.org/jira/browse/HBASE-19129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-19129: --- Priority: Trivial (was: Minor) > TestChoreService is flaky (branch-1 / branch-1.4) > - > > Key: HBASE-19129 > URL: https://issues.apache.org/jira/browse/HBASE-19129 > Project: HBase > Issue Type: Bug >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Trivial > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0 > > Attachments: HBASE-19129.patch > > > Rare flake. > [ERROR] Tests run: 17, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: > 21.788 s <<< FAILURE! - in org.apache.hadoop.hbase.TestChoreService > [ERROR] testForceTrigger(org.apache.hadoop.hbase.TestChoreService) Time > elapsed: 1.044 s <<< FAILURE! > java.lang.AssertionError > at > org.apache.hadoop.hbase.TestChoreService.testForceTrigger(TestChoreService.java:381) > Increase the delta to compensate for environmental variance. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19129) TestChoreService is flaky (branch-1 / branch-1.4)
[ https://issues.apache.org/jira/browse/HBASE-19129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-19129: --- Attachment: HBASE-19129.patch > TestChoreService is flaky (branch-1 / branch-1.4) > - > > Key: HBASE-19129 > URL: https://issues.apache.org/jira/browse/HBASE-19129 > Project: HBase > Issue Type: Bug >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0 > > Attachments: HBASE-19129.patch > > > Rare flake. > [ERROR] Tests run: 17, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: > 21.788 s <<< FAILURE! - in org.apache.hadoop.hbase.TestChoreService > [ERROR] testForceTrigger(org.apache.hadoop.hbase.TestChoreService) Time > elapsed: 1.044 s <<< FAILURE! > java.lang.AssertionError > at > org.apache.hadoop.hbase.TestChoreService.testForceTrigger(TestChoreService.java:381) > Increase the delta to compensate for environmental variance. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19129) TestChoreService is flaky (branch-1 / branch-1.4)
Andrew Purtell created HBASE-19129: -- Summary: TestChoreService is flaky (branch-1 / branch-1.4) Key: HBASE-19129 URL: https://issues.apache.org/jira/browse/HBASE-19129 Project: HBase Issue Type: Bug Reporter: Andrew Purtell Assignee: Andrew Purtell Priority: Minor Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0 Rare flake. [ERROR] Tests run: 17, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 21.788 s <<< FAILURE! - in org.apache.hadoop.hbase.TestChoreService [ERROR] testForceTrigger(org.apache.hadoop.hbase.TestChoreService) Time elapsed: 1.044 s <<< FAILURE! java.lang.AssertionError at org.apache.hadoop.hbase.TestChoreService.testForceTrigger(TestChoreService.java:381) Increase the delta to compensate for environmental variance. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18770) Remove bypass method in ObserverContext and implement the 'bypass' logic case by case
[ https://issues.apache.org/jira/browse/HBASE-18770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225993#comment-16225993 ] stack commented on HBASE-18770: --- I looked at the original dev mailing list again. The two cases [~ghelmling] raises, bypass on preDelete (Tephra) and bypass of preGet and preIncrementAfterRRowLock are still in place. Phoenix calls bypass on preIncrement. That is still in place. Timeline Server calls bypass on preGet too. I think that is it for the TODOs. .011 address all the nice RB review. > Remove bypass method in ObserverContext and implement the 'bypass' logic case > by case > - > > Key: HBASE-18770 > URL: https://issues.apache.org/jira/browse/HBASE-18770 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Assignee: stack >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18770.master.001.patch, > HBASE-18770.master.002.patch, HBASE-18770.master.003.patch, > HBASE-18770.master.004.patch, HBASE-18770.master.005.patch, > HBASE-18770.master.006.patch, HBASE-18770.master.007.patch, > HBASE-18770.master.008.patch, HBASE-18770.master.009.patch, > HBASE-18770.master.010.patch, HBASE-18770.master.011.patch > > > http://search-hadoop.com/m/HBase/YGbbXd0RDCIHSC1 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18770) Remove bypass method in ObserverContext and implement the 'bypass' logic case by case
[ https://issues.apache.org/jira/browse/HBASE-18770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18770: -- Attachment: HBASE-18770.master.011.patch > Remove bypass method in ObserverContext and implement the 'bypass' logic case > by case > - > > Key: HBASE-18770 > URL: https://issues.apache.org/jira/browse/HBASE-18770 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Assignee: stack >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18770.master.001.patch, > HBASE-18770.master.002.patch, HBASE-18770.master.003.patch, > HBASE-18770.master.004.patch, HBASE-18770.master.005.patch, > HBASE-18770.master.006.patch, HBASE-18770.master.007.patch, > HBASE-18770.master.008.patch, HBASE-18770.master.009.patch, > HBASE-18770.master.010.patch, HBASE-18770.master.011.patch > > > http://search-hadoop.com/m/HBase/YGbbXd0RDCIHSC1 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19035) Miss metrics when coprocessor use region scanner to read data
[ https://issues.apache.org/jira/browse/HBASE-19035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225980#comment-16225980 ] stack commented on HBASE-19035: --- [~zghaobac] We still need this it looks like. The philosophy is not to expose core metrics to coprocessors and to update counters even if the coprocessor bypasses normal operation. In the patch, the remove of the counting in RSRpcServer looks good as does purge of metrics methods from Region. Over in https://issues.apache.org/jira/browse/HBASE-18770?focusedCommentId=16225855=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16225855 there is a table on updates to make it so we up metrics even when coprocessor calls bypass. > Miss metrics when coprocessor use region scanner to read data > - > > Key: HBASE-19035 > URL: https://issues.apache.org/jira/browse/HBASE-19035 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19035.master.001.patch, > HBASE-19035.master.002.patch > > > Region interface is exposed to coprocessor. So coprocessor use getScanner to > get a region scanner to read data. But the scan metrics was only updated in > region server level. So we will miss some scan metrics for the read from > coprocessor. > || Region Operation || When to update requests metric || > | get | update read metric in nextRaw() | > | put | update write metric in batchMutate() | > | delete | update write metric in batchMutate() | > | increment | update read metric by get() and update write metric in > doDelta() | > | append | update read metric by get() and update write metric in doDelta() | > | mutateRow | update write metric in processRowsWithLocks() | > | mutateRowsWithLocks | update write metric in processRowsWithLocks() | > | batchMutate | update write metric in batchMutate() | > | checkAndMutate | update read metric by get() and update write metric by > mutateRow() | > | checkAndRowMutate | update read metric by get() and update write metric by > doBatchMutate() | > | processRowsWithLocks | update write metric in processRowsWithLocks() | > Meanwhile, remove requestRowActionCount in RSRpcServices. This metric can be > computed by region's readRequestsCount and writeRequestsCount. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18770) Remove bypass method in ObserverContext and implement the 'bypass' logic case by case
[
https://issues.apache.org/jira/browse/HBASE-18770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-18770:
--
Release Note:
Removes blanket bypass mechanism (Observer#bypass). Instead, a curated subset
of methods are bypassable.
Changes Coprocessor ObserverContext 'bypass' semantic. We flip the
default so bypass is NOT supported on Observer invocations; only a
couple of preXXX methods in RegionObserver allow it: e.g. preGet
and prePut but not preFlush, etc. Everywhere else, we throw
a Exception if a Coprocessor Observer tries to invoke bypass. Master
Observers can no longer stop or change move, split, assign, create table,
etc.
preBatchMutate can no longer be bypassed (bypass the finer-grained
prePut, preDelete, etc. instead)
Ditto on complete, the mechanism that allowed a Coprocessor
rule that all subsequent Coprocessors are skipped in an
invocation chain; now, complete is only available to
bypassable methods (and Coprocessors will get an exception if
they try to 'complete' when it is not allowed).
See javadoc for whether a Coprocessor Observer method supports
'bypass'. If no mention, 'bypass' is NOT supported.
The below methods have been marked deprecated in hbase2. We would have liked to
have removed them because they use IA.Private parameters but they are in use by
CoreCoprocessors or are critical to downstreamers and we have no alternatives
to provide currently.
@Deprecated public boolean prePrepareTimeStampForDeleteVersion(final Mutation
mutation, final Cell kv, final byte[] byteNow, final Get get) throws
IOException {
@Deprecated public boolean preWALRestore(final RegionInfo info, final WALKey
logKey, final WALEdit logEdit) throws IOException {
@Deprecated public void postWALRestore(final RegionInfo info, final WALKey
logKey, final WALEdit logEdit) throws IOException {
@Deprecated public DeleteTracker postInstantiateDeleteTracker(DeleteTracker
result) throws IOException
Metrics are updated now even if the Coprocessor does a bypass; e.g. The put
count is updated even if a Coprocessor bypasses the core put operation (We do
it this way so no need for Coprocessors to have access to our core metrics
system).
was:
Removes blanket bypass mechanism (Observer#bypass). Instead, a curated ubset of
methods are bypassable.
Changes Coprocessor ObserverContext 'bypass' semantic. We flip the
default so bypass is NOT supported on Observer invocations; only a
couple of preXXX methods in RegionObserver allow it: e.g. preGet
and prePut but not preFlush, etc. Everywhere else, we throw
a Exception if a Coprocessor Observer tries to invoke bypass. Master
Observers can no longer stop or change move, split, assign, create table,
etc.
preBatchMutate can no longer be bypassed (bypass the finer-grained
prePut, preDelete, etc. instead)
Ditto on complete, the mechanism that allowed a Coprocessor
rule that all subsequent Coprocessors are skipped in an
invocation chain; now, complete is only available to
bypassable methods (and Coprocessors will get an exception if
they try to 'complete' when it is not allowed).
See javadoc for whether a Coprocessor Observer method supports
'bypass'. If no mention, 'bypass' is NOT supported.
The below methods have been marked deprecated in hbase2. We would have liked to
have removed them because they use IA.Private parameters but they are in use by
CoreCoprocessors or are critical to downstreamers and we have no alternatives
to provide currently.
@Deprecated public boolean prePrepareTimeStampForDeleteVersion(final Mutation
mutation, final Cell kv, final byte[] byteNow, final Get get) throws
IOException {
@Deprecated public boolean preWALRestore(final RegionInfo info, final WALKey
logKey, final WALEdit logEdit) throws IOException {
@Deprecated public void postWALRestore(final RegionInfo info, final WALKey
logKey, final WALEdit logEdit) throws IOException {
@Deprecated public DeleteTracker postInstantiateDeleteTracker(DeleteTracker
result) throws IOException
Metrics are updated now even if the Coprocessor does a bypass; e.g. The put
count is updated even if a Coprocessor bypasses the core put operation (We do
it this way so no need for Coprocessors to have access to our core metrics
system).
> Remove bypass method in ObserverContext and implement the 'bypass' logic case
> by case
> -
>
> Key: HBASE-18770
> URL: https://issues.apache.org/jira/browse/HBASE-18770
> Project: HBase
> Issue Type: Sub-task
> Components: Coprocessors
>Reporter: Duo Zhang
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0-alpha-4
>
>
[jira] [Resolved] (HBASE-11131) Improve essential column family handling when more than one SingleColumnValueFilter is involved
[
https://issues.apache.org/jira/browse/HBASE-11131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu resolved HBASE-11131.
Resolution: Won't Fix
Stale issue
> Improve essential column family handling when more than one
> SingleColumnValueFilter is involved
> ---
>
> Key: HBASE-11131
> URL: https://issues.apache.org/jira/browse/HBASE-11131
> Project: HBase
> Issue Type: Task
> Components: Filters
>Reporter: Ted Yu
>Priority: Minor
>
> Currently, SingleColumnValueFilter determines essential column family based
> on:
> {code}
> public boolean isFamilyEssential(byte[] name) {
> return !this.filterIfMissing || Bytes.equals(name, this.columnFamily);
> {code}
> When more than one SingleColumnValueFilter is used through FilterList, some
> optimization can be done.
> e.g. when they are chained by Operator.MUST_PASS_ALL, the second and
> subsequent SingleColumnValueFilter can declare to be non-essential.
> This way their evaluation would depend on the outcome of the first
> SingleColumnValueFilter.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19106) Backup self validation for its correctness.
[ https://issues.apache.org/jira/browse/HBASE-19106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225942#comment-16225942 ] Ted Yu commented on HBASE-19106: The amount of data in the source tables may be big. Where should the validation job restore to ? Considering that restore job may take non-trivial duration, how should the result of restore be delivered ? Thanks > Backup self validation for its correctness. > --- > > Key: HBASE-19106 > URL: https://issues.apache.org/jira/browse/HBASE-19106 > Project: HBase > Issue Type: Improvement >Reporter: Amit Kabra > > Backups are critical and if they don't work when we need them at the time of > restore than they are not useful. We should do sanity test for each backup > job we run that it is restorable and hence can be trusted. > A self validation feature can be added for the same to the backups where > whenever a backup is run , once it finishes it will trigger a validation job > that will do a sample restoration of the backed up data and will make sure > that it compares well with actual data. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19127) Set State.SPLITTING, MERGING, MERGING_NEW, SPLITTING_NEW properly in RegionStatesNode
[
https://issues.apache.org/jira/browse/HBASE-19127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Liang updated HBASE-19127:
-
Attachment: state.patch
Try unit test,
[~stack] [~jerryhe], I found some issues about regionstates especially about
the intermediate state.
In the patch, I removed MERGE_TABLE_REGIONS_MOVE_REGION_TO_SAME_RS, this
MERGE_TABLE_REGIONS_MOVE_REGION_TO_SAME_RS has never been used.
and also removed MERGE_TABLE_REGIONS_SET_MERGING_TABLE_STATE, And the method
under this step is commented out, so nothing will be done under this step. And
I think we do not need to have a specific step for it.
After read RegionState.java; Set above states follow rules below
1. SPLITTING => After check the parent regions is splittable, set it to parent
region
2. SPLITTING_NEW => Set it after create daughter regions and before Assign
these daughters as OPEN in their region states.
3. Merging => After check 2 parent regions are mergeable, set it to both parent
regions.
4. Merging_new => After create merged regions and before assign it as OPEN.
above states won't affect the real procedure work, we set it because in the
metrics will use them, and also RegionStates need to keep latest/correct state.
> Set State.SPLITTING, MERGING, MERGING_NEW, SPLITTING_NEW properly in
> RegionStatesNode
> -
>
> Key: HBASE-19127
> URL: https://issues.apache.org/jira/browse/HBASE-19127
> Project: HBase
> Issue Type: Improvement
>Reporter: Yi Liang
>Assignee: Yi Liang
> Attachments: state.patch
>
>
> In current code, we did not set above states to a region node at all, but we
> still have statements like below to check if node have above states.
> {code}
> else if (!regionNode.isInState(State.CLOSING, State.SPLITTING)) {
>
> }
> {code}
> We need to set above states in a correct place.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (HBASE-19127) Set State.SPLITTING, MERGING, MERGING_NEW, SPLITTING_NEW properly in RegionStatesNode
[
https://issues.apache.org/jira/browse/HBASE-19127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Liang reassigned HBASE-19127:
Assignee: Yi Liang
> Set State.SPLITTING, MERGING, MERGING_NEW, SPLITTING_NEW properly in
> RegionStatesNode
> -
>
> Key: HBASE-19127
> URL: https://issues.apache.org/jira/browse/HBASE-19127
> Project: HBase
> Issue Type: Improvement
>Reporter: Yi Liang
>Assignee: Yi Liang
>
> In current code, we did not set above states to a region node at all, but we
> still have statements like below to check if node have above states.
> {code}
> else if (!regionNode.isInState(State.CLOSING, State.SPLITTING)) {
>
> }
> {code}
> We need to set above states in a correct place.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19128) Purge Distributed Log Replay from codebase, configurations, text; mark the feature as unsupported, broken.
stack created HBASE-19128: - Summary: Purge Distributed Log Replay from codebase, configurations, text; mark the feature as unsupported, broken. Key: HBASE-19128 URL: https://issues.apache.org/jira/browse/HBASE-19128 Project: HBase Issue Type: Sub-task Reporter: stack Kill it. It keeps coming up and over again. Needs proper burial. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-17591) Update documentation to say distributed log replay defaults to 'false'
[
https://issues.apache.org/jira/browse/HBASE-17591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-17591:
--
Fix Version/s: 2.0.0-beta-1
> Update documentation to say distributed log replay defaults to 'false'
> --
>
> Key: HBASE-17591
> URL: https://issues.apache.org/jira/browse/HBASE-17591
> Project: HBase
> Issue Type: Improvement
> Components: documentation
>Reporter: Aki Ariga
>Priority: Trivial
> Fix For: 2.0.0-beta-1
>
>
> As consequence of HBASE-12577, `hbase.master.distributed.log.replay` is no
> longer default true. But in the documentation, it is still noted as default
> true as follows:
> {quote}
> To enable distributed log replay, set hbase.master.distributed.log.replay to
> true. This will be the default for HBase 0.99 (HBASE-10888).
> {quote}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-7006) [MTTR] Improve Region Server Recovery Time - Distributed Log Replay
[ https://issues.apache.org/jira/browse/HBASE-7006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225891#comment-16225891 ] stack commented on HBASE-7006: -- Just to comment that this feature has been abandoned. It never worked properly. It will not be supported, at least in the guise it had here. > [MTTR] Improve Region Server Recovery Time - Distributed Log Replay > --- > > Key: HBASE-7006 > URL: https://issues.apache.org/jira/browse/HBASE-7006 > Project: HBase > Issue Type: New Feature > Components: MTTR >Reporter: stack >Assignee: Jeffrey Zhong >Priority: Critical > Fix For: 0.98.0, 0.95.1 > > Attachments: 7006-addendum-3.txt, LogSplitting Comparison.pdf, > ProposaltoimprovelogsplittingprocessregardingtoHBASE-7006-v2.pdf, > hbase-7006-addendum.patch, hbase-7006-combined-v1.patch, > hbase-7006-combined-v4.patch, hbase-7006-combined-v5.patch, > hbase-7006-combined-v6.patch, hbase-7006-combined-v7.patch, > hbase-7006-combined-v8.patch, hbase-7006-combined-v9.patch, > hbase-7006-combined.patch > > > Just saw interesting issue where a cluster went down hard and 30 nodes had > 1700 WALs to replay. Replay took almost an hour. It looks like it could run > faster that much of the time is spent zk'ing and nn'ing. > Putting in 0.96 so it gets a look at least. Can always punt. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19127) Set State.SPLITTING, MERGING, MERGING_NEW, SPLITTING_NEW properly in RegionStatesNode
Yi Liang created HBASE-19127:
Summary: Set State.SPLITTING, MERGING, MERGING_NEW, SPLITTING_NEW
properly in RegionStatesNode
Key: HBASE-19127
URL: https://issues.apache.org/jira/browse/HBASE-19127
Project: HBase
Issue Type: Improvement
Reporter: Yi Liang
In current code, we did not set above states to a region node at all, but we
still have statements like below to check if node have above states.
{code}
else if (!regionNode.isInState(State.CLOSING, State.SPLITTING)) {
}
{code}
We need to set above states in a correct place.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18232) Add variable size chunks to the MSLAB
[ https://issues.apache.org/jira/browse/HBASE-18232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anastasia Braginsky updated HBASE-18232: Attachment: HBASE-18232-V10.patch > Add variable size chunks to the MSLAB > - > > Key: HBASE-18232 > URL: https://issues.apache.org/jira/browse/HBASE-18232 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Attachments: HBASE-18232-V03.patch, HBASE-18232-V05.patch, > HBASE-18232-V06.patch, HBASE-18232-V07.patch, HBASE-18232-V08.patch, > HBASE-18232-V09.patch, HBASE-18232-V10.patch > > > Add possibility to create a variable size chunks of memory, so any cell (of > any size) can reside on a chunk. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18770) Remove bypass method in ObserverContext and implement the 'bypass' logic case by case
[
https://issues.apache.org/jira/browse/HBASE-18770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-18770:
--
Release Note:
Removes blanket bypass mechanism (Observer#bypass). Instead, a curated ubset of
methods are bypassable.
Changes Coprocessor ObserverContext 'bypass' semantic. We flip the
default so bypass is NOT supported on Observer invocations; only a
couple of preXXX methods in RegionObserver allow it: e.g. preGet
and prePut but not preFlush, etc. Everywhere else, we throw
a Exception if a Coprocessor Observer tries to invoke bypass. Master
Observers can no longer stop or change move, split, assign, create table,
etc.
preBatchMutate can no longer be bypassed (bypass the finer-grained
prePut, preDelete, etc. instead)
Ditto on complete, the mechanism that allowed a Coprocessor
rule that all subsequent Coprocessors are skipped in an
invocation chain; now, complete is only available to
bypassable methods (and Coprocessors will get an exception if
they try to 'complete' when it is not allowed).
See javadoc for whether a Coprocessor Observer method supports
'bypass'. If no mention, 'bypass' is NOT supported.
The below methods have been marked deprecated in hbase2. We would have liked to
have removed them because they use IA.Private parameters but they are in use by
CoreCoprocessors or are critical to downstreamers and we have no alternatives
to provide currently.
@Deprecated public boolean prePrepareTimeStampForDeleteVersion(final Mutation
mutation, final Cell kv, final byte[] byteNow, final Get get) throws
IOException {
@Deprecated public boolean preWALRestore(final RegionInfo info, final WALKey
logKey, final WALEdit logEdit) throws IOException {
@Deprecated public void postWALRestore(final RegionInfo info, final WALKey
logKey, final WALEdit logEdit) throws IOException {
@Deprecated public DeleteTracker postInstantiateDeleteTracker(DeleteTracker
result) throws IOException
Metrics are updated now even if the Coprocessor does a bypass; e.g. The put
count is updated even if a Coprocessor bypasses the core put operation (We do
it this way so no need for Coprocessors to have access to our core metrics
system).
was:
Removes blanket bypass mechanism (Observer#bypass). Instead, a curated ubset of
methods are bypassable.
Changes Coprocessor ObserverContext 'bypass' semantic. We flip the
default so bypass is NOT supported on Observer invocations; only a
couple of preXXX methods in RegionObserver allow it: e.g. preGet
and prePut but not preFlush, etc. Everywhere else, we throw
a Exception if a Coprocessor Observer tries to invoke bypass. Master
Observers can no longer stop or change move, split, assign, create table,
etc.
preBatchMutate can no longer be bypassed (bypass the finer-grained
prePut, preDelete, etc. instead)
Ditto on complete, the mechanism that allowed a Coprocessor
rule that all subsequent Coprocessors are skipped in an
invocation chain; now, complete is only available to
bypassable methods (and Coprocessors will get an exception if
they try to 'complete' when it is not allowed).
See javadoc for whether a Coprocessor Observer method supports
'bypass'. If no mention, 'bypass' is NOT supported.
The below methods have been marked deprecated in hbase2. We would have liked to
have removed them because they use IA.Private parameters but they are in use by
CoreCoprocessors or are critical to downstreamers and we have no alternatives
to provide currently.
@Deprecated public boolean prePrepareTimeStampForDeleteVersion(final Mutation
mutation, final Cell kv, final byte[] byteNow, final Get get) throws
IOException {
@Deprecated public boolean preWALRestore(final RegionInfo info, final WALKey
logKey, final WALEdit logEdit) throws IOException {
@Deprecated public void postWALRestore(final RegionInfo info, final WALKey
logKey, final WALEdit logEdit) throws IOException {
@Deprecated public DeleteTracker postInstantiateDeleteTracker(DeleteTracker
result) throws IOException
> Remove bypass method in ObserverContext and implement the 'bypass' logic case
> by case
> -
>
> Key: HBASE-18770
> URL: https://issues.apache.org/jira/browse/HBASE-18770
> Project: HBase
> Issue Type: Sub-task
> Components: Coprocessors
>Reporter: Duo Zhang
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-18770.master.001.patch,
> HBASE-18770.master.002.patch, HBASE-18770.master.003.patch,
> HBASE-18770.master.004.patch, HBASE-18770.master.005.patch,
> HBASE-18770.master.006.patch, HBASE-18770.master.007.patch,
>
[jira] [Commented] (HBASE-18770) Remove bypass method in ObserverContext and implement the 'bypass' logic case by case
[ https://issues.apache.org/jira/browse/HBASE-18770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225855#comment-16225855 ] stack commented on HBASE-18770: --- h1. Survey on Metric updates when bypass Highlevel, all bypasses update their respective metrics now (Previous, about half did and half did not). h2. RegionObserver ||method||metrics updated||notes|| |preCompactSelection|yes|CP can change the files to compact on the compaction request.| |preGetOp|yes|We now do metrics update on bypass -- two places, in HRegion and RSRpcServers| |preExists|yes|No change necessary; done in RSRPCServices; same metric update whether CP or not| |prePut|yes|One path, RowProcessor is being remove; other path, added update of metrics when we do the prePut and we bypass| |prePrepareTimeStampForDeleteVersion|yes|Just changes timestamp; same metrics get called| |preCheckAndPut|yes|We were updating metrics whether CP bypass or not| |preCheckAndPutAfterRowLock|yes|Inside Region. Upper level increment already happened. Usually a compound get + put, etc. They do their own metrics| |preCheckAndDelete|yes|Same as preCheckAndPut| |preCheckAndDeleteAfterRowLock|yes|Same as preCheckAndPutAfterRowLock| |preAppend|yes|Metrics were being updated whether CP bypass or not| |preAppendAfterRowLock|yes|Metrics updated in finally block whether CP bypass or not| |preIncrement|yes|Metrics updated regardless of CP bypass or not| |preIncrementAfterRowLock|yes|Same as preAppendAfterRowLock| h2. MasterObserver preSetSplitOrMergeEnabled yes preSetSplitOrMergeEnabled yes > Remove bypass method in ObserverContext and implement the 'bypass' logic case > by case > - > > Key: HBASE-18770 > URL: https://issues.apache.org/jira/browse/HBASE-18770 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Assignee: stack >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18770.master.001.patch, > HBASE-18770.master.002.patch, HBASE-18770.master.003.patch, > HBASE-18770.master.004.patch, HBASE-18770.master.005.patch, > HBASE-18770.master.006.patch, HBASE-18770.master.007.patch, > HBASE-18770.master.008.patch, HBASE-18770.master.009.patch, > HBASE-18770.master.010.patch > > > http://search-hadoop.com/m/HBase/YGbbXd0RDCIHSC1 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19126) [AMv2] RegionStates/RegionStateNode needs cleanup
[ https://issues.apache.org/jira/browse/HBASE-19126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang updated HBASE-19126: - Description: // Mutable/Immutable? Changes have to be synchronized or not? // Data members are volatile which seems to say multi-threaded access is fine. // In the below we do check and set but the check state could change before // we do the set because no synchronizationwhich seems dodgy. Clear up // understanding here... how many threads accessing? Do locks make it so one // thread at a time working on a single Region's RegionStateNode? Lets presume // so for now. Odd is that elsewhere in this RegionStates, we synchronize on // the RegionStateNode instance Copied from TODO in RegionState.java Open this jira to track some cleanups for RegionStates/RegionStateNode was: // Mutable/Immutable? Changes have to be synchronized or not? // Data members are volatile which seems to say multi-threaded access is fine. // In the below we do check and set but the check state could change before // we do the set because no synchronizationwhich seems dodgy. Clear up // understanding here... how many threads accessing? Do locks make it so one // thread at a time working on a single Region's RegionStateNode? Lets presume // so for now. Odd is that elsewhere in this RegionStates, we synchronize on // the RegionStateNode instance Open this jira to track some cleanups for RegionStates/RegionStateNode > [AMv2] RegionStates/RegionStateNode needs cleanup > - > > Key: HBASE-19126 > URL: https://issues.apache.org/jira/browse/HBASE-19126 > Project: HBase > Issue Type: Improvement >Reporter: Yi Liang > Fix For: 2.0.0-beta-1 > > > // Mutable/Immutable? Changes have to be synchronized or not? > // Data members are volatile which seems to say multi-threaded access is > fine. > // In the below we do check and set but the check state could change before > // we do the set because no synchronizationwhich seems dodgy. Clear up > // understanding here... how many threads accessing? Do locks make it so one > // thread at a time working on a single Region's RegionStateNode? Lets > presume > // so for now. Odd is that elsewhere in this RegionStates, we synchronize on > // the RegionStateNode instance > Copied from TODO in RegionState.java > Open this jira to track some cleanups for RegionStates/RegionStateNode -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19126) [AMv2] RegionStates/RegionStateNode needs cleanup
[ https://issues.apache.org/jira/browse/HBASE-19126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang updated HBASE-19126: - Summary: [AMv2] RegionStates/RegionStateNode needs cleanup (was: [AMv2] RegionStates/RegionStateNode cleanup) > [AMv2] RegionStates/RegionStateNode needs cleanup > - > > Key: HBASE-19126 > URL: https://issues.apache.org/jira/browse/HBASE-19126 > Project: HBase > Issue Type: Improvement >Reporter: Yi Liang > Fix For: 2.0.0-beta-1 > > > // Mutable/Immutable? Changes have to be synchronized or not? > // Data members are volatile which seems to say multi-threaded access is > fine. > // In the below we do check and set but the check state could change before > // we do the set because no synchronizationwhich seems dodgy. Clear up > // understanding here... how many threads accessing? Do locks make it so one > // thread at a time working on a single Region's RegionStateNode? Lets > presume > // so for now. Odd is that elsewhere in this RegionStates, we synchronize on > // the RegionStateNode instance > Open this jira to track some cleanups for RegionStates/RegionStateNode -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19126) [AMv2] RegionStates/RegionStateNode cleanup
Yi Liang created HBASE-19126: Summary: [AMv2] RegionStates/RegionStateNode cleanup Key: HBASE-19126 URL: https://issues.apache.org/jira/browse/HBASE-19126 Project: HBase Issue Type: Improvement Reporter: Yi Liang Fix For: 2.0.0-beta-1 // Mutable/Immutable? Changes have to be synchronized or not? // Data members are volatile which seems to say multi-threaded access is fine. // In the below we do check and set but the check state could change before // we do the set because no synchronizationwhich seems dodgy. Clear up // understanding here... how many threads accessing? Do locks make it so one // thread at a time working on a single Region's RegionStateNode? Lets presume // so for now. Odd is that elsewhere in this RegionStates, we synchronize on // the RegionStateNode instance Open this jira to track some cleanups for RegionStates/RegionStateNode -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19125) TestReplicator is flaky
Andrew Purtell created HBASE-19125: -- Summary: TestReplicator is flaky Key: HBASE-19125 URL: https://issues.apache.org/jira/browse/HBASE-19125 Project: HBase Issue Type: Bug Reporter: Andrew Purtell Assignee: Andrew Purtell Priority: Minor Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0 TestReplicator fails now and again. I had a look at the test. This is something I contributed a while back but looking at it again it needs a different approach. I'm going to disable it for now until this issue is resolved. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19089) Fix the list of included moduleSets in src and binary tars
[ https://issues.apache.org/jira/browse/HBASE-19089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225829#comment-16225829 ] Appy commented on HBASE-19089: -- Filed HBASE-19124. Thanks [~busbey]. > Fix the list of included moduleSets in src and binary tars > -- > > Key: HBASE-19089 > URL: https://issues.apache.org/jira/browse/HBASE-19089 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy > Attachments: HBASE-19089.master.001.patch > > > List of moduleSets included in src.xml and hadoop-two-compat.xml differ quite > a lot. Particularly, hadoop-two-compat.xml is missing quite a few modules. > The core issue is duplication involved in list. Let me try to get > rid of it by using a shared list and including it using -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19124) Move HBase-Nightly test from JenkinsFile to a script in dev-support
Appy created HBASE-19124: Summary: Move HBase-Nightly test from JenkinsFile to a script in dev-support Key: HBASE-19124 URL: https://issues.apache.org/jira/browse/HBASE-19124 Project: HBase Issue Type: Bug Reporter: Appy ref: https://issues.apache.org/jira/browse/HBASE-19089?focusedCommentId=16221551=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16221551 Moving that jenkins test to script will make it easier to test changes to hbase-assembly. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18925) Need updated mockito for using java optional
[ https://issues.apache.org/jira/browse/HBASE-18925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-18925: - Attachment: HBASE-18925.master.007.patch > Need updated mockito for using java optional > > > Key: HBASE-18925 > URL: https://issues.apache.org/jira/browse/HBASE-18925 > Project: HBase > Issue Type: Improvement >Reporter: Appy >Assignee: Appy > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-18925.master.001.patch, > HBASE-18925.master.002.patch, HBASE-18925.master.002.patch, > HBASE-18925.master.003.patch, HBASE-18925.master.004.patch, > HBASE-18925.master.005.patch, HBASE-18925.master.006.patch, > HBASE-18925.master.007.patch > > > Came up when i was trying to test HBASE-18878. > It kept failing because mock of RpcCall returned null where return type was > Optional. > Instead, we want it to return Optional.empty(). > New mockito versions support this (and other java8 things) - > https://github.com/mockito/mockito/wiki/What%27s-new-in-Mockito-2 > We use mockito-all which was last released in Dec2014. However, mockito-core > has had more than 50 releases after that > (https://mvnrepository.com/artifact/org.mockito/mockito-core). > We need to change our deps from mockito-all to mockito-core. > However that comes with fair breakages, so this is not a simple task of > changing pom files. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18995) Move methods that are for internal usage from CellUtil to Private util class
[
https://issues.apache.org/jira/browse/HBASE-18995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225805#comment-16225805
]
Mike Drob commented on HBASE-18995:
---
Hold up... right in the CellUtil javadoc it has...
(before the changes)
{noformat}
* Utility methods helpful slinging {@link Cell} instances.
* Some methods below are for internal use only and are marked
InterfaceAudience.Private at the
* method level.
{noformat}
So I don't understand the motivation for moving things to a new class when they
were already annotated?
> Move methods that are for internal usage from CellUtil to Private util class
>
>
> Key: HBASE-18995
> URL: https://issues.apache.org/jira/browse/HBASE-18995
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.0.0-alpha-3
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-18995-branch-2.002.patch,
> HBASE-18995-branch-2.patch, HBASE-18995-branch-2_1.patch,
> HBASE-18995-branch-2_1.patch, HBASE-18995-branch-2_1.patch,
> HBASE-18995-branch-2_1.patch, HBASE-18995-branch-2_2.patch,
> HBASE-18995_002-branch-2.patch, HBASE-18995_002-branch-2.patch,
> HBASE-18995_003-branch-2.patch, HBASE-18995_1.patch, HBASE-18995_2.patch,
> HBASE-18995_2.patch
>
>
> This was brought up long time back. We need to move some of the public APIs
> from CellUtil to internal private Util class because they are used in some
> internal flow and does not make sense to have it in a @public exposed Util
> class.
> The topic again came in HBASE-18945 RB comments also.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19120) IllegalArgumentException from ZNodeClearer when master shuts down
[
https://issues.apache.org/jira/browse/HBASE-19120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225796#comment-16225796
]
Ted Yu commented on HBASE-19120:
Ran the crashed tests locally with patch which passed.
> IllegalArgumentException from ZNodeClearer when master shuts down
> -
>
> Key: HBASE-19120
> URL: https://issues.apache.org/jira/browse/HBASE-19120
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 19120.v1.txt
>
>
> Found the following in master log (build as of commit eee3b0) :
> {code}
> 2017-10-30 15:40:24,383 ERROR [main] util.ServerCommandLine: Failed to run
> java.lang.IllegalArgumentException: Path must start with / character
> at
> org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:51)
> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:851)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:182)
> at
> org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNodeFailSilent(ZKUtil.java:1266)
> at
> org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNodeFailSilent(ZKUtil.java:1258)
> at org.apache.hadoop.hbase.ZNodeClearer.clear(ZNodeClearer.java:186)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:143)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:127)
> at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2873)
> {code}
> Looking at ZNodeClearer, it seems that intention was to remove znode under
> /rs subtree.
> However, the znode name was passed without path.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19119) hbase-http shouldn't have a native profile
[ https://issues.apache.org/jira/browse/HBASE-19119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225793#comment-16225793 ] Hudson commented on HBASE-19119: FAILURE: Integrated in Jenkins build HBase-2.0 #774 (See [https://builds.apache.org/job/HBase-2.0/774/]) HBASE-19119 hbase-http shouldn't have native profile (mdrob: rev 3de44f918b39912dc3d72948e724aed052419717) * (edit) hbase-http/pom.xml > hbase-http shouldn't have a native profile > -- > > Key: HBASE-19119 > URL: https://issues.apache.org/jira/browse/HBASE-19119 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 2.0.0-alpha-4 >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Blocker > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-19119.patch > > > When splitting out the hbase-http module in HBASE-19053 we accidentally > included a native profile that will fail when invoked. I think we can just > nix that section since the native code is still in hbase-server. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19106) Backup self validation for its correctness.
[ https://issues.apache.org/jira/browse/HBASE-19106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-19106: --- Issue Type: Improvement (was: Bug) > Backup self validation for its correctness. > --- > > Key: HBASE-19106 > URL: https://issues.apache.org/jira/browse/HBASE-19106 > Project: HBase > Issue Type: Improvement >Reporter: Amit Kabra > > Backups are critical and if they don't work when we need them at the time of > restore than they are not useful. We should do sanity test for each backup > job we run that it is restorable and hence can be trusted. > A self validation feature can be added for the same to the backups where > whenever a backup is run , once it finishes it will trigger a validation job > that will do a sample restoration of the backed up data and will make sure > that it compares well with actual data. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19104) Add filtering during restore in HBase backups.
[ https://issues.apache.org/jira/browse/HBASE-19104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-19104: --- Issue Type: New Feature (was: Bug) > Add filtering during restore in HBase backups. > -- > > Key: HBASE-19104 > URL: https://issues.apache.org/jira/browse/HBASE-19104 > Project: HBase > Issue Type: New Feature >Reporter: Amit Kabra > > When we deal with large amount of data, it would be great , if we can do data > restore from backups based on tenant , based on time range , etc , so that if > finishes faster and we restore only what's required. > Currently restore take backup id as input and restore all the data will that > backup id time stamp. We may not need to restore all data in a given backup > id. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19105) Add ability to compare backups in HBase backups.
[ https://issues.apache.org/jira/browse/HBASE-19105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-19105: --- Issue Type: New Feature (was: Bug) > Add ability to compare backups in HBase backups. > > > Key: HBASE-19105 > URL: https://issues.apache.org/jira/browse/HBASE-19105 > Project: HBase > Issue Type: New Feature >Reporter: Amit Kabra > > For certain scenarios eg DR scenario, before making a site switch we need to > ensure that backups in primary and dr is same. Tool to compare the backups > helps in such case that can do cross cluster backups validation. > > Current backups generate data in backup_ format and this can be > different in primary and dr and is not easily comparable. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19107) Backup should resume from last failed state
[ https://issues.apache.org/jira/browse/HBASE-19107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-19107: --- Labels: backup (was: ) > Backup should resume from last failed state > --- > > Key: HBASE-19107 > URL: https://issues.apache.org/jira/browse/HBASE-19107 > Project: HBase > Issue Type: New Feature >Reporter: Vishal Khandelwal > Labels: backup > > Consider that incremental backup is happenng for 10 tables for a set of 100 > WALs. if backup @ 10th fails, then on net iteration it would be triggered > again from 100 WALs + new WALs all tables > This is a issue because resource and time used to do he backup of 9 table & > 100 WALs are wasted > Rather it should have functionality to resume from previous state complete > that backup and then do new set of WALs. This will make it more performant -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19107) Backup should resume from last failed state
[ https://issues.apache.org/jira/browse/HBASE-19107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-19107: --- Description: Consider that incremental backup is happenng for 10 tables for a set of 100 WALs. if backup @ 10th fails, then on net iteration it would be triggered again from 100 WALs + new WALs all tables This is a issue because resource and time used to do he backup of 9 table & 100 WALs are wasted Rather it should have functionality to resume from previous state complete that backup and then do new set of WALs. This will make it more performant > Backup should resume from last failed state > --- > > Key: HBASE-19107 > URL: https://issues.apache.org/jira/browse/HBASE-19107 > Project: HBase > Issue Type: New Feature >Reporter: Vishal Khandelwal > > Consider that incremental backup is happenng for 10 tables for a set of 100 > WALs. if backup @ 10th fails, then on net iteration it would be triggered > again from 100 WALs + new WALs all tables > This is a issue because resource and time used to do he backup of 9 table & > 100 WALs are wasted > Rather it should have functionality to resume from previous state complete > that backup and then do new set of WALs. This will make it more performant -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19108) Backups should understand the frequency of full and incremental backup
[ https://issues.apache.org/jira/browse/HBASE-19108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-19108: --- Description: Backup scheduler which figures out tables for which backups are pending and executes all required backups. In case of failures, this automates required (full / incremental) backup execution based on the set frequency. Thus the support to set frequency and expire old backups should be added. Thsi will reduce and admin overhead. > Backups should understand the frequency of full and incremental backup > -- > > Key: HBASE-19108 > URL: https://issues.apache.org/jira/browse/HBASE-19108 > Project: HBase > Issue Type: New Feature >Reporter: Vishal Khandelwal > Labels: backup > > Backup scheduler which figures out tables for which backups are pending and > executes all required backups. > In case of failures, this automates required (full / incremental) backup > execution based on the set frequency. Thus the support to set frequency and > expire old backups should be added. Thsi will reduce and admin overhead. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19108) Backups should understand the frequency of full and incremental backup
[ https://issues.apache.org/jira/browse/HBASE-19108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-19108: --- Environment: (was: Backup scheduler which figures out tables for which backups are pending and executes all required backups. In case of failures, this automates required (full / incremental) backup execution based on the set frequency. Thus the support to set frequency and expire old backups should be added. Thsi will reduce and admin overhead. ) > Backups should understand the frequency of full and incremental backup > -- > > Key: HBASE-19108 > URL: https://issues.apache.org/jira/browse/HBASE-19108 > Project: HBase > Issue Type: New Feature >Reporter: Vishal Khandelwal > Labels: backup > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19107) Backup should resume from last failed state
[ https://issues.apache.org/jira/browse/HBASE-19107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-19107: --- Environment: (was: Consider that incremental backup is happenng for 10 tables for a set of 100 WALs. if backup @ 10th fails, then on net iteration it would be triggered again from 100 WALs + new WALs all tables This is a issue because resource and time used to do he backup of 9 table & 100 WALs are wasted Rather it should have functionality to resume from previous state complete that backup and then do new set of WALs. This will make it more performant ) > Backup should resume from last failed state > --- > > Key: HBASE-19107 > URL: https://issues.apache.org/jira/browse/HBASE-19107 > Project: HBase > Issue Type: New Feature >Reporter: Vishal Khandelwal > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19119) hbase-http shouldn't have a native profile
[ https://issues.apache.org/jira/browse/HBASE-19119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225777#comment-16225777 ] Hudson commented on HBASE-19119: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3974 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3974/]) HBASE-19119 hbase-http shouldn't have native profile (mdrob: rev a79b66b32b4e3eb98fb1c03cb545b8edeab26647) * (edit) hbase-http/pom.xml > hbase-http shouldn't have a native profile > -- > > Key: HBASE-19119 > URL: https://issues.apache.org/jira/browse/HBASE-19119 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 2.0.0-alpha-4 >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Blocker > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-19119.patch > > > When splitting out the hbase-http module in HBASE-19053 we accidentally > included a native profile that will fail when invoked. I think we can just > nix that section since the native code is still in hbase-server. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19122) preCompact and preFlush can bypass by returning null scanner; shut it down
[
https://issues.apache.org/jira/browse/HBASE-19122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-19122:
--
Priority: Critical (was: Major)
> preCompact and preFlush can bypass by returning null scanner; shut it down
> --
>
> Key: HBASE-19122
> URL: https://issues.apache.org/jira/browse/HBASE-19122
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors, Scanners
>Reporter: stack
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
>
> Noticed by [~anoop.hbase] during review of HBASE-18770, preCompact and
> preFlush can bypass normal processing by returning null. They are not
> bypasable by ordained route. We should shut down this avenue.
> The preCompact at least may be new coming in with:
> {code}
> tree dbf13093842f85a713f023d7219caccf8f4eb05f
> parent a4dcf51415616772e462091ce93622f070ea8810
> author zhangduo Sat Apr 9 16:18:08 2016 +0800
> committer zhangduo Sun Apr 10 09:26:28 2016 +0800
> HBASE-15527 Refactor Compactor related classes
> {code}
> Would have to dig in more to figure for sure.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19122) preCompact and preFlush can bypass by returning null scanner; shut it down
[
https://issues.apache.org/jira/browse/HBASE-19122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225766#comment-16225766
]
stack commented on HBASE-19122:
---
Thank you [~andrew.purt...@gmail.com] Helps. This is a quick fix.
> preCompact and preFlush can bypass by returning null scanner; shut it down
> --
>
> Key: HBASE-19122
> URL: https://issues.apache.org/jira/browse/HBASE-19122
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors, Scanners
>Reporter: stack
> Fix For: 2.0.0-beta-1
>
>
> Noticed by [~anoop.hbase] during review of HBASE-18770, preCompact and
> preFlush can bypass normal processing by returning null. They are not
> bypasable by ordained route. We should shut down this avenue.
> The preCompact at least may be new coming in with:
> {code}
> tree dbf13093842f85a713f023d7219caccf8f4eb05f
> parent a4dcf51415616772e462091ce93622f070ea8810
> author zhangduo Sat Apr 9 16:18:08 2016 +0800
> committer zhangduo Sun Apr 10 09:26:28 2016 +0800
> HBASE-15527 Refactor Compactor related classes
> {code}
> Would have to dig in more to figure for sure.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19120) IllegalArgumentException from ZNodeClearer when master shuts down
[
https://issues.apache.org/jira/browse/HBASE-19120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225754#comment-16225754
]
Hadoop QA commented on HBASE-19120:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m
21s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
2s{color} | {color:blue} The patch file was not named according to hbase's
naming conventions. Please see
https://yetus.apache.org/documentation/0.5.0/precommit-patchnames for
instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
57s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
6s{color} | {color:red} hbase-server: The patch generated 1 new + 3 unchanged -
0 fixed = 4 total (was 3) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
54s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
54m 16s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}100m 44s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}177m 49s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19120 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12894795/19120.v1.txt |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 658ca239c0f6 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26
18:04:35 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 6712f8f632 |
| Default Java | 1.8.0_141 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/9520/artifact/patchprocess/diff-checkstyle-hbase-server.txt
|
| unit |
