[jira] [Created] (HBASE-16077) Replication status doesnt show failed RS metrics in CLI
Bibin A Chundatt created HBASE-16077:
Summary: Replication status doesnt show failed RS metrics in CLI
Key: HBASE-16077
URL: https://issues.apache.org/jira/browse/HBASE-16077
Project: HBase
Issue Type: Bug
Reporter: Bibin A Chundatt
Steps to reproduce
# Create 2 clusters and configure replication
# Create TABLE 1 and enable table replication
# Shutdown Cluster 2 for short period.
# Load data to TABLE 1
# Shutdown Region Server whr Region of TABLE 1 is available
# Check metrics using CLI
{noformat}
hbase(main):003:0* status 'replication'
2016-06-14 00:58:04,664 INFO [main] ipc.AbstractRpcClient: RPC Server Kerberos
principal name for service=MasterService is hbase/hadoop.hadoop@hadoop.com
version 1.0.2
3 live servers
host-10-19-92-200:
SOURCE: PeerID=11, SizeOfLogQueue=0, ShippedBatches=30, ShippedOps=1351,
ShippedBytes=1513127672, LogReadInBytes=662648911, LogEditsRead=1546,
LogEditsFiltered=1409, SizeOfLogToReplicate=0,
TimeWillBeTakenForLogToReplicate=0, ShippedHFiles=0, SizeOfHFileRefsQueue=0,
AgeOfLastShippedOp=0, TimeStampsOfLastShippedOp=Tue Jun 14 00:58:01 IST 2016,
Replication Lag=0
SINK : AppliedBatches=2, AppliedOps=5, AppliedHFiles=3,
AgeOfLastAppliedOp=0, TimeStampsOfLastAppliedOp=Mon Jun 13 02:18:06 IST 2016
host-10-19-92-187:
SOURCE: PeerID=11, SizeOfLogQueue=0, ShippedBatches=0, ShippedOps=0,
ShippedBytes=0, LogReadInBytes=65719, LogEditsRead=112, LogEditsFiltered=112,
SizeOfLogToReplicate=0, TimeWillBeTakenForLogToReplicate=0, ShippedHFiles=0,
SizeOfHFileRefsQueue=0, AgeOfLastShippedOp=0, TimeStampsOfLastShippedOp=Tue Jun
14 00:58:01 IST 2016, Replication Lag=0
SINK : AppliedBatches=0, AppliedOps=0, AppliedHFiles=0,
AgeOfLastAppliedOp=0, TimeStampsOfLastAppliedOp=Mon Jun 13 09:07:20 IST 2016
host-10-19-92-188:
SOURCE: PeerID=11, SizeOfLogQueue=0, ShippedBatches=39, ShippedOps=1730,
ShippedBytes=1937609744, LogReadInBytes=848439638, LogEditsRead=1671,
LogEditsFiltered=1497, SizeOfLogToReplicate=0,
TimeWillBeTakenForLogToReplicate=0, ShippedHFiles=0, SizeOfHFileRefsQueue=0,
AgeOfLastShippedOp=0, TimeStampsOfLastShippedOp=Tue Jun 14 00:58:03 IST 2016,
Replication Lag=0
SINK : AppliedBatches=1, AppliedOps=1, AppliedHFiles=0,
AgeOfLastAppliedOp=0, TimeStampsOfLastAppliedOp=Mon Jun 13 01:53:53 IST 2016
{noformat}
*JMX output*
{noformat}
{
"name" : "Hadoop:service=HBase,name=RegionServer,sub=Replication",
"modelerType" : "RegionServer,sub=Replication",
"tag.Context" : "regionserver",
"tag.Hostname" : "host-10-19-92-200",
"source.11.sizeOfLogToReplicate" : 537,
"source.11-host-10-19-92-187,21302,1465787242095.sizeOfLogToReplicate" :
282766680,
"source.shippedHFiles" : 0,
"source.ageOfLastShippedOp" : 0,
"source.11.shippedHFiles" : 0,
"source.11-host-10-19-92-187,21302,1465787242095.ageOfLastShippedOp" : 0,
"source.shippedKBs" : 1477663,
"source.sizeOfHFileRefsQueue" : 0,
"source.logReadInBytes" : 691148656,
"source.11-host-10-19-92-187,21302,1465787242095.logEditsRead" : 39,
"source.11-host-10-19-92-187,21302,1465787242095.shippedOps" : 0,
"source.11.logEditsFiltered" : 1244,
"source.sizeOfLogQueue" : 4,
"source.timeWillBeTakenForLogToReplicate" : 1,
"sink.ageOfLastAppliedOp" : 0,
"source.11-host-10-19-92-187,21302,1465787242095.timeWillBeTakenForLogToReplicate"
: 0,
"source.logEditsRead" : 1420,
"source.11.sizeOfLogQueue" : 0,
"source.11-host-10-19-92-187,21302,1465787242095.logEditsFiltered" : 32,
"source.11-host-10-19-92-187,21302,1465787242095.shippedHFiles" : 0,
"source.shippedOps" : 1351,
"source.11.shippedKBs" : 1477663,
"source.11.logReadInBytes" : 662562515,
"sink.appliedHFiles" : 3,
"source.11.sizeOfHFileRefsQueue" : 0,
"source.logEditsFiltered" : 1276,
"source.shippedBytes" : 1513127672,
"source.11-host-10-19-92-187,21302,1465787242095.shippedBatches" : 0,
"source.11.shippedBytes" : 1513127672,
"sink.appliedOps" : 5,
"source.11-host-10-19-92-187,21302,1465787242095.sizeOfLogQueue" : 4,
"source.11.shippedBatches" : 30,
"source.11-host-10-19-92-187,21302,1465787242095.sizeOfHFileRefsQueue" : 0,
"source.11.timeWillBeTakenForLogToReplicate" : 1,
"source.11-host-10-19-92-187,21302,1465787242095.logReadInBytes" : 28586141,
"source.11.shippedOps" : 1351,
"source.shippedBatches" : 30,
"source.11-host-10-19-92-187,21302,1465787242095.shippedKBs" : 0,
"source.sizeOfLogToReplicate" : 537,
"source.11.ageOfLastShippedOp" : 0,
"sink.appliedBatches" : 2,
"source.11.logEditsRead" : 1381,
"source.11-host-10-19-92-187,21302,1465787242095.shippedBytes" : 0
}
{noformat}
*source.11-host-10-19-92-187,21302,1465787242095* not available in CLI
--
This message was sent by Atlassian JIRA
[jira] [Commented] (HBASE-15783) AccessControlConstants#OP_ATTRIBUTE_ACL_STRATEGY_CELL_FIRST not used any more.
[ https://issues.apache.org/jira/browse/HBASE-15783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15341112#comment-15341112 ] Anoop Sam John commented on HBASE-15783: Ya ideally the deprecation has to be in whole of a major version and then remove. Am +1 for remove in 2.0 and deprecate in 1.4 > AccessControlConstants#OP_ATTRIBUTE_ACL_STRATEGY_CELL_FIRST not used any more. > -- > > Key: HBASE-15783 > URL: https://issues.apache.org/jira/browse/HBASE-15783 > Project: HBase > Issue Type: Bug >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0, 1.0.4, 1.4.0 > > Attachments: HBASE-15783.patch > > > This is based on a mail in the user list. > OP_ATTRIBUTE_ACL_STRATEGY_CELL_FIRST in AccessControlConstants is not used > any more in the code and AccessControlconstants is Public. We need to either > bring in this feature or remove the constant from the Public APi which may be > misleading. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13701) Consolidate SecureBulkLoadEndpoint into HBase core as default for bulk load
[ https://issues.apache.org/jira/browse/HBASE-13701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15341109#comment-15341109 ] Jerry He commented on HBASE-13701: -- Then for compatibility, both existing non-secure and secure services needs to be maintained on RSRpcServices, and both non-secure and secure protobuf messages. I would like to see cleanup. But it turns out to be mandated, I will do what is needed. > Consolidate SecureBulkLoadEndpoint into HBase core as default for bulk load > --- > > Key: HBASE-13701 > URL: https://issues.apache.org/jira/browse/HBASE-13701 > Project: HBase > Issue Type: Improvement >Reporter: Jerry He >Assignee: Jerry He > > HBASE-12052 makes SecureBulkLoadEndpoint work in a non-secure env to solve > HDFS permission issues. > We have encountered some of the permission issues and have to use this > SecureBulkLoadEndpoint to workaround issues. > We should probably consolidate SecureBulkLoadEndpoint into HBase core as > default for bulk load since it is able to handle both secure Kerberos and > non-secure cases. > Maintaining two versions of bulk load implementation is also a cause of > confusion, and having to explicitly set it is also inconvenient. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15783) AccessControlConstants#OP_ATTRIBUTE_ACL_STRATEGY_CELL_FIRST not used any more.
[ https://issues.apache.org/jira/browse/HBASE-15783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15341106#comment-15341106 ] ramkrishna.s.vasudevan commented on HBASE-15783: No refs in book. So no doc updation needed. bq.This breaks src compatibility. But ya users must change/remove code if compiled with new version. So we will only deprecate in branch-1 ? > AccessControlConstants#OP_ATTRIBUTE_ACL_STRATEGY_CELL_FIRST not used any more. > -- > > Key: HBASE-15783 > URL: https://issues.apache.org/jira/browse/HBASE-15783 > Project: HBase > Issue Type: Bug >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0, 1.0.4, 1.4.0 > > Attachments: HBASE-15783.patch > > > This is based on a mail in the user list. > OP_ATTRIBUTE_ACL_STRATEGY_CELL_FIRST in AccessControlConstants is not used > any more in the code and AccessControlconstants is Public. We need to either > bring in this feature or remove the constant from the Public APi which may be > misleading. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15783) AccessControlConstants#OP_ATTRIBUTE_ACL_STRATEGY_CELL_FIRST not used any more.
[ https://issues.apache.org/jira/browse/HBASE-15783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15341102#comment-15341102 ] Anoop Sam John commented on HBASE-15783: We can keep it commit in 2.0 only? This breaks src compatibility. But ya users must change/remove code if compiled with new version. Any ref for these in book? > AccessControlConstants#OP_ATTRIBUTE_ACL_STRATEGY_CELL_FIRST not used any more. > -- > > Key: HBASE-15783 > URL: https://issues.apache.org/jira/browse/HBASE-15783 > Project: HBase > Issue Type: Bug >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0, 1.0.4, 1.4.0 > > Attachments: HBASE-15783.patch > > > This is based on a mail in the user list. > OP_ATTRIBUTE_ACL_STRATEGY_CELL_FIRST in AccessControlConstants is not used > any more in the code and AccessControlconstants is Public. We need to either > bring in this feature or remove the constant from the Public APi which may be > misleading. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16055) PutSortReducer loses any Visibility/acl attribute set on the Puts
[ https://issues.apache.org/jira/browse/HBASE-16055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15341096#comment-15341096 ] ramkrishna.s.vasudevan commented on HBASE-16055: The test report looks clean with the new test case added also passing. Good to commit in all the affected branches? > PutSortReducer loses any Visibility/acl attribute set on the Puts > -- > > Key: HBASE-16055 > URL: https://issues.apache.org/jira/browse/HBASE-16055 > Project: HBase > Issue Type: Bug > Components: security >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 2.0.0 > > Attachments: HBASE-16055_1.patch > > > Based on a user discussion, I think as the user pointed out rightly, when a > PutSortReducer is used any visibility attribute or external attribute set on > the Put will be lost as we create KVs out of the cells in the puts whereas > the ACL and visibility are all set as Attributes. > In TextSortReducer we tend to read the information we tend to read the > information from the parsed line but here in PutSortReducer we don't do it. I > think this problem should be in all the existing versions where we support > Tags. Correct me if am wrong here. > [~anoop.hbase], [~andrew.purt...@gmail.com]? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15783) AccessControlConstants#OP_ATTRIBUTE_ACL_STRATEGY_CELL_FIRST not used any more.
[ https://issues.apache.org/jira/browse/HBASE-15783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15341091#comment-15341091 ] ramkrishna.s.vasudevan commented on HBASE-15783: Will commit this unless objections. > AccessControlConstants#OP_ATTRIBUTE_ACL_STRATEGY_CELL_FIRST not used any more. > -- > > Key: HBASE-15783 > URL: https://issues.apache.org/jira/browse/HBASE-15783 > Project: HBase > Issue Type: Bug >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0, 1.0.4, 1.4.0 > > Attachments: HBASE-15783.patch > > > This is based on a mail in the user list. > OP_ATTRIBUTE_ACL_STRATEGY_CELL_FIRST in AccessControlConstants is not used > any more in the code and AccessControlconstants is Public. We need to either > bring in this feature or remove the constant from the Public APi which may be > misleading. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16070) Mapreduce Serialization classes do not Interface audience
[ https://issues.apache.org/jira/browse/HBASE-16070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15341089#comment-15341089 ] ramkrishna.s.vasudevan commented on HBASE-16070: Will commit this unless objections to trunk and branch-1. > Mapreduce Serialization classes do not Interface audience > - > > Key: HBASE-16070 > URL: https://issues.apache.org/jira/browse/HBASE-16070 > Project: HBase > Issue Type: Bug >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0 > > Attachments: HBASE-16070.patch > > > KeyValueSerilization, ResultSerialization and MutationSerialization do not > Interface audience. They are exposed interfaces and should be Public if am > not wrong. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16076) Cannot configure split policy in HBase shell
Youngjoon Kim created HBASE-16076:
-
Summary: Cannot configure split policy in HBase shell
Key: HBASE-16076
URL: https://issues.apache.org/jira/browse/HBASE-16076
Project: HBase
Issue Type: Bug
Components: shell
Affects Versions: 2.0.0
Reporter: Youngjoon Kim
Priority: Minor
The reference guide explains how to configure split policy in HBase
shell([link|http://hbase.apache.org/book.html#_custom_split_policies]).
{noformat}
Configuring the Split Policy On a Table Using HBase Shell
hbase> create 'test', {METHOD => 'table_att', CONFIG => {'SPLIT_POLICY' =>
'org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy'}},
{NAME => 'cf1'}
{noformat}
But if run that command, shell complains 'An argument ignored (unknown or
overridden): CONFIG', and the table description has no split policy.
{noformat}
hbase(main):067:0* create 'test', {METHOD => 'table_att', CONFIG =>
{'SPLIT_POLICY' =>
'org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy'}}, {NAME
=> 'cf1'}
An argument ignored (unknown or overridden): CONFIG
Created table test
Took 1.2180 seconds
hbase(main):068:0> describe 'test'
Table test is ENABLED
test
COLUMN FAMILIES DESCRIPTION
{NAME => 'cf1', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW',
REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1', TTL =>
'FOREVER', MIN_VERSIONS => '0', IN_MEMORY_COMPACTION => 'false',
KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE => '65536', IN_MEMORY => '
false', BLOCKCACHE => 'true'}
1 row(s)
Took 0.0200 seconds
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15870) Specify columns in REST multi gets
[ https://issues.apache.org/jira/browse/HBASE-15870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry He updated HBASE-15870: - Issue Type: Improvement (was: New Feature) > Specify columns in REST multi gets > -- > > Key: HBASE-15870 > URL: https://issues.apache.org/jira/browse/HBASE-15870 > Project: HBase > Issue Type: Improvement > Components: REST >Reporter: Dean Gurvitz >Assignee: Matt Warhaftig >Priority: Minor > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2 > > Attachments: hbase-15870-v1.patch > > > The REST multi-gets feature currently does not allow specifying only certain > columns or column families. Adding support for these should be quite simple > and improve the usability of the multi-gets feature. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15870) Specify columns in REST multi gets
[ https://issues.apache.org/jira/browse/HBASE-15870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15341057#comment-15341057 ] Jerry He commented on HBASE-15870: -- Ok. I can revert it from branch-1.2 I was testing it on my 1.2 cluster actually. I have people inquiring about it and our platform is on 1.2. Alternatively, I can make the JIRA as 'improvement', and it would be a good little improvement. It's your call [~busbey]. > Specify columns in REST multi gets > -- > > Key: HBASE-15870 > URL: https://issues.apache.org/jira/browse/HBASE-15870 > Project: HBase > Issue Type: New Feature > Components: REST >Reporter: Dean Gurvitz >Assignee: Matt Warhaftig >Priority: Minor > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2 > > Attachments: hbase-15870-v1.patch > > > The REST multi-gets feature currently does not allow specifying only certain > columns or column families. Adding support for these should be quite simple > and improve the usability of the multi-gets feature. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16035) Nested AutoCloseables might not all get closed
[
https://issues.apache.org/jira/browse/HBASE-16035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15341028#comment-15341028
]
Hudson commented on HBASE-16035:
FAILURE: Integrated in HBase-Trunk_matrix #1085 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/1085/])
HBASE-16035 Nested AutoCloseables might not all get closed (Sean (tedyu: rev
3de0c631050ea7dfb72fa8e1430aa8e7c0e87088)
* hbase-server/src/main/java/org/apache/hadoop/hbase/http/log/LogLevel.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/JarFinder.java
> Nested AutoCloseables might not all get closed
> --
>
> Key: HBASE-16035
> URL: https://issues.apache.org/jira/browse/HBASE-16035
> Project: HBase
> Issue Type: Bug
>Reporter: Sean Mackrory
> Fix For: 2.0.0
>
> Attachments: HBASE-16035-v1.patch
>
>
> Subtle problem in HBASE-15891:
> {code}try (A myA = new A(new B())){code}
> An exception thrown between B starting to open an A finishing initialization
> may not result in B being closed. A safer syntax would be:
> {code}try(B myB = new B(); A myA = newA(myB)){code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15870) Specify columns in REST multi gets
[ https://issues.apache.org/jira/browse/HBASE-15870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340977#comment-15340977 ] Hudson commented on HBASE-15870: FAILURE: Integrated in HBase-1.3 #750 (See [https://builds.apache.org/job/HBase-1.3/750/]) HBASE-15870 Specify columns in REST multi gets (Matt Warhaftig) (jerryjch: rev d91e42b8407dd68dce702c76fb4843326c8e38e4) * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/MultiRowResource.java * hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/TestMultiRowResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/TableResource.java > Specify columns in REST multi gets > -- > > Key: HBASE-15870 > URL: https://issues.apache.org/jira/browse/HBASE-15870 > Project: HBase > Issue Type: New Feature > Components: REST >Reporter: Dean Gurvitz >Assignee: Matt Warhaftig >Priority: Minor > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2 > > Attachments: hbase-15870-v1.patch > > > The REST multi-gets feature currently does not allow specifying only certain > columns or column families. Adding support for these should be quite simple > and improve the usability of the multi-gets feature. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16067) Add READ_COMMITTED IsolationLevel for scan and the current is SNAPSHOT.
[
https://issues.apache.org/jira/browse/HBASE-16067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340974#comment-15340974
]
Heng Chen commented on HBASE-16067:
---
The idea sounds reasonable.
{code}
+ storeHeap.updateMvccReadPoint(readPt);
+ if (joinedHeap != null) {
+joinedHeap.updateMvccReadPoint(readPt);
+ }
{code}
Will readPt be updated between above two update operations?
And should we move "if (isolationLevel == IsolationLevel.READ_COMMITTED)" check
before we set needUpdateReadPt to be true?
> Add READ_COMMITTED IsolationLevel for scan and the current is SNAPSHOT.
> ---
>
> Key: HBASE-16067
> URL: https://issues.apache.org/jira/browse/HBASE-16067
> Project: HBase
> Issue Type: New Feature
>Reporter: binlijin
> Attachments: HBASE-16067.patch
>
>
> When scan a region, openScan get a mvcc readpoint and do not advance the
> readpoint within it lifetime, so all committed write after the readpoint do
> not visible for the scan.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16036) Fix ReplicationTableBase initialization to make it non-blocking
[ https://issues.apache.org/jira/browse/HBASE-16036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340926#comment-15340926 ] Ted Yu commented on HBASE-16036: +1 > Fix ReplicationTableBase initialization to make it non-blocking > --- > > Key: HBASE-16036 > URL: https://issues.apache.org/jira/browse/HBASE-16036 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > Attachments: HBASE-16036.patch > > > Currently there is a bug inside of TableBasedReplicationQueuesImpl > construction that prevents ReplicationServices from starting before Master is > initialized. So currently each of the RS, including HMaster, with Replication > enabled will attempt to create the ReplicationTable on initialization. > Currently HMaster's initialization: serviceThreads.start() -> new > TableBasedReplicationQueuesImpl() -> Replication Table Creation -> HMaster > sets initialized flags. > But this fails when we try to create the Replication Table as the > HMaster.checkInitialized() flag fails. This ends up blocking HMaster > initialization and results in a deadlock. > So in this patch, I will create the Replication Table in the background of > TableBasedReplicationQueuesImpl and only block when we actually call methods > that access it. > This also requires a small refactoring of ReplicationSourceManager.init() so > that we run the abandoned queue adoption in the background > Review board at: https://reviews.apache.org/r/48763/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16036) Fix ReplicationTableBase initialization to make it non-blocking
[
https://issues.apache.org/jira/browse/HBASE-16036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340917#comment-15340917
]
Hadoop QA commented on HBASE-16036:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m 25s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
27s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 13s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 55s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
32s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
12s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 58s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
9s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 13s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 13s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 54s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 54s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
53s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 25s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
35s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 8s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 57s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 2s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 86m 56s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
31s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 142m 31s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12812047/HBASE-16036.patch |
| JIRA Issue | HBASE-16036 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
|
[jira] [Commented] (HBASE-15870) Specify columns in REST multi gets
[ https://issues.apache.org/jira/browse/HBASE-15870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340915#comment-15340915 ] Hudson commented on HBASE-15870: SUCCESS: Integrated in HBase-1.2 #655 (See [https://builds.apache.org/job/HBase-1.2/655/]) HBASE-15870 Specify columns in REST multi gets (Matt Warhaftig) (jerryjch: rev 910726e1a71514ceca7499e0fdd6ce32ca4a5e6c) * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/MultiRowResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/TableResource.java * hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/TestMultiRowResource.java > Specify columns in REST multi gets > -- > > Key: HBASE-15870 > URL: https://issues.apache.org/jira/browse/HBASE-15870 > Project: HBase > Issue Type: New Feature > Components: REST >Reporter: Dean Gurvitz >Assignee: Matt Warhaftig >Priority: Minor > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2 > > Attachments: hbase-15870-v1.patch > > > The REST multi-gets feature currently does not allow specifying only certain > columns or column families. Adding support for these should be quite simple > and improve the usability of the multi-gets feature. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15870) Specify columns in REST multi gets
[ https://issues.apache.org/jira/browse/HBASE-15870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340904#comment-15340904 ] Hudson commented on HBASE-15870: FAILURE: Integrated in HBase-1.4 #230 (See [https://builds.apache.org/job/HBase-1.4/230/]) HBASE-15870 Specify columns in REST multi gets (Matt Warhaftig) (jerryjch: rev 541b9ff25c24cf566b45c7b3c054b82e509d2ac6) * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/TableResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/MultiRowResource.java * hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/TestMultiRowResource.java > Specify columns in REST multi gets > -- > > Key: HBASE-15870 > URL: https://issues.apache.org/jira/browse/HBASE-15870 > Project: HBase > Issue Type: New Feature > Components: REST >Reporter: Dean Gurvitz >Assignee: Matt Warhaftig >Priority: Minor > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2 > > Attachments: hbase-15870-v1.patch > > > The REST multi-gets feature currently does not allow specifying only certain > columns or column families. Adding support for these should be quite simple > and improve the usability of the multi-gets feature. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16075) TestAcidGuarantees is flaky
Mikhail Antonov created HBASE-16075: --- Summary: TestAcidGuarantees is flaky Key: HBASE-16075 URL: https://issues.apache.org/jira/browse/HBASE-16075 Project: HBase Issue Type: Bug Components: test Affects Versions: 1.3.0 Reporter: Mikhail Antonov https://builds.apache.org/job/HBase-1.3/744/jdk=latest1.7,label=yahoo-not-h2/testReport/junit/TEST-org.apache.hadoop.hbase.TestAcidGuarantees/xml/_init_/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16049) TestRowProcessorEndpoint is failing on Apache Builds
[ https://issues.apache.org/jira/browse/HBASE-16049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-16049: Affects Version/s: 1.4.0 1.3.0 > TestRowProcessorEndpoint is failing on Apache Builds > > > Key: HBASE-16049 > URL: https://issues.apache.org/jira/browse/HBASE-16049 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.0, 1.4.0 >Reporter: Mikhail Antonov > > example log > https://paste.apache.org/46Uh -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16012) Major compaction can't work because left scanner read point in RegionServer
[ https://issues.apache.org/jira/browse/HBASE-16012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340863#comment-15340863 ] Guanghao Zhang commented on HBASE-16012: Yeah, the exception is thrown when initialize region scanner, so it didn't create lease for this scanner. > Major compaction can't work because left scanner read point in RegionServer > --- > > Key: HBASE-16012 > URL: https://issues.apache.org/jira/browse/HBASE-16012 > Project: HBase > Issue Type: Bug > Components: Compaction, Scanners >Affects Versions: 2.0.0, 0.94.27, 1.1.6, 1.3.1, 0.98.21, 1.2.3 >Reporter: Guanghao Zhang > Fix For: 2.0.0 > > Attachments: HBASE-16012-v1.patch, HBASE-16012-v2.patch, > HBASE-16012-v3.patch, HBASE-16012-v4.patch, HBASE-16012.patch > > > When new RegionScanner, it will add a scanner read point in > scannerReadPoints. But if we got a exception after add read point, the read > point will keep in regions server and the delete after this mvcc number will > never be compacted. > Our hbase version is base 0.94. If it throws other exception when initialize > RegionScanner, the master branch has this bug, too. > ERROR org.apache.hadoop.hbase.regionserver.HRegionServer: Failed openScanner > java.io.IOException: Could not seek StoreFileScanner > at > org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:160) > at > org.apache.hadoop.hbase.regionserver.StoreScanner.seekScanners(StoreScanner.java:268) > at > org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:168) > at org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:2232) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.(HRegion.java:4026) > at > org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1895) > at > org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1879) > at > org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1854) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.internalOpenScanner(HRegionServer.java:3032) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:2995) > at sun.reflect.GeneratedMethodAccessor67.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.hadoop.hbase.ipc.SecureRpcEngine$Server.call(SecureRpcEngine.java:338) > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1595) > Caused by: org.apache.hadoop.hbase.ipc.CallerDisconnectedException: Aborting > call openScanner, since caller disconnected > at > org.apache.hadoop.hbase.ipc.HBaseServer$Call.throwExceptionIfCallerDisconnected(HBaseServer.java:475) > at > org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1443) > at > org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockDataInternal(HFileBlock.java:1902) > at > org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1766) > at > org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:345) > at > org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.loadDataBlockWithScanInfo(HFileBlockIndex.java:254) > at > org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:499) > at > org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:520) > at > org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:235) > at > org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:148) > ... 14 more -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16012) Major compaction can't work because left scanner read point in RegionServer
[ https://issues.apache.org/jira/browse/HBASE-16012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340855#comment-15340855 ] Duo Zhang commented on HBASE-16012: --- [~lhofhansl] The exception is thrown inside the constructor. We haven't setup a lease yet at that time... And yes, there are some design patterns say that you should not put much code in the constructor, especially if it throws exceptions, instead, use an 'init' method. But this is not the target of this issue I think. Thanks. > Major compaction can't work because left scanner read point in RegionServer > --- > > Key: HBASE-16012 > URL: https://issues.apache.org/jira/browse/HBASE-16012 > Project: HBase > Issue Type: Bug > Components: Compaction, Scanners >Affects Versions: 2.0.0, 0.94.27, 1.1.6, 1.3.1, 0.98.21, 1.2.3 >Reporter: Guanghao Zhang > Fix For: 2.0.0 > > Attachments: HBASE-16012-v1.patch, HBASE-16012-v2.patch, > HBASE-16012-v3.patch, HBASE-16012-v4.patch, HBASE-16012.patch > > > When new RegionScanner, it will add a scanner read point in > scannerReadPoints. But if we got a exception after add read point, the read > point will keep in regions server and the delete after this mvcc number will > never be compacted. > Our hbase version is base 0.94. If it throws other exception when initialize > RegionScanner, the master branch has this bug, too. > ERROR org.apache.hadoop.hbase.regionserver.HRegionServer: Failed openScanner > java.io.IOException: Could not seek StoreFileScanner > at > org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:160) > at > org.apache.hadoop.hbase.regionserver.StoreScanner.seekScanners(StoreScanner.java:268) > at > org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:168) > at org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:2232) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.(HRegion.java:4026) > at > org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1895) > at > org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1879) > at > org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1854) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.internalOpenScanner(HRegionServer.java:3032) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:2995) > at sun.reflect.GeneratedMethodAccessor67.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.hadoop.hbase.ipc.SecureRpcEngine$Server.call(SecureRpcEngine.java:338) > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1595) > Caused by: org.apache.hadoop.hbase.ipc.CallerDisconnectedException: Aborting > call openScanner, since caller disconnected > at > org.apache.hadoop.hbase.ipc.HBaseServer$Call.throwExceptionIfCallerDisconnected(HBaseServer.java:475) > at > org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1443) > at > org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockDataInternal(HFileBlock.java:1902) > at > org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1766) > at > org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:345) > at > org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.loadDataBlockWithScanInfo(HFileBlockIndex.java:254) > at > org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:499) > at > org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:520) > at > org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:235) > at > org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:148) > ... 14 more -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16012) Major compaction can't work because left scanner read point in RegionServer
[ https://issues.apache.org/jira/browse/HBASE-16012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340782#comment-15340782 ] Lars Hofhansl commented on HBASE-16012: --- Why is this not handled when the scanner's lease expires? Are we saying the exception occurs before we setup the lease? > Major compaction can't work because left scanner read point in RegionServer > --- > > Key: HBASE-16012 > URL: https://issues.apache.org/jira/browse/HBASE-16012 > Project: HBase > Issue Type: Bug > Components: Compaction, Scanners >Affects Versions: 2.0.0, 0.94.27, 1.1.6, 1.3.1, 0.98.21, 1.2.3 >Reporter: Guanghao Zhang > Fix For: 2.0.0 > > Attachments: HBASE-16012-v1.patch, HBASE-16012-v2.patch, > HBASE-16012-v3.patch, HBASE-16012-v4.patch, HBASE-16012.patch > > > When new RegionScanner, it will add a scanner read point in > scannerReadPoints. But if we got a exception after add read point, the read > point will keep in regions server and the delete after this mvcc number will > never be compacted. > Our hbase version is base 0.94. If it throws other exception when initialize > RegionScanner, the master branch has this bug, too. > ERROR org.apache.hadoop.hbase.regionserver.HRegionServer: Failed openScanner > java.io.IOException: Could not seek StoreFileScanner > at > org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:160) > at > org.apache.hadoop.hbase.regionserver.StoreScanner.seekScanners(StoreScanner.java:268) > at > org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:168) > at org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:2232) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.(HRegion.java:4026) > at > org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1895) > at > org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1879) > at > org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1854) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.internalOpenScanner(HRegionServer.java:3032) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:2995) > at sun.reflect.GeneratedMethodAccessor67.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.hadoop.hbase.ipc.SecureRpcEngine$Server.call(SecureRpcEngine.java:338) > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1595) > Caused by: org.apache.hadoop.hbase.ipc.CallerDisconnectedException: Aborting > call openScanner, since caller disconnected > at > org.apache.hadoop.hbase.ipc.HBaseServer$Call.throwExceptionIfCallerDisconnected(HBaseServer.java:475) > at > org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1443) > at > org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockDataInternal(HFileBlock.java:1902) > at > org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1766) > at > org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:345) > at > org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.loadDataBlockWithScanInfo(HFileBlockIndex.java:254) > at > org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:499) > at > org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:520) > at > org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:235) > at > org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:148) > ... 14 more -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14644) Region in transition metric is broken
[ https://issues.apache.org/jira/browse/HBASE-14644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340753#comment-15340753 ] Hudson commented on HBASE-14644: FAILURE: Integrated in HBase-Trunk_matrix #1084 (See [https://builds.apache.org/job/HBase-Trunk_matrix/1084/]) HBASE-14644 Region in transition metric is broken -- addendum (Huaxiang (matteo.bertozzi: rev 313a3d23a8bd815a7197a7225b7e7fe9614e2bbc) * hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java > Region in transition metric is broken > - > > Key: HBASE-14644 > URL: https://issues.apache.org/jira/browse/HBASE-14644 > Project: HBase > Issue Type: Bug >Reporter: Elliott Clark >Assignee: huaxiang sun > Fix For: 2.0.0, 1.3.0, 1.2.2 > > Attachments: HBASE-14644-v001.patch, HBASE-14644-v002-addendum.patch, > HBASE-14644-v002.patch, HBASE-14644-v002.patch, branch-1.diff > > > ritCount stays 0 no matter what -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16059) Region normalizer fails to trigger merge action where one of the regions is empty
[
https://issues.apache.org/jira/browse/HBASE-16059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340754#comment-15340754
]
Hudson commented on HBASE-16059:
FAILURE: Integrated in HBase-Trunk_matrix #1084 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/1084/])
HBASE-16059 Region normalizer fails to trigger merge action where one of
(tedyu: rev be56b5ae7119bc58d4c755974f89f5eabc70a43e)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SimpleRegionNormalizer.java
> Region normalizer fails to trigger merge action where one of the regions is
> empty
> -
>
> Key: HBASE-16059
> URL: https://issues.apache.org/jira/browse/HBASE-16059
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Romil Choksi
>Assignee: Ted Yu
> Labels: normalization
> Fix For: 2.0.0, 1.4.0
>
> Attachments: 16059.v1.txt
>
>
> Region normalizer failed to trigger merge action where expected
> Steps to reproduce:
> - Pre-split the test table into 5 regions with keys 1,3,7,8
> - Insert some data for each of the split. 27K rows for regions starting with
> key 1, and 100K rows for each of the regions with start key 3,7 and 8
> - Scan the test table, and verify that these regions exists - 1) STARTKEY =>
> ‘' ENDKEY => ’1’ 2) STARTKEY => ’1’ ENDKEY => ’3’
> - Turn on normalization, verify normalization switch is enabled and that
> normalization is true for test table
> - Run normalizer a few times
> - Scan test table again, verify that regions don’t exist anymore 1) STARTKEY
> => ‘' ENDKEY => ’1’ 2) STARTKEY => ’1’ ENDKEY => ’3’, but instead a new
> region is created with STARTKEY => ’’ ENDKEY => ’3’
> The test now fails, with the last step failing at assertion.
> Looking into the Master log, I see that normalization plan was computed for
> the test table but it decides that no normalization is needed for the test
> table, and that the regions look good.
> {code:title = Master.log}
> 2016-06-17 00:41:46,895 DEBUG
> [B.defaultRpcServer.handler=4,queue=1,port=2]
> normalizer.SimpleRegionNormalizer: Computing normalization plan for table:
> table_zrof6ea383, number of regions: 5
> 2016-06-17 00:41:46,895 DEBUG
> [B.defaultRpcServer.handler=4,queue=1,port=2]
> normalizer.SimpleRegionNormalizer: Table table_zrof6ea383, total aggregated
> regions size: 13
> 2016-06-17 00:41:46,896 DEBUG
> [B.defaultRpcServer.handler=4,queue=1,port=2]
> normalizer.SimpleRegionNormalizer: Table table_zrof6ea383, average region
> size: 2.6
> 2016-06-17 00:41:46,896 DEBUG
> [B.defaultRpcServer.handler=4,queue=1,port=2]
> normalizer.SimpleRegionNormalizer: No normalization needed, regions look good
> for table: table_zrof6ea383
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15870) Specify columns in REST multi gets
[ https://issues.apache.org/jira/browse/HBASE-15870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340755#comment-15340755 ] Hudson commented on HBASE-15870: FAILURE: Integrated in HBase-Trunk_matrix #1084 (See [https://builds.apache.org/job/HBase-Trunk_matrix/1084/]) HBASE-15870 Specify columns in REST multi gets (Matt Warhaftig) (jerryjch: rev 4ca4cd856ef5b4596aa43c048e2a8478d08dc773) * hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/TestMultiRowResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/TableResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/MultiRowResource.java > Specify columns in REST multi gets > -- > > Key: HBASE-15870 > URL: https://issues.apache.org/jira/browse/HBASE-15870 > Project: HBase > Issue Type: New Feature > Components: REST >Reporter: Dean Gurvitz >Assignee: Matt Warhaftig >Priority: Minor > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2 > > Attachments: hbase-15870-v1.patch > > > The REST multi-gets feature currently does not allow specifying only certain > columns or column families. Adding support for these should be quite simple > and improve the usability of the multi-gets feature. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16030) All Regions are flushed at about same time when MEMSTORE_PERIODIC_FLUSH is on, causing flush spike
[ https://issues.apache.org/jira/browse/HBASE-16030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340748#comment-15340748 ] Enis Soztutar commented on HBASE-16030: --- bq. It sounds also a problem if flush is stalled for 5 mins (although not as bad as 30 mins) Indeed. bq. Feels like the right way of jittering should not be putting into a blocking queue ahead of time. What do you think? I think we can still jitter for more, but we have to solve the underlying problem first. The correct solution might be to re-queue the request in regular flush request with no-delay. > All Regions are flushed at about same time when MEMSTORE_PERIODIC_FLUSH is > on, causing flush spike > -- > > Key: HBASE-16030 > URL: https://issues.apache.org/jira/browse/HBASE-16030 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.2.1 >Reporter: Tianying Chang >Assignee: Tianying Chang > Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.3 > > Attachments: Screen Shot 2016-06-15 at 11.35.42 PM.png, Screen Shot > 2016-06-15 at 11.52.38 PM.png, hbase-16030-v2.patch, hbase-16030.patch > > > In our production cluster, we observed that memstore flush spike every hour > for all regions/RS. (we use the default memstore periodic flush time of 1 > hour). > This will happend when two conditions are met: > 1. the memstore does not have enough data to be flushed before 1 hour limit > reached; > 2. all regions are opened around the same time, (e.g. all RS are started at > the same time when start a cluster). > With above two conditions, all the regions will be flushed around the same > time at: startTime+1hour-delay again and again. > We added a flush jittering time to randomize the flush time of each region, > so that they don't get flushed at around the same time. We had this feature > running in our 94.7 and 94.26 cluster. Recently, we upgrade to 1.2, found > this issue still there in 1.2. So we are porting this into 1.2 branch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15870) Specify columns in REST multi gets
[ https://issues.apache.org/jira/browse/HBASE-15870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340733#comment-15340733 ] Hudson commented on HBASE-15870: SUCCESS: Integrated in HBase-1.2-IT #536 (See [https://builds.apache.org/job/HBase-1.2-IT/536/]) HBASE-15870 Specify columns in REST multi gets (Matt Warhaftig) (jerryjch: rev 910726e1a71514ceca7499e0fdd6ce32ca4a5e6c) * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/MultiRowResource.java * hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/TestMultiRowResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/TableResource.java > Specify columns in REST multi gets > -- > > Key: HBASE-15870 > URL: https://issues.apache.org/jira/browse/HBASE-15870 > Project: HBase > Issue Type: New Feature > Components: REST >Reporter: Dean Gurvitz >Assignee: Matt Warhaftig >Priority: Minor > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2 > > Attachments: hbase-15870-v1.patch > > > The REST multi-gets feature currently does not allow specifying only certain > columns or column families. Adding support for these should be quite simple > and improve the usability of the multi-gets feature. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16074) ITBLL fails, reports lost big or tine families
[
https://issues.apache.org/jira/browse/HBASE-16074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340735#comment-15340735
]
Elliott Clark commented on HBASE-16074:
---
{code}Loop
2
5000
/tmp/itbll/
5
1000
-m
slowDeterministic
{code}
We're seeing the REFERENCED equal to the num machines * 5000 * 2. However
the LOST_FAMILIES is greater than 0. So there is some data that's coming in
without one of the other column families.
Setting the config to use only one column family would probably make this work,
however it seems to suggest something is wrong with scanner batching ( again ).
> ITBLL fails, reports lost big or tine families
> --
>
> Key: HBASE-16074
> URL: https://issues.apache.org/jira/browse/HBASE-16074
> Project: HBase
> Issue Type: Bug
> Components: integration tests
>Affects Versions: 1.3.0
>Reporter: Mikhail Antonov
>Assignee: Mikhail Antonov
>Priority: Blocker
> Fix For: 1.3.0
>
>
> Underlying MR jobs succeed but I'm seeing the following in the logs (mid-size
> distributed test cluster):
> ERROR test.IntegrationTestBigLinkedList$Verify: Found nodes which lost big or
> tiny families, count=164
> I do not know exactly yet whether it's a bug, a test issue or env setup
> issue, but need figure it out. Opening this to raise awareness and see if
> someone saw that recently.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16036) Fix ReplicationTableBase initialization to make it non-blocking
[ https://issues.apache.org/jira/browse/HBASE-16036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-16036: --- Status: Patch Available (was: Open) > Fix ReplicationTableBase initialization to make it non-blocking > --- > > Key: HBASE-16036 > URL: https://issues.apache.org/jira/browse/HBASE-16036 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > Attachments: HBASE-16036.patch > > > Currently there is a bug inside of TableBasedReplicationQueuesImpl > construction that prevents ReplicationServices from starting before Master is > initialized. So currently each of the RS, including HMaster, with Replication > enabled will attempt to create the ReplicationTable on initialization. > Currently HMaster's initialization: serviceThreads.start() -> new > TableBasedReplicationQueuesImpl() -> Replication Table Creation -> HMaster > sets initialized flags. > But this fails when we try to create the Replication Table as the > HMaster.checkInitialized() flag fails. This ends up blocking HMaster > initialization and results in a deadlock. > So in this patch, I will create the Replication Table in the background of > TableBasedReplicationQueuesImpl and only block when we actually call methods > that access it. > This also requires a small refactoring of ReplicationSourceManager.init() so > that we run the abandoned queue adoption in the background > Review board at: https://reviews.apache.org/r/48763/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16036) Fix ReplicationTableBase initialization to make it non-blocking
[ https://issues.apache.org/jira/browse/HBASE-16036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-16036: --- Status: Open (was: Patch Available) > Fix ReplicationTableBase initialization to make it non-blocking > --- > > Key: HBASE-16036 > URL: https://issues.apache.org/jira/browse/HBASE-16036 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > Attachments: HBASE-16036.patch > > > Currently there is a bug inside of TableBasedReplicationQueuesImpl > construction that prevents ReplicationServices from starting before Master is > initialized. So currently each of the RS, including HMaster, with Replication > enabled will attempt to create the ReplicationTable on initialization. > Currently HMaster's initialization: serviceThreads.start() -> new > TableBasedReplicationQueuesImpl() -> Replication Table Creation -> HMaster > sets initialized flags. > But this fails when we try to create the Replication Table as the > HMaster.checkInitialized() flag fails. This ends up blocking HMaster > initialization and results in a deadlock. > So in this patch, I will create the Replication Table in the background of > TableBasedReplicationQueuesImpl and only block when we actually call methods > that access it. > This also requires a small refactoring of ReplicationSourceManager.init() so > that we run the abandoned queue adoption in the background > Review board at: https://reviews.apache.org/r/48763/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15870) Specify columns in REST multi gets
[ https://issues.apache.org/jira/browse/HBASE-15870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340723#comment-15340723 ] Sean Busbey commented on HBASE-15870: - As a new feature, this shouldn't be in branch-1.2 > Specify columns in REST multi gets > -- > > Key: HBASE-15870 > URL: https://issues.apache.org/jira/browse/HBASE-15870 > Project: HBase > Issue Type: New Feature > Components: REST >Reporter: Dean Gurvitz >Assignee: Matt Warhaftig >Priority: Minor > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2 > > Attachments: hbase-15870-v1.patch > > > The REST multi-gets feature currently does not allow specifying only certain > columns or column families. Adding support for these should be quite simple > and improve the usability of the multi-gets feature. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16036) Fix ReplicationTableBase initialization to make it non-blocking
[ https://issues.apache.org/jira/browse/HBASE-16036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-16036: --- Attachment: (was: HBASE-16036.patch) > Fix ReplicationTableBase initialization to make it non-blocking > --- > > Key: HBASE-16036 > URL: https://issues.apache.org/jira/browse/HBASE-16036 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > Attachments: HBASE-16036.patch > > > Currently there is a bug inside of TableBasedReplicationQueuesImpl > construction that prevents ReplicationServices from starting before Master is > initialized. So currently each of the RS, including HMaster, with Replication > enabled will attempt to create the ReplicationTable on initialization. > Currently HMaster's initialization: serviceThreads.start() -> new > TableBasedReplicationQueuesImpl() -> Replication Table Creation -> HMaster > sets initialized flags. > But this fails when we try to create the Replication Table as the > HMaster.checkInitialized() flag fails. This ends up blocking HMaster > initialization and results in a deadlock. > So in this patch, I will create the Replication Table in the background of > TableBasedReplicationQueuesImpl and only block when we actually call methods > that access it. > This also requires a small refactoring of ReplicationSourceManager.init() so > that we run the abandoned queue adoption in the background > Review board at: https://reviews.apache.org/r/48763/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16036) Fix ReplicationTableBase initialization to make it non-blocking
[ https://issues.apache.org/jira/browse/HBASE-16036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-16036: --- Attachment: HBASE-16036.patch > Fix ReplicationTableBase initialization to make it non-blocking > --- > > Key: HBASE-16036 > URL: https://issues.apache.org/jira/browse/HBASE-16036 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > Attachments: HBASE-16036.patch > > > Currently there is a bug inside of TableBasedReplicationQueuesImpl > construction that prevents ReplicationServices from starting before Master is > initialized. So currently each of the RS, including HMaster, with Replication > enabled will attempt to create the ReplicationTable on initialization. > Currently HMaster's initialization: serviceThreads.start() -> new > TableBasedReplicationQueuesImpl() -> Replication Table Creation -> HMaster > sets initialized flags. > But this fails when we try to create the Replication Table as the > HMaster.checkInitialized() flag fails. This ends up blocking HMaster > initialization and results in a deadlock. > So in this patch, I will create the Replication Table in the background of > TableBasedReplicationQueuesImpl and only block when we actually call methods > that access it. > This also requires a small refactoring of ReplicationSourceManager.init() so > that we run the abandoned queue adoption in the background > Review board at: https://reviews.apache.org/r/48763/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16030) All Regions are flushed at about same time when MEMSTORE_PERIODIC_FLUSH is on, causing flush spike
[ https://issues.apache.org/jira/browse/HBASE-16030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340691#comment-15340691 ] Tianying Chang commented on HBASE-16030: [~enis] good scenario! So with the previous hard code 5 mins delay, the flush WILL NOT happen for 5 mins. It sounds also a problem if flush is stalled for 5 mins (although not as bad as 30 mins) Feels like the right way of jittering should not be putting into a blocking queue ahead of time. What do you think? > All Regions are flushed at about same time when MEMSTORE_PERIODIC_FLUSH is > on, causing flush spike > -- > > Key: HBASE-16030 > URL: https://issues.apache.org/jira/browse/HBASE-16030 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.2.1 >Reporter: Tianying Chang >Assignee: Tianying Chang > Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.3 > > Attachments: Screen Shot 2016-06-15 at 11.35.42 PM.png, Screen Shot > 2016-06-15 at 11.52.38 PM.png, hbase-16030-v2.patch, hbase-16030.patch > > > In our production cluster, we observed that memstore flush spike every hour > for all regions/RS. (we use the default memstore periodic flush time of 1 > hour). > This will happend when two conditions are met: > 1. the memstore does not have enough data to be flushed before 1 hour limit > reached; > 2. all regions are opened around the same time, (e.g. all RS are started at > the same time when start a cluster). > With above two conditions, all the regions will be flushed around the same > time at: startTime+1hour-delay again and again. > We added a flush jittering time to randomize the flush time of each region, > so that they don't get flushed at around the same time. We had this feature > running in our 94.7 and 94.26 cluster. Recently, we upgrade to 1.2, found > this issue still there in 1.2. So we are porting this into 1.2 branch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16012) Major compaction can't work because left scanner read point in RegionServer
[ https://issues.apache.org/jira/browse/HBASE-16012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340683#comment-15340683 ] Ted Yu commented on HBASE-16012: v4 lgtm > Major compaction can't work because left scanner read point in RegionServer > --- > > Key: HBASE-16012 > URL: https://issues.apache.org/jira/browse/HBASE-16012 > Project: HBase > Issue Type: Bug > Components: Compaction, Scanners >Affects Versions: 2.0.0, 0.94.27, 1.1.6, 1.3.1, 0.98.21, 1.2.3 >Reporter: Guanghao Zhang > Fix For: 2.0.0 > > Attachments: HBASE-16012-v1.patch, HBASE-16012-v2.patch, > HBASE-16012-v3.patch, HBASE-16012-v4.patch, HBASE-16012.patch > > > When new RegionScanner, it will add a scanner read point in > scannerReadPoints. But if we got a exception after add read point, the read > point will keep in regions server and the delete after this mvcc number will > never be compacted. > Our hbase version is base 0.94. If it throws other exception when initialize > RegionScanner, the master branch has this bug, too. > ERROR org.apache.hadoop.hbase.regionserver.HRegionServer: Failed openScanner > java.io.IOException: Could not seek StoreFileScanner > at > org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:160) > at > org.apache.hadoop.hbase.regionserver.StoreScanner.seekScanners(StoreScanner.java:268) > at > org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:168) > at org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:2232) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.(HRegion.java:4026) > at > org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1895) > at > org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1879) > at > org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1854) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.internalOpenScanner(HRegionServer.java:3032) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:2995) > at sun.reflect.GeneratedMethodAccessor67.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.hadoop.hbase.ipc.SecureRpcEngine$Server.call(SecureRpcEngine.java:338) > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1595) > Caused by: org.apache.hadoop.hbase.ipc.CallerDisconnectedException: Aborting > call openScanner, since caller disconnected > at > org.apache.hadoop.hbase.ipc.HBaseServer$Call.throwExceptionIfCallerDisconnected(HBaseServer.java:475) > at > org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1443) > at > org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockDataInternal(HFileBlock.java:1902) > at > org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1766) > at > org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:345) > at > org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.loadDataBlockWithScanInfo(HFileBlockIndex.java:254) > at > org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:499) > at > org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:520) > at > org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:235) > at > org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:148) > ... 14 more -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16036) Fix ReplicationTableBase initialization to make it non-blocking
[
https://issues.apache.org/jira/browse/HBASE-16036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340667#comment-15340667
]
Hadoop QA commented on HBASE-16036:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 3m 11s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
5s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 47s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 13s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
24s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
18s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 8s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 21s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 14s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 12s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
28s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 38s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 38s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 16s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 16s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
5s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
47m 0s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
38s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 13s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 13s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 18s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 108m 49s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
21s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 198m 37s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests |
org.apache.hadoop.hbase.coprocessor.TestRegionObserverInterface |
| | org.apache.hadoop.hbase.backup.TestHFileArchiving |
| | org.apache.hadoop.hbase.snapshot.TestExportSnapshot |
| | org.apache.hadoop.hbase.regionserver.TestHRegionServerBulkLoad |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12811976/HBASE-16036.patch |
| JIRA Issue |
[jira] [Updated] (HBASE-16035) Nested AutoCloseables might not all get closed
[
https://issues.apache.org/jira/browse/HBASE-16035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-16035:
---
Hadoop Flags: Reviewed
Fix Version/s: 2.0.0
> Nested AutoCloseables might not all get closed
> --
>
> Key: HBASE-16035
> URL: https://issues.apache.org/jira/browse/HBASE-16035
> Project: HBase
> Issue Type: Bug
>Reporter: Sean Mackrory
> Fix For: 2.0.0
>
> Attachments: HBASE-16035-v1.patch
>
>
> Subtle problem in HBASE-15891:
> {code}try (A myA = new A(new B())){code}
> An exception thrown between B starting to open an A finishing initialization
> may not result in B being closed. A safer syntax would be:
> {code}try(B myB = new B(); A myA = newA(myB)){code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16035) Nested AutoCloseables might not all get closed
[
https://issues.apache.org/jira/browse/HBASE-16035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340665#comment-15340665
]
Ted Yu commented on HBASE-16035:
{code}
Failed tests:
TestReplicationKillMasterRSCompressedWithMultipleWAL>TestReplicationKillMasterRS.killOneMasterRS:34->TestReplicationKillRS.loadTableAndKillRS:88
Waited too much time for queueFailover replication. Waited 15112ms.
Flaked tests:
{code}
The above is not related to the patch.
> Nested AutoCloseables might not all get closed
> --
>
> Key: HBASE-16035
> URL: https://issues.apache.org/jira/browse/HBASE-16035
> Project: HBase
> Issue Type: Bug
>Reporter: Sean Mackrory
> Attachments: HBASE-16035-v1.patch
>
>
> Subtle problem in HBASE-15891:
> {code}try (A myA = new A(new B())){code}
> An exception thrown between B starting to open an A finishing initialization
> may not result in B being closed. A safer syntax would be:
> {code}try(B myB = new B(); A myA = newA(myB)){code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16074) ITBLL fails, reports lost big or tine families
[ https://issues.apache.org/jira/browse/HBASE-16074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340649#comment-15340649 ] Mikhail Antonov commented on HBASE-16074: - Well that's it (module num nodes), I'm just running several iterations of Loop according to this usage: Usage: Loop [ ] You can try with Loop 2 > 5000 /tmp/itbll > ITBLL fails, reports lost big or tine families > -- > > Key: HBASE-16074 > URL: https://issues.apache.org/jira/browse/HBASE-16074 > Project: HBase > Issue Type: Bug > Components: integration tests >Affects Versions: 1.3.0 >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov >Priority: Blocker > Fix For: 1.3.0 > > > Underlying MR jobs succeed but I'm seeing the following in the logs (mid-size > distributed test cluster): > ERROR test.IntegrationTestBigLinkedList$Verify: Found nodes which lost big or > tiny families, count=164 > I do not know exactly yet whether it's a bug, a test issue or env setup > issue, but need figure it out. Opening this to raise awareness and see if > someone saw that recently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16023) Fastpath for the FIFO rpcscheduler
[ https://issues.apache.org/jira/browse/HBASE-16023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340614#comment-15340614 ] Hudson commented on HBASE-16023: SUCCESS: Integrated in HBase-1.3-IT #722 (See [https://builds.apache.org/job/HBase-1.3-IT/722/]) HBASE-16023 Fastpath for the FIFO rpcscheduler AMENDMENT (stack: rev 6c519f36210f9f917e8cf40c96223cac755bb272) * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/FifoWithFastPathBalancedQueueRpcExecutor.java > Fastpath for the FIFO rpcscheduler > -- > > Key: HBASE-16023 > URL: https://issues.apache.org/jira/browse/HBASE-16023 > Project: HBase > Issue Type: Sub-task > Components: Performance, rpc >Affects Versions: 2.0.0, 1.3.0 >Reporter: stack >Assignee: stack > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-16023.addenum.patch, > HBASE-16023.branch-1.001.patch, hits.nofifo.fifoplusfp.fifownofp.hacks.png > > > This is an idea copied from kudu where we skip queuing a request if there is > a handler ready to go; we just do a direct handoff from reader to handler. > Makes for close to a %20 improvement in random read workloadc testing moving > the bottleneck to HBASE-15716 and to returning the results. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13372) Unit tests for SplitTransaction and RegionMergeTransaction listeners
[ https://issues.apache.org/jira/browse/HBASE-13372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340611#comment-15340611 ] Hudson commented on HBASE-13372: SUCCESS: Integrated in HBase-1.3-IT #722 (See [https://builds.apache.org/job/HBase-1.3-IT/722/]) HBASE-13372 Add unit tests for SplitTransaction and (antonov: rev e141ac89baf2fd91a2eeeddb55b0d9932fb924fa) * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionMergeTransaction.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransaction.java > Unit tests for SplitTransaction and RegionMergeTransaction listeners > > > Key: HBASE-13372 > URL: https://issues.apache.org/jira/browse/HBASE-13372 > Project: HBase > Issue Type: Test >Affects Versions: 2.0.0, 1.1.0 >Reporter: Andrew Purtell >Assignee: Gabor Liptak > Labels: beginner > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-13372.2.patch, HBASE-13372.3.patch, > HBASE-13372.4.patch > > > We have new Listener interfaces in SplitTransaction and > RegionMergeTransaction. There are no use cases for these yet, nor unit tests. > We should have unit tests for these that do something just a bit nontrivial > so as to provide a useful example. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12940) Expose listPeerConfigs and getPeerConfig to the HBase shell
[ https://issues.apache.org/jira/browse/HBASE-12940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340613#comment-15340613 ] Hudson commented on HBASE-12940: SUCCESS: Integrated in HBase-1.3-IT #722 (See [https://builds.apache.org/job/HBase-1.3-IT/722/]) HBASE-12940 Expose listPeerConfigs and getPeerConfig to the HBase shell (antonov: rev 089494d837fcc3715eb27e0b3c0da9264979dae5) * hbase-shell/src/main/ruby/shell.rb * hbase-shell/src/main/ruby/hbase/replication_admin.rb * hbase-shell/src/test/ruby/hbase/replication_admin_test.rb * hbase-shell/src/main/ruby/shell/commands/list_peer_configs.rb * hbase-shell/src/main/ruby/shell/commands/get_peer_config.rb > Expose listPeerConfigs and getPeerConfig to the HBase shell > --- > > Key: HBASE-12940 > URL: https://issues.apache.org/jira/browse/HBASE-12940 > Project: HBase > Issue Type: Improvement > Components: shell >Reporter: Kevin Risden >Assignee: Geoffrey Jacoby > Fix For: 2.0.0, 1.3.0, 0.98.18, 1.4.0 > > Attachments: HBASE-12940-v1.patch, HBASE-12940.patch > > > In HBASE-12867 found that listPeerConfigs and getPeerConfig from > ReplicationAdmin are not exposed to the HBase shell. This makes looking at > details for custom replication endpoints and testing of add_peer from > HBASE-12867 impossible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14644) Region in transition metric is broken
[ https://issues.apache.org/jira/browse/HBASE-14644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340610#comment-15340610 ] Hudson commented on HBASE-14644: SUCCESS: Integrated in HBase-1.3-IT #722 (See [https://builds.apache.org/job/HBase-1.3-IT/722/]) HBASE-14644 Region in transition metric is broken -- addendum (Huaxiang (matteo.bertozzi: rev c6953d68d0fd4db46e947a66b83ff5f7a2833fc5) * hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java > Region in transition metric is broken > - > > Key: HBASE-14644 > URL: https://issues.apache.org/jira/browse/HBASE-14644 > Project: HBase > Issue Type: Bug >Reporter: Elliott Clark >Assignee: huaxiang sun > Fix For: 2.0.0, 1.3.0, 1.2.2 > > Attachments: HBASE-14644-v001.patch, HBASE-14644-v002-addendum.patch, > HBASE-14644-v002.patch, HBASE-14644-v002.patch, branch-1.diff > > > ritCount stays 0 no matter what -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15870) Specify columns in REST multi gets
[ https://issues.apache.org/jira/browse/HBASE-15870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340612#comment-15340612 ] Hudson commented on HBASE-15870: SUCCESS: Integrated in HBase-1.3-IT #722 (See [https://builds.apache.org/job/HBase-1.3-IT/722/]) HBASE-15870 Specify columns in REST multi gets (Matt Warhaftig) (jerryjch: rev d91e42b8407dd68dce702c76fb4843326c8e38e4) * hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/TestMultiRowResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/MultiRowResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/TableResource.java > Specify columns in REST multi gets > -- > > Key: HBASE-15870 > URL: https://issues.apache.org/jira/browse/HBASE-15870 > Project: HBase > Issue Type: New Feature > Components: REST >Reporter: Dean Gurvitz >Assignee: Matt Warhaftig >Priority: Minor > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2 > > Attachments: hbase-15870-v1.patch > > > The REST multi-gets feature currently does not allow specifying only certain > columns or column families. Adding support for these should be quite simple > and improve the usability of the multi-gets feature. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16074) ITBLL fails, reports lost big or tine families
[ https://issues.apache.org/jira/browse/HBASE-16074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340595#comment-15340595 ] Dima Spivak commented on HBASE-16074: - Wanna paste your exact command? I can spin up a cluster on my end in Docker and loop it a few times to see if it's reproducible in my env. > ITBLL fails, reports lost big or tine families > -- > > Key: HBASE-16074 > URL: https://issues.apache.org/jira/browse/HBASE-16074 > Project: HBase > Issue Type: Bug > Components: integration tests >Affects Versions: 1.3.0 >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov >Priority: Blocker > Fix For: 1.3.0 > > > Underlying MR jobs succeed but I'm seeing the following in the logs (mid-size > distributed test cluster): > ERROR test.IntegrationTestBigLinkedList$Verify: Found nodes which lost big or > tiny families, count=164 > I do not know exactly yet whether it's a bug, a test issue or env setup > issue, but need figure it out. Opening this to raise awareness and see if > someone saw that recently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16036) Fix ReplicationTableBase initialization to make it non-blocking
[ https://issues.apache.org/jira/browse/HBASE-16036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340576#comment-15340576 ] Ashu Pachauri commented on HBASE-16036: --- I just had an offline chat with [~Vegetable26] [~eclark]. The reason we can't use createTableProcedure is that we want to keep HBase Master outside the whole replication scenario (which is consistent with current replication behavior), which makes perfect sense. > Fix ReplicationTableBase initialization to make it non-blocking > --- > > Key: HBASE-16036 > URL: https://issues.apache.org/jira/browse/HBASE-16036 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > Attachments: HBASE-16036.patch > > > Currently there is a bug inside of TableBasedReplicationQueuesImpl > construction that prevents ReplicationServices from starting before Master is > initialized. So currently each of the RS, including HMaster, with Replication > enabled will attempt to create the ReplicationTable on initialization. > Currently HMaster's initialization: serviceThreads.start() -> new > TableBasedReplicationQueuesImpl() -> Replication Table Creation -> HMaster > sets initialized flags. > But this fails when we try to create the Replication Table as the > HMaster.checkInitialized() flag fails. This ends up blocking HMaster > initialization and results in a deadlock. > So in this patch, I will create the Replication Table in the background of > TableBasedReplicationQueuesImpl and only block when we actually call methods > that access it. > This also requires a small refactoring of ReplicationSourceManager.init() so > that we run the abandoned queue adoption in the background > Review board at: https://reviews.apache.org/r/48763/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14644) Region in transition metric is broken
[ https://issues.apache.org/jira/browse/HBASE-14644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340568#comment-15340568 ] Hudson commented on HBASE-14644: FAILURE: Integrated in HBase-1.4 #229 (See [https://builds.apache.org/job/HBase-1.4/229/]) HBASE-14644 Region in transition metric is broken -- addendum (Huaxiang (matteo.bertozzi: rev 6605f8f683bce1993942df78597ecc7c277e556c) * hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java > Region in transition metric is broken > - > > Key: HBASE-14644 > URL: https://issues.apache.org/jira/browse/HBASE-14644 > Project: HBase > Issue Type: Bug >Reporter: Elliott Clark >Assignee: huaxiang sun > Fix For: 2.0.0, 1.3.0, 1.2.2 > > Attachments: HBASE-14644-v001.patch, HBASE-14644-v002-addendum.patch, > HBASE-14644-v002.patch, HBASE-14644-v002.patch, branch-1.diff > > > ritCount stays 0 no matter what -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16059) Region normalizer fails to trigger merge action where one of the regions is empty
[
https://issues.apache.org/jira/browse/HBASE-16059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340569#comment-15340569
]
Hudson commented on HBASE-16059:
FAILURE: Integrated in HBase-1.4 #229 (See
[https://builds.apache.org/job/HBase-1.4/229/])
HBASE-16059 Region normalizer fails to trigger merge action where one of
(tedyu: rev 78f52628960547f4717621e0863456fe1af0cda6)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SimpleRegionNormalizer.java
> Region normalizer fails to trigger merge action where one of the regions is
> empty
> -
>
> Key: HBASE-16059
> URL: https://issues.apache.org/jira/browse/HBASE-16059
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Romil Choksi
>Assignee: Ted Yu
> Labels: normalization
> Fix For: 2.0.0, 1.4.0
>
> Attachments: 16059.v1.txt
>
>
> Region normalizer failed to trigger merge action where expected
> Steps to reproduce:
> - Pre-split the test table into 5 regions with keys 1,3,7,8
> - Insert some data for each of the split. 27K rows for regions starting with
> key 1, and 100K rows for each of the regions with start key 3,7 and 8
> - Scan the test table, and verify that these regions exists - 1) STARTKEY =>
> ‘' ENDKEY => ’1’ 2) STARTKEY => ’1’ ENDKEY => ’3’
> - Turn on normalization, verify normalization switch is enabled and that
> normalization is true for test table
> - Run normalizer a few times
> - Scan test table again, verify that regions don’t exist anymore 1) STARTKEY
> => ‘' ENDKEY => ’1’ 2) STARTKEY => ’1’ ENDKEY => ’3’, but instead a new
> region is created with STARTKEY => ’’ ENDKEY => ’3’
> The test now fails, with the last step failing at assertion.
> Looking into the Master log, I see that normalization plan was computed for
> the test table but it decides that no normalization is needed for the test
> table, and that the regions look good.
> {code:title = Master.log}
> 2016-06-17 00:41:46,895 DEBUG
> [B.defaultRpcServer.handler=4,queue=1,port=2]
> normalizer.SimpleRegionNormalizer: Computing normalization plan for table:
> table_zrof6ea383, number of regions: 5
> 2016-06-17 00:41:46,895 DEBUG
> [B.defaultRpcServer.handler=4,queue=1,port=2]
> normalizer.SimpleRegionNormalizer: Table table_zrof6ea383, total aggregated
> regions size: 13
> 2016-06-17 00:41:46,896 DEBUG
> [B.defaultRpcServer.handler=4,queue=1,port=2]
> normalizer.SimpleRegionNormalizer: Table table_zrof6ea383, average region
> size: 2.6
> 2016-06-17 00:41:46,896 DEBUG
> [B.defaultRpcServer.handler=4,queue=1,port=2]
> normalizer.SimpleRegionNormalizer: No normalization needed, regions look good
> for table: table_zrof6ea383
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16074) ITBLL fails, reports lost big or tine families
[ https://issues.apache.org/jira/browse/HBASE-16074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340563#comment-15340563 ] Mikhail Antonov commented on HBASE-16074: - I ran Standard Loop with 5000 nodes per mapper. > ITBLL fails, reports lost big or tine families > -- > > Key: HBASE-16074 > URL: https://issues.apache.org/jira/browse/HBASE-16074 > Project: HBase > Issue Type: Bug > Components: integration tests >Affects Versions: 1.3.0 >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov >Priority: Blocker > Fix For: 1.3.0 > > > Underlying MR jobs succeed but I'm seeing the following in the logs (mid-size > distributed test cluster): > ERROR test.IntegrationTestBigLinkedList$Verify: Found nodes which lost big or > tiny families, count=164 > I do not know exactly yet whether it's a bug, a test issue or env setup > issue, but need figure it out. Opening this to raise awareness and see if > someone saw that recently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13701) Consolidate SecureBulkLoadEndpoint into HBase core as default for bulk load
[ https://issues.apache.org/jira/browse/HBASE-13701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340555#comment-15340555 ] Enis Soztutar commented on HBASE-13701: --- bq. I wondered how that is going to work out for the Compatibility if I do away the SecureBulkLoadEndpoint but provide the same service on the RSRpcServices. We can keep SecureBulkLoadEndpoint as a class, which is just a thin wrapper to call the underlying methods in the regionserver. So that even if you have configured it, 1.x client will still work agains 2.x clusters. > Consolidate SecureBulkLoadEndpoint into HBase core as default for bulk load > --- > > Key: HBASE-13701 > URL: https://issues.apache.org/jira/browse/HBASE-13701 > Project: HBase > Issue Type: Improvement >Reporter: Jerry He >Assignee: Jerry He > > HBASE-12052 makes SecureBulkLoadEndpoint work in a non-secure env to solve > HDFS permission issues. > We have encountered some of the permission issues and have to use this > SecureBulkLoadEndpoint to workaround issues. > We should probably consolidate SecureBulkLoadEndpoint into HBase core as > default for bulk load since it is able to handle both secure Kerberos and > non-secure cases. > Maintaining two versions of bulk load implementation is also a cause of > confusion, and having to explicitly set it is also inconvenient. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14644) Region in transition metric is broken
[ https://issues.apache.org/jira/browse/HBASE-14644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340556#comment-15340556 ] Hudson commented on HBASE-14644: SUCCESS: Integrated in HBase-1.2 #654 (See [https://builds.apache.org/job/HBase-1.2/654/]) HBASE-14644 Region in transition metric is broken -- addendum (Huaxiang (matteo.bertozzi: rev ba725759a1ea113c9c91345541b3635135c1ed5a) * hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java > Region in transition metric is broken > - > > Key: HBASE-14644 > URL: https://issues.apache.org/jira/browse/HBASE-14644 > Project: HBase > Issue Type: Bug >Reporter: Elliott Clark >Assignee: huaxiang sun > Fix For: 2.0.0, 1.3.0, 1.2.2 > > Attachments: HBASE-14644-v001.patch, HBASE-14644-v002-addendum.patch, > HBASE-14644-v002.patch, HBASE-14644-v002.patch, branch-1.diff > > > ritCount stays 0 no matter what -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12940) Expose listPeerConfigs and getPeerConfig to the HBase shell
[ https://issues.apache.org/jira/browse/HBASE-12940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340535#comment-15340535 ] Hudson commented on HBASE-12940: FAILURE: Integrated in HBase-1.3 #749 (See [https://builds.apache.org/job/HBase-1.3/749/]) HBASE-12940 Expose listPeerConfigs and getPeerConfig to the HBase shell (antonov: rev 089494d837fcc3715eb27e0b3c0da9264979dae5) * hbase-shell/src/main/ruby/shell/commands/list_peer_configs.rb * hbase-shell/src/main/ruby/hbase/replication_admin.rb * hbase-shell/src/test/ruby/hbase/replication_admin_test.rb * hbase-shell/src/main/ruby/shell/commands/get_peer_config.rb * hbase-shell/src/main/ruby/shell.rb > Expose listPeerConfigs and getPeerConfig to the HBase shell > --- > > Key: HBASE-12940 > URL: https://issues.apache.org/jira/browse/HBASE-12940 > Project: HBase > Issue Type: Improvement > Components: shell >Reporter: Kevin Risden >Assignee: Geoffrey Jacoby > Fix For: 2.0.0, 1.3.0, 0.98.18, 1.4.0 > > Attachments: HBASE-12940-v1.patch, HBASE-12940.patch > > > In HBASE-12867 found that listPeerConfigs and getPeerConfig from > ReplicationAdmin are not exposed to the HBase shell. This makes looking at > details for custom replication endpoints and testing of add_peer from > HBASE-12867 impossible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14644) Region in transition metric is broken
[ https://issues.apache.org/jira/browse/HBASE-14644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340533#comment-15340533 ] Hudson commented on HBASE-14644: FAILURE: Integrated in HBase-1.3 #749 (See [https://builds.apache.org/job/HBase-1.3/749/]) HBASE-14644 Region in transition metric is broken -- addendum (Huaxiang (matteo.bertozzi: rev c6953d68d0fd4db46e947a66b83ff5f7a2833fc5) * hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java > Region in transition metric is broken > - > > Key: HBASE-14644 > URL: https://issues.apache.org/jira/browse/HBASE-14644 > Project: HBase > Issue Type: Bug >Reporter: Elliott Clark >Assignee: huaxiang sun > Fix For: 2.0.0, 1.3.0, 1.2.2 > > Attachments: HBASE-14644-v001.patch, HBASE-14644-v002-addendum.patch, > HBASE-14644-v002.patch, HBASE-14644-v002.patch, branch-1.diff > > > ritCount stays 0 no matter what -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13372) Unit tests for SplitTransaction and RegionMergeTransaction listeners
[ https://issues.apache.org/jira/browse/HBASE-13372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340534#comment-15340534 ] Hudson commented on HBASE-13372: FAILURE: Integrated in HBase-1.3 #749 (See [https://builds.apache.org/job/HBase-1.3/749/]) HBASE-13372 Add unit tests for SplitTransaction and (antonov: rev e141ac89baf2fd91a2eeeddb55b0d9932fb924fa) * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransaction.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionMergeTransaction.java > Unit tests for SplitTransaction and RegionMergeTransaction listeners > > > Key: HBASE-13372 > URL: https://issues.apache.org/jira/browse/HBASE-13372 > Project: HBase > Issue Type: Test >Affects Versions: 2.0.0, 1.1.0 >Reporter: Andrew Purtell >Assignee: Gabor Liptak > Labels: beginner > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-13372.2.patch, HBASE-13372.3.patch, > HBASE-13372.4.patch > > > We have new Listener interfaces in SplitTransaction and > RegionMergeTransaction. There are no use cases for these yet, nor unit tests. > We should have unit tests for these that do something just a bit nontrivial > so as to provide a useful example. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16074) ITBLL fails, reports lost big or tine families
[ https://issues.apache.org/jira/browse/HBASE-16074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340494#comment-15340494 ] Dima Spivak commented on HBASE-16074: - Hey [~mantonov], I run ITBLL pretty regularly. What command invocation are you using that leads to this? > ITBLL fails, reports lost big or tine families > -- > > Key: HBASE-16074 > URL: https://issues.apache.org/jira/browse/HBASE-16074 > Project: HBase > Issue Type: Bug > Components: integration tests >Affects Versions: 1.3.0 >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov >Priority: Blocker > Fix For: 1.3.0 > > > Underlying MR jobs succeed but I'm seeing the following in the logs (mid-size > distributed test cluster): > ERROR test.IntegrationTestBigLinkedList$Verify: Found nodes which lost big or > tiny families, count=164 > I do not know exactly yet whether it's a bug, a test issue or env setup > issue, but need figure it out. Opening this to raise awareness and see if > someone saw that recently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16074) ITBLL fails, reports lost big or tine families
[ https://issues.apache.org/jira/browse/HBASE-16074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-16074: Description: Underlying MR jobs succeed but I'm seeing the following in the logs (mid-size distributed test cluster): ERROR test.IntegrationTestBigLinkedList$Verify: Found nodes which lost big or tiny families, count=164 I do not know exactly yet whether it's a bug, a test issue or env setup issue, but need figure it out. Opening this to raise awareness and see if someone saw that recently. was: Underlying MR jobs succeed but I'm seeing the following in the logs (mid-size distributed test cluster): ERROR test.IntegrationTestBigLinkedList$Verify: Found nodes which lost big or tiny families, count=164 I do know know exactly yet whether it's a bug, a test issue or env setup issue, but need figure it out. Opening this to raise awareness and see if someone saw that recently. > ITBLL fails, reports lost big or tine families > -- > > Key: HBASE-16074 > URL: https://issues.apache.org/jira/browse/HBASE-16074 > Project: HBase > Issue Type: Bug > Components: integration tests >Affects Versions: 1.3.0 >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov >Priority: Blocker > Fix For: 1.3.0 > > > Underlying MR jobs succeed but I'm seeing the following in the logs (mid-size > distributed test cluster): > ERROR test.IntegrationTestBigLinkedList$Verify: Found nodes which lost big or > tiny families, count=164 > I do not know exactly yet whether it's a bug, a test issue or env setup > issue, but need figure it out. Opening this to raise awareness and see if > someone saw that recently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16074) ITBLL fails, reports lost big or tine families
[ https://issues.apache.org/jira/browse/HBASE-16074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-16074: Description: Underlying MR jobs succeed but I'm seeing the following in the logs (mid-size distributed test cluster): ERROR test.IntegrationTestBigLinkedList$Verify: Found nodes which lost big or tiny families, count=164 I do know know exactly yet whether it's a bug, a test issue or env setup issue, but need figure it out. Opening this to raise awareness and see if someone saw that recently. was: Underlying MR jobs succeed but I'm seeing the following in the logs: ERROR test.IntegrationTestBigLinkedList$Verify: Found nodes which lost big or tiny families, count=164 I do know know exactly yet whether it's a bug, a test issue or env setup issue, but need figure it out. Opening this to raise awareness and see if someone saw that recently. > ITBLL fails, reports lost big or tine families > -- > > Key: HBASE-16074 > URL: https://issues.apache.org/jira/browse/HBASE-16074 > Project: HBase > Issue Type: Bug > Components: integration tests >Affects Versions: 1.3.0 >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov >Priority: Blocker > Fix For: 1.3.0 > > > Underlying MR jobs succeed but I'm seeing the following in the logs (mid-size > distributed test cluster): > ERROR test.IntegrationTestBigLinkedList$Verify: Found nodes which lost big or > tiny families, count=164 > I do know know exactly yet whether it's a bug, a test issue or env setup > issue, but need figure it out. Opening this to raise awareness and see if > someone saw that recently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16074) ITBLL fails, reports lost big or tine families
Mikhail Antonov created HBASE-16074: --- Summary: ITBLL fails, reports lost big or tine families Key: HBASE-16074 URL: https://issues.apache.org/jira/browse/HBASE-16074 Project: HBase Issue Type: Bug Components: integration tests Affects Versions: 1.3.0 Reporter: Mikhail Antonov Assignee: Mikhail Antonov Priority: Blocker Fix For: 1.3.0 Underlying MR jobs succeed but I'm seeing the following in the logs: ERROR test.IntegrationTestBigLinkedList$Verify: Found nodes which lost big or tiny families, count=164 I do know know exactly yet whether it's a bug, a test issue or env setup issue, but need figure it out. Opening this to raise awareness and see if someone saw that recently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13701) Consolidate SecureBulkLoadEndpoint into HBase core as default for bulk load
[ https://issues.apache.org/jira/browse/HBASE-13701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340473#comment-15340473 ] Jerry He commented on HBASE-13701: -- I wondered how that is going to work out for the Compatibility if I do away the SecureBulkLoadEndpoint but provide the same service on the RSRpcServices. SecureBulkLoadEndpoint is private, but it provides a public service. Need any deprecation? Old client won't work against new server. Is it Ok for 2.0? > Consolidate SecureBulkLoadEndpoint into HBase core as default for bulk load > --- > > Key: HBASE-13701 > URL: https://issues.apache.org/jira/browse/HBASE-13701 > Project: HBase > Issue Type: Improvement >Reporter: Jerry He >Assignee: Jerry He > > HBASE-12052 makes SecureBulkLoadEndpoint work in a non-secure env to solve > HDFS permission issues. > We have encountered some of the permission issues and have to use this > SecureBulkLoadEndpoint to workaround issues. > We should probably consolidate SecureBulkLoadEndpoint into HBase core as > default for bulk load since it is able to handle both secure Kerberos and > non-secure cases. > Maintaining two versions of bulk load implementation is also a cause of > confusion, and having to explicitly set it is also inconvenient. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16060) 1.x clients cannot access table state talking to 2.0 cluster
[
https://issues.apache.org/jira/browse/HBASE-16060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340465#comment-15340465
]
Jerry He commented on HBASE-16060:
--
[~enis]
I was thinking about doing HBASE-13701. I wondered how that is going to work
out for the Compatibility if I do away the SecureBulkLoadEndpoint but provide
the same service on the RSRpcServices.
> 1.x clients cannot access table state talking to 2.0 cluster
>
>
> Key: HBASE-16060
> URL: https://issues.apache.org/jira/browse/HBASE-16060
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
> Fix For: 2.0.0
>
>
> Since table state is migrated to meta instead of zk in 2.0, 1.x clients
> talking to 2.0 cluster cannot access the table state. This causes some weird
> behavior since from a client perspective, {{Admin.isTableEnabled()}} and
> {{Admin.isTableDisabled()}} both return false.
> One option we can do is to add code in 1.x clients so that they can access
> the table state in meta if needed. Otherwise, we can mirror the table state
> in zk (while keeping meta as the source of truth) during 2.x lifecycle so
> that any 1.x client can still work correctly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15406) Split / merge switch left disabled after early termination of hbck
[
https://issues.apache.org/jira/browse/HBASE-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340448#comment-15340448
]
Enis Soztutar commented on HBASE-15406:
---
Problam with Runtime.getRuntime().addShutdownHook() is that if you kill the
process with {{kill -9}} the hooks will not run. Having zk ephemeral node or
master procedure or something, we guarantee that the restore is run with 100%
guarantee.
> Split / merge switch left disabled after early termination of hbck
> --
>
> Key: HBASE-15406
> URL: https://issues.apache.org/jira/browse/HBASE-15406
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Heng Chen
> Labels: reviewed
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-15406.patch, HBASE-15406.v1.patch,
> HBASE-15406_v1.patch, HBASE-15406_v2.patch, test.patch, wip.patch
>
>
> This was what I did on cluster with 1.4.0-SNAPSHOT built Thursday:
> Run 'hbase hbck -disableSplitAndMerge' on gateway node of the cluster
> Terminate hbck early
> Enter hbase shell where I observed:
> {code}
> hbase(main):001:0> splitormerge_enabled 'SPLIT'
> false
> 0 row(s) in 0.3280 seconds
> hbase(main):002:0> splitormerge_enabled 'MERGE'
> false
> 0 row(s) in 0.0070 seconds
> {code}
> Expectation is that the split / merge switches should be restored to default
> value after hbck exits.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16036) Fix ReplicationTableBase initialization to make it non-blocking
[ https://issues.apache.org/jira/browse/HBASE-16036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340442#comment-15340442 ] Ashu Pachauri commented on HBASE-16036: --- I had a quick look at the code. Most things look fine to me but I only have two little concerns: 1. "hbase.replication.queues.createtable.workers": Why do you need more than one thread to create the replication table? 2. The current behavior is that regionservers would block on replication table to become available, which means we can't too any real work if replication is not initialized. If this is the case, you need not write a separate executor to deal with this and you can use createTableProcedure to bypass the checkInitialized check. See TableNamespaceManager#createNamespaceTable. The current approach also works similarly, so, I am not too particular about it. > Fix ReplicationTableBase initialization to make it non-blocking > --- > > Key: HBASE-16036 > URL: https://issues.apache.org/jira/browse/HBASE-16036 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > Attachments: HBASE-16036.patch > > > Currently there is a bug inside of TableBasedReplicationQueuesImpl > construction that prevents ReplicationServices from starting before Master is > initialized. So currently each of the RS, including HMaster, with Replication > enabled will attempt to create the ReplicationTable on initialization. > Currently HMaster's initialization: serviceThreads.start() -> new > TableBasedReplicationQueuesImpl() -> Replication Table Creation -> HMaster > sets initialized flags. > But this fails when we try to create the Replication Table as the > HMaster.checkInitialized() flag fails. This ends up blocking HMaster > initialization and results in a deadlock. > So in this patch, I will create the Replication Table in the background of > TableBasedReplicationQueuesImpl and only block when we actually call methods > that access it. > This also requires a small refactoring of ReplicationSourceManager.init() so > that we run the abandoned queue adoption in the background > Review board at: https://reviews.apache.org/r/48763/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-7612) [JDK8] Replace use of high-scale-lib counters with intrinsic facilities
[
https://issues.apache.org/jira/browse/HBASE-7612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340438#comment-15340438
]
Enis Soztutar commented on HBASE-7612:
--
Now that HBase-2.0 is JDK-8 only, we can revisit this. I've done a basic test,
and LongAdder is faster than our forked Counter implementation from my
micro-benchmark:
threads=1
{code}
Benchmark Mode Cnt Score
Error Units
CountersBenchmark.testIncrementAtomicLong thrpt 200 92234852.622 ±
162115.932 ops/s
CountersBenchmark.testIncrementCounter thrpt 200 59320305.701 ±
482597.475 ops/s
CountersBenchmark.testIncrementLongAdder thrpt 200 74932942.690 ±
87463.568 ops/s
{code}
threads=8
{code}
Benchmark Mode Cnt Score
Error Units
CountersBenchmark.testIncrementAtomicLong thrpt 200 34087507.496 ±
666013.635 ops/s
CountersBenchmark.testIncrementCounter thrpt 200 452497965.994 ±
7419222.873 ops/s
CountersBenchmark.testIncrementLongAdder thrpt 200 564211175.749 ±
2306961.876 ops/s
{code}
threads=max (24)
{code}
# Detecting actual CPU count: 24 detected
Benchmark Mode Cnt Score
Error Units
CountersBenchmark.testIncrementAtomicLong thrpt 20035218226.606 ±
998791.734 ops/s
CountersBenchmark.testIncrementCounter thrpt 200 653494810.370 ±
48383212.204 ops/s
CountersBenchmark.testIncrementLongAdder thrpt 200 1169425248.335 ±
4128542.456 ops/s
{code}
This is with jdk1.8.0_71 and 24 virtual cores.
Counter is one of the classes used in histogram metrics, which shows up pretty
hot in the workloadc profiles. cc [~saint@gmail.com], [~eclark].
> [JDK8] Replace use of high-scale-lib counters with intrinsic facilities
> ---
>
> Key: HBASE-7612
> URL: https://issues.apache.org/jira/browse/HBASE-7612
> Project: HBase
> Issue Type: Sub-task
>Reporter: Andrew Purtell
>Priority: Trivial
> Fix For: 2.0.0
>
>
> JEP155 introduces a few new classes (DoubleAccumulator, DoubleAdder,
> LongAccumulator, LongAdder) that "internally employ contention-reduction
> techniques that provide huge throughput improvements as compared to Atomic
> variables". There are applications of these where we are currently using
> Cliff Click's high-scale-lib and for metrics.
> See http://openjdk.java.net/jeps/155
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16059) Region normalizer fails to trigger merge action where one of the regions is empty
[
https://issues.apache.org/jira/browse/HBASE-16059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-16059:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 1.4.0
Status: Resolved (was: Patch Available)
Thanks for the review, Stephen.
> Region normalizer fails to trigger merge action where one of the regions is
> empty
> -
>
> Key: HBASE-16059
> URL: https://issues.apache.org/jira/browse/HBASE-16059
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Romil Choksi
>Assignee: Ted Yu
> Labels: normalization
> Fix For: 2.0.0, 1.4.0
>
> Attachments: 16059.v1.txt
>
>
> Region normalizer failed to trigger merge action where expected
> Steps to reproduce:
> - Pre-split the test table into 5 regions with keys 1,3,7,8
> - Insert some data for each of the split. 27K rows for regions starting with
> key 1, and 100K rows for each of the regions with start key 3,7 and 8
> - Scan the test table, and verify that these regions exists - 1) STARTKEY =>
> ‘' ENDKEY => ’1’ 2) STARTKEY => ’1’ ENDKEY => ’3’
> - Turn on normalization, verify normalization switch is enabled and that
> normalization is true for test table
> - Run normalizer a few times
> - Scan test table again, verify that regions don’t exist anymore 1) STARTKEY
> => ‘' ENDKEY => ’1’ 2) STARTKEY => ’1’ ENDKEY => ’3’, but instead a new
> region is created with STARTKEY => ’’ ENDKEY => ’3’
> The test now fails, with the last step failing at assertion.
> Looking into the Master log, I see that normalization plan was computed for
> the test table but it decides that no normalization is needed for the test
> table, and that the regions look good.
> {code:title = Master.log}
> 2016-06-17 00:41:46,895 DEBUG
> [B.defaultRpcServer.handler=4,queue=1,port=2]
> normalizer.SimpleRegionNormalizer: Computing normalization plan for table:
> table_zrof6ea383, number of regions: 5
> 2016-06-17 00:41:46,895 DEBUG
> [B.defaultRpcServer.handler=4,queue=1,port=2]
> normalizer.SimpleRegionNormalizer: Table table_zrof6ea383, total aggregated
> regions size: 13
> 2016-06-17 00:41:46,896 DEBUG
> [B.defaultRpcServer.handler=4,queue=1,port=2]
> normalizer.SimpleRegionNormalizer: Table table_zrof6ea383, average region
> size: 2.6
> 2016-06-17 00:41:46,896 DEBUG
> [B.defaultRpcServer.handler=4,queue=1,port=2]
> normalizer.SimpleRegionNormalizer: No normalization needed, regions look good
> for table: table_zrof6ea383
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15870) Specify columns in REST multi gets
[ https://issues.apache.org/jira/browse/HBASE-15870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry He updated HBASE-15870: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 1.2.2 1.4.0 1.3.0 2.0.0 Status: Resolved (was: Patch Available) Tested with a Rest server. Works well. Pushed to branch-1, branch-1.2, branch-1.3, master FYI [~busbey], [~mantonov] > Specify columns in REST multi gets > -- > > Key: HBASE-15870 > URL: https://issues.apache.org/jira/browse/HBASE-15870 > Project: HBase > Issue Type: New Feature > Components: REST >Reporter: Dean Gurvitz >Assignee: Matt Warhaftig >Priority: Minor > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2 > > Attachments: hbase-15870-v1.patch > > > The REST multi-gets feature currently does not allow specifying only certain > columns or column families. Adding support for these should be quite simple > and improve the usability of the multi-gets feature. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-7612) [JDK8] Replace use of high-scale-lib counters with intrinsic facilities
[ https://issues.apache.org/jira/browse/HBASE-7612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-7612: - Fix Version/s: 2.0.0 > [JDK8] Replace use of high-scale-lib counters with intrinsic facilities > --- > > Key: HBASE-7612 > URL: https://issues.apache.org/jira/browse/HBASE-7612 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Priority: Trivial > Fix For: 2.0.0 > > > JEP155 introduces a few new classes (DoubleAccumulator, DoubleAdder, > LongAccumulator, LongAdder) that "internally employ contention-reduction > techniques that provide huge throughput improvements as compared to Atomic > variables". There are applications of these where we are currently using > Cliff Click's high-scale-lib and for metrics. > See http://openjdk.java.net/jeps/155 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16073) update compatibility_checker for jacc dropping comma sep args
[
https://issues.apache.org/jira/browse/HBASE-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340362#comment-15340362
]
Hadoop QA commented on HBASE-16073:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
4s {color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m
0s {color} | {color:green} There were no new shelldocs issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
32m 56s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
43s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 36m 9s {color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12811977/HBASE-16073_v2.patch |
| JIRA Issue | HBASE-16073 |
| Optional Tests | asflicense shellcheck shelldocs |
| uname | Linux proserpina.apache.org 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 313a3d2 |
| shellcheck | v0.3.3 (This is an old version that has serious bugs. Consider
upgrading.) |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/2312/console |
| Powered by | Apache Yetus 0.2.1 http://yetus.apache.org |
This message was automatically generated.
> update compatibility_checker for jacc dropping comma sep args
> -
>
> Key: HBASE-16073
> URL: https://issues.apache.org/jira/browse/HBASE-16073
> Project: HBase
> Issue Type: Task
> Components: build, documentation
>Reporter: Sean Busbey
>Assignee: Dima Spivak
>Priority: Critical
> Attachments: HBASE-16073_v1.patch, HBASE-16073_v2.patch
>
>
> the japi-compliance-checker has a change in place (post the 1.7 release) that
> removes the ability to give a comma separated list of jars on the cli.
> we should switch to generating descriptor xml docs since that will still be
> supported, or update to use the expanded tooling suggested in the issue:
> https://github.com/lvc/japi-compliance-checker/issues/27
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16036) Fix ReplicationTableBase initialization to make it non-blocking
[ https://issues.apache.org/jira/browse/HBASE-16036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340334#comment-15340334 ] Ted Yu commented on HBASE-16036: Latest patch is cleaner. Let's see what QA bot says. > Fix ReplicationTableBase initialization to make it non-blocking > --- > > Key: HBASE-16036 > URL: https://issues.apache.org/jira/browse/HBASE-16036 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > Attachments: HBASE-16036.patch > > > Currently there is a bug inside of TableBasedReplicationQueuesImpl > construction that prevents ReplicationServices from starting before Master is > initialized. So currently each of the RS, including HMaster, with Replication > enabled will attempt to create the ReplicationTable on initialization. > Currently HMaster's initialization: serviceThreads.start() -> new > TableBasedReplicationQueuesImpl() -> Replication Table Creation -> HMaster > sets initialized flags. > But this fails when we try to create the Replication Table as the > HMaster.checkInitialized() flag fails. This ends up blocking HMaster > initialization and results in a deadlock. > So in this patch, I will create the Replication Table in the background of > TableBasedReplicationQueuesImpl and only block when we actually call methods > that access it. > This also requires a small refactoring of ReplicationSourceManager.init() so > that we run the abandoned queue adoption in the background > Review board at: https://reviews.apache.org/r/48763/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14644) Region in transition metric is broken
[ https://issues.apache.org/jira/browse/HBASE-14644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340318#comment-15340318 ] Hudson commented on HBASE-14644: SUCCESS: Integrated in HBase-1.2-IT #535 (See [https://builds.apache.org/job/HBase-1.2-IT/535/]) HBASE-14644 Region in transition metric is broken -- addendum (Huaxiang (matteo.bertozzi: rev ba725759a1ea113c9c91345541b3635135c1ed5a) * hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java > Region in transition metric is broken > - > > Key: HBASE-14644 > URL: https://issues.apache.org/jira/browse/HBASE-14644 > Project: HBase > Issue Type: Bug >Reporter: Elliott Clark >Assignee: huaxiang sun > Fix For: 2.0.0, 1.3.0, 1.2.2 > > Attachments: HBASE-14644-v001.patch, HBASE-14644-v002-addendum.patch, > HBASE-14644-v002.patch, HBASE-14644-v002.patch, branch-1.diff > > > ritCount stays 0 no matter what -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15935) Have a separate Walker task running concurrently with Generator
[
https://issues.apache.org/jira/browse/HBASE-15935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340310#comment-15340310
]
Hadoop QA commented on HBASE-15935:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
32s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 24s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 16s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 0s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 17s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 9s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 24s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 24s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 16s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 16s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 50s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 0s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 17s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 8s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 16s
{color} | {color:green} hbase-it in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 34m 18s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12811963/HBASE-15935.patch |
| JIRA Issue | HBASE-15935 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 313a3d2 |
| Default Java | 1.7.0_80 |
| Multi-JDK versions | /home/jenkins/tools/java/jdk1.8.0:1.8.0
/home/jenkins/jenkins-slave/tools/hudson.model.JDK/JDK_1.7_latest_:1.7.0_80 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/2310/testReport/ |
| modules | C: hbase-it U:
[jira] [Updated] (HBASE-15526) Make SnapshotManager accessible through MasterServices
[ https://issues.apache.org/jira/browse/HBASE-15526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-15526: --- Fix Version/s: 1.4.0 > Make SnapshotManager accessible through MasterServices > -- > > Key: HBASE-15526 > URL: https://issues.apache.org/jira/browse/HBASE-15526 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-15526.v1.patch > > > See this comment for background: > https://issues.apache.org/jira/browse/HBASE-15411?focusedCommentId=15209640=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15209640 > When procedure executes on master and performs snapshot, the procedure needs > to access SnapshotManager. > This JIRA is to add accessor to MasterServices. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-15558) Add random failure co processor to hbase-it
[ https://issues.apache.org/jira/browse/HBASE-15558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph reassigned HBASE-15558: -- Assignee: Joseph > Add random failure co processor to hbase-it > --- > > Key: HBASE-15558 > URL: https://issues.apache.org/jira/browse/HBASE-15558 > Project: HBase > Issue Type: Bug >Reporter: Elliott Clark >Assignee: Joseph > > HBase integration tests don't seem to be able to stress HBase all that much. > They don't add enough for there to be failures due to load. So lets add a > co-processor that can randomly fail rpc requests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16073) update compatibility_checker for jacc dropping comma sep args
[
https://issues.apache.org/jira/browse/HBASE-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dima Spivak updated HBASE-16073:
Attachment: HBASE-16073_v2.patch
Oy, long weekend. Uploaded a proper patch created with {{git format-patch}}
(instead of {{git diff}}).
> update compatibility_checker for jacc dropping comma sep args
> -
>
> Key: HBASE-16073
> URL: https://issues.apache.org/jira/browse/HBASE-16073
> Project: HBase
> Issue Type: Task
> Components: build, documentation
>Reporter: Sean Busbey
>Assignee: Dima Spivak
>Priority: Critical
> Attachments: HBASE-16073_v1.patch, HBASE-16073_v2.patch
>
>
> the japi-compliance-checker has a change in place (post the 1.7 release) that
> removes the ability to give a comma separated list of jars on the cli.
> we should switch to generating descriptor xml docs since that will still be
> supported, or update to use the expanded tooling suggested in the issue:
> https://github.com/lvc/japi-compliance-checker/issues/27
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16073) update compatibility_checker for jacc dropping comma sep args
[
https://issues.apache.org/jira/browse/HBASE-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340274#comment-15340274
]
Dima Spivak commented on HBASE-16073:
-
We can wait for {{yetus}} to run, but I've confirmed that report generation
works again on my local machine with the patch applied.
> update compatibility_checker for jacc dropping comma sep args
> -
>
> Key: HBASE-16073
> URL: https://issues.apache.org/jira/browse/HBASE-16073
> Project: HBase
> Issue Type: Task
> Components: build, documentation
>Reporter: Sean Busbey
>Assignee: Dima Spivak
>Priority: Critical
> Attachments: HBASE-16073_v1.patch
>
>
> the japi-compliance-checker has a change in place (post the 1.7 release) that
> removes the ability to give a comma separated list of jars on the cli.
> we should switch to generating descriptor xml docs since that will still be
> supported, or update to use the expanded tooling suggested in the issue:
> https://github.com/lvc/japi-compliance-checker/issues/27
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15881) Allow BZIP2 compression
[ https://issues.apache.org/jira/browse/HBASE-15881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-15881: Fix Version/s: (was: 1.3.1) 1.3.0 > Allow BZIP2 compression > --- > > Key: HBASE-15881 > URL: https://issues.apache.org/jira/browse/HBASE-15881 > Project: HBase > Issue Type: New Feature > Components: HFile >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.20 > > Attachments: 15881-0.98.txt > > > BZIP2 is a very efficient compressor in terms of compression rate. > Compression speed is very slow, de-compression is equivalent or faster than > GZIP. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15876) Remove doBulkLoad(Path hfofDir, final HTable table) though it has not been through a full deprecation cycle
[
https://issues.apache.org/jira/browse/HBASE-15876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-15876:
Fix Version/s: 1.3.0
> Remove doBulkLoad(Path hfofDir, final HTable table) though it has not been
> through a full deprecation cycle
> ---
>
> Key: HBASE-15876
> URL: https://issues.apache.org/jira/browse/HBASE-15876
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: stack
>Assignee: Jurriaan Mous
>Priority: Blocker
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-15876.patch
>
>
> HBASE-15875 purges the properly deprecated HTable. The method doBulkLoad(Path
> hfofDir, final HTable table), while it has a deprecated param, the method
> itself did not get a deprecation label; it is a public method in a public
> class marked stable. This issue is about getting consensus that it is ok to
> remove this method used by offline tooling that will break until updated on
> upgrade to 2.0 w/o a proper deprecation cycle (I think it will be ok to do
> this -- the benefit of our being able to remove HTable is worth this minor
> inconvenience). We'll do some ugly patching of this oversight by adding a
> late deprecation and flagging the removal of this offline method as an
> incompatible change in 2.0. We'll add the deprecation in a subissue.
> It is a problem removing
> doBulkLoad(Path hfofDir, final HTable table)
> ... since this is a public/stable Class.
> There is the alternate:
> public void doBulkLoad(Path hfofDir, final Admin admin, Table table,
> RegionLocator regionLocator) throws TableNotFoundException, IOException
> {
> The former calls the latter.
> The latter went in here:
> {code}
> commit ac95cc1fbb951bb9db96f2738f621d1d7cd45739
> Author: tedyu
> Date: Fri Jan 2 19:48:06 2015 -0800
> HBASE-12783 Create efficient RegionLocator implementation (Solomon Duskis)
> {code}
> This was added in time for release 1.1.0.
> So, this method has not gone through a full major version of deprecation.
> It will break when someone moves to 2.0. But it is offline method. Not the
> end of the world.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16055) PutSortReducer loses any Visibility/acl attribute set on the Puts
[
https://issues.apache.org/jira/browse/HBASE-16055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340270#comment-15340270
]
Hadoop QA commented on HBASE-16055:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
10s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 45s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 6s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 30s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 38s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 43s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 43s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 34s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 80m 39s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 124m 41s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12811866/HBASE-16055_1.patch |
| JIRA Issue | HBASE-16055 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 095a825 |
| Default Java | 1.7.0_80 |
| Multi-JDK versions | /home/jenkins/tools/java/jdk1.8.0:1.8.0
/home/jenkins/jenkins-slave/tools/hudson.model.JDK/JDK_1.7_latest_:1.7.0_80 |
| findbugs | v3.0.0 |
| Test Results |
[jira] [Updated] (HBASE-16036) Fix ReplicationTableBase initialization to make it non-blocking
[ https://issues.apache.org/jira/browse/HBASE-16036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-16036: --- Status: Patch Available (was: Open) > Fix ReplicationTableBase initialization to make it non-blocking > --- > > Key: HBASE-16036 > URL: https://issues.apache.org/jira/browse/HBASE-16036 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > Attachments: HBASE-16036.patch > > > Currently there is a bug inside of TableBasedReplicationQueuesImpl > construction that prevents ReplicationServices from starting before Master is > initialized. So currently each of the RS, including HMaster, with Replication > enabled will attempt to create the ReplicationTable on initialization. > Currently HMaster's initialization: serviceThreads.start() -> new > TableBasedReplicationQueuesImpl() -> Replication Table Creation -> HMaster > sets initialized flags. > But this fails when we try to create the Replication Table as the > HMaster.checkInitialized() flag fails. This ends up blocking HMaster > initialization and results in a deadlock. > So in this patch, I will create the Replication Table in the background of > TableBasedReplicationQueuesImpl and only block when we actually call methods > that access it. > This also requires a small refactoring of ReplicationSourceManager.init() so > that we run the abandoned queue adoption in the background > Review board at: https://reviews.apache.org/r/48763/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15609) Remove PB references from Result, DoubleColumnInterpreter and any such public facing class for 2.0
[ https://issues.apache.org/jira/browse/HBASE-15609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-15609: Fix Version/s: 1.3.0 > Remove PB references from Result, DoubleColumnInterpreter and any such public > facing class for 2.0 > -- > > Key: HBASE-15609 > URL: https://issues.apache.org/jira/browse/HBASE-15609 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15609-addendum.patch, HBASE-15609.patch, > HBASE-15609.patch, HBASE-15609_1.patch, HBASE-15609_4.patch, > HBASE-15609_branch-1.patch > > > This is a sub-task for HBASE-15174. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15607) Remove PB references from Admin for 2.0
[ https://issues.apache.org/jira/browse/HBASE-15607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-15607: Fix Version/s: 1.3.0 > Remove PB references from Admin for 2.0 > --- > > Key: HBASE-15607 > URL: https://issues.apache.org/jira/browse/HBASE-15607 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15607.patch, HBASE-15607_1.patch, > HBASE-15607_2.patch, HBASE-15607_3.patch, HBASE-15607_3.patch, > HBASE-15607_4.patch, HBASE-15607_4.patch, HBASE-15607_branch-1.patch > > > This is a sub-task for HBASE-15174. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15608) Remove PB references from SnapShot related Exceptions
[ https://issues.apache.org/jira/browse/HBASE-15608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-15608: Fix Version/s: 1.3.0 > Remove PB references from SnapShot related Exceptions > - > > Key: HBASE-15608 > URL: https://issues.apache.org/jira/browse/HBASE-15608 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15608-branch-1.patch, HBASE-15608.patch, > HBASE-15608_1.patch, HBASE-15608_2.patch, HBASE-15608_3.patch > > > This is a sub-task for HBASE-15174. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16036) Fix ReplicationTableBase initialization to make it non-blocking
[ https://issues.apache.org/jira/browse/HBASE-16036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-16036: --- Status: Open (was: Patch Available) > Fix ReplicationTableBase initialization to make it non-blocking > --- > > Key: HBASE-16036 > URL: https://issues.apache.org/jira/browse/HBASE-16036 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > Attachments: HBASE-16036.patch > > > Currently there is a bug inside of TableBasedReplicationQueuesImpl > construction that prevents ReplicationServices from starting before Master is > initialized. So currently each of the RS, including HMaster, with Replication > enabled will attempt to create the ReplicationTable on initialization. > Currently HMaster's initialization: serviceThreads.start() -> new > TableBasedReplicationQueuesImpl() -> Replication Table Creation -> HMaster > sets initialized flags. > But this fails when we try to create the Replication Table as the > HMaster.checkInitialized() flag fails. This ends up blocking HMaster > initialization and results in a deadlock. > So in this patch, I will create the Replication Table in the background of > TableBasedReplicationQueuesImpl and only block when we actually call methods > that access it. > This also requires a small refactoring of ReplicationSourceManager.init() so > that we run the abandoned queue adoption in the background > Review board at: https://reviews.apache.org/r/48763/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16036) Fix ReplicationTableBase initialization to make it non-blocking
[ https://issues.apache.org/jira/browse/HBASE-16036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-16036: --- Attachment: (was: HBASE-16036.patch) > Fix ReplicationTableBase initialization to make it non-blocking > --- > > Key: HBASE-16036 > URL: https://issues.apache.org/jira/browse/HBASE-16036 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > Attachments: HBASE-16036.patch > > > Currently there is a bug inside of TableBasedReplicationQueuesImpl > construction that prevents ReplicationServices from starting before Master is > initialized. So currently each of the RS, including HMaster, with Replication > enabled will attempt to create the ReplicationTable on initialization. > Currently HMaster's initialization: serviceThreads.start() -> new > TableBasedReplicationQueuesImpl() -> Replication Table Creation -> HMaster > sets initialized flags. > But this fails when we try to create the Replication Table as the > HMaster.checkInitialized() flag fails. This ends up blocking HMaster > initialization and results in a deadlock. > So in this patch, I will create the Replication Table in the background of > TableBasedReplicationQueuesImpl and only block when we actually call methods > that access it. > This also requires a small refactoring of ReplicationSourceManager.init() so > that we run the abandoned queue adoption in the background > Review board at: https://reviews.apache.org/r/48763/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16036) Fix ReplicationTableBase initialization to make it non-blocking
[ https://issues.apache.org/jira/browse/HBASE-16036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-16036: --- Attachment: HBASE-16036.patch > Fix ReplicationTableBase initialization to make it non-blocking > --- > > Key: HBASE-16036 > URL: https://issues.apache.org/jira/browse/HBASE-16036 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > Attachments: HBASE-16036.patch > > > Currently there is a bug inside of TableBasedReplicationQueuesImpl > construction that prevents ReplicationServices from starting before Master is > initialized. So currently each of the RS, including HMaster, with Replication > enabled will attempt to create the ReplicationTable on initialization. > Currently HMaster's initialization: serviceThreads.start() -> new > TableBasedReplicationQueuesImpl() -> Replication Table Creation -> HMaster > sets initialized flags. > But this fails when we try to create the Replication Table as the > HMaster.checkInitialized() flag fails. This ends up blocking HMaster > initialization and results in a deadlock. > So in this patch, I will create the Replication Table in the background of > TableBasedReplicationQueuesImpl and only block when we actually call methods > that access it. > This also requires a small refactoring of ReplicationSourceManager.init() so > that we run the abandoned queue adoption in the background > Review board at: https://reviews.apache.org/r/48763/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15605) Remove PB references from HCD and HTD for 2.0
[ https://issues.apache.org/jira/browse/HBASE-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-15605: Fix Version/s: 1.3.0 > Remove PB references from HCD and HTD for 2.0 > - > > Key: HBASE-15605 > URL: https://issues.apache.org/jira/browse/HBASE-15605 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15605.patch, HBASE-15605_1.patch, > HBASE-15605_2.patch, HBASE-15605_branch-1.patch > > > This task is sub-task for HBASE-15174. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16073) update compatibility_checker for jacc dropping comma sep args
[ https://issues.apache.org/jira/browse/HBASE-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak updated HBASE-16073: Attachment: HBASE-16073_v1.patch Simple fix that pulls down the 1.7 tag instead of whatever HEAD is pointing to. > update compatibility_checker for jacc dropping comma sep args > - > > Key: HBASE-16073 > URL: https://issues.apache.org/jira/browse/HBASE-16073 > Project: HBase > Issue Type: Task > Components: build, documentation >Reporter: Sean Busbey >Assignee: Dima Spivak >Priority: Critical > Attachments: HBASE-16073_v1.patch > > > the japi-compliance-checker has a change in place (post the 1.7 release) that > removes the ability to give a comma separated list of jars on the cli. > we should switch to generating descriptor xml docs since that will still be > supported, or update to use the expanded tooling suggested in the issue: > https://github.com/lvc/japi-compliance-checker/issues/27 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15526) Make SnapshotManager accessible through MasterServices
[ https://issues.apache.org/jira/browse/HBASE-15526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-15526: Fix Version/s: (was: 1.4.0) 1.3.0 > Make SnapshotManager accessible through MasterServices > -- > > Key: HBASE-15526 > URL: https://issues.apache.org/jira/browse/HBASE-15526 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15526.v1.patch > > > See this comment for background: > https://issues.apache.org/jira/browse/HBASE-15411?focusedCommentId=15209640=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15209640 > When procedure executes on master and performs snapshot, the procedure needs > to access SnapshotManager. > This JIRA is to add accessor to MasterServices. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16073) update compatibility_checker for jacc dropping comma sep args
[ https://issues.apache.org/jira/browse/HBASE-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak updated HBASE-16073: Status: Patch Available (was: Open) > update compatibility_checker for jacc dropping comma sep args > - > > Key: HBASE-16073 > URL: https://issues.apache.org/jira/browse/HBASE-16073 > Project: HBase > Issue Type: Task > Components: build, documentation >Reporter: Sean Busbey >Assignee: Dima Spivak >Priority: Critical > Attachments: HBASE-16073_v1.patch > > > the japi-compliance-checker has a change in place (post the 1.7 release) that > removes the ability to give a comma separated list of jars on the cli. > we should switch to generating descriptor xml docs since that will still be > supported, or update to use the expanded tooling suggested in the issue: > https://github.com/lvc/japi-compliance-checker/issues/27 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14485) ConnectionImplementation leaks on construction failure
[
https://issues.apache.org/jira/browse/HBASE-14485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-14485:
Resolution: Fixed
Status: Resolved (was: Patch Available)
> ConnectionImplementation leaks on construction failure
> --
>
> Key: HBASE-14485
> URL: https://issues.apache.org/jira/browse/HBASE-14485
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 2.0.0, 1.2.0, 1.0.1.1, 1.1.2, 1.3.0
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-14485-v0.patch, HBASE-14485-v1.patch,
> HBASE-14485-v1_branch-1.patch
>
>
> If an exception is thrown in the constructor of ConnectionImplementation we
> will have a leak zkRegistry, rpcClient, ...
> an example was clusterId parse error, causing zk (registry) leaks
> {noformat}
> org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238)
> ... 22 more
> Caused by: java.lang.ExceptionInInitializerError
> at org.apache.hadoop.hbase.ClusterId.parseFrom(ClusterId.java:64)
> at
> org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:75)
> at
> org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:86)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:850)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14485) ConnectionImplementation leaks on construction failure
[
https://issues.apache.org/jira/browse/HBASE-14485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-14485:
Fix Version/s: 1.3.0
2.0.0
> ConnectionImplementation leaks on construction failure
> --
>
> Key: HBASE-14485
> URL: https://issues.apache.org/jira/browse/HBASE-14485
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 2.0.0, 1.2.0, 1.0.1.1, 1.1.2, 1.3.0
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-14485-v0.patch, HBASE-14485-v1.patch,
> HBASE-14485-v1_branch-1.patch
>
>
> If an exception is thrown in the constructor of ConnectionImplementation we
> will have a leak zkRegistry, rpcClient, ...
> an example was clusterId parse error, causing zk (registry) leaks
> {noformat}
> org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238)
> ... 22 more
> Caused by: java.lang.ExceptionInInitializerError
> at org.apache.hadoop.hbase.ClusterId.parseFrom(ClusterId.java:64)
> at
> org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:75)
> at
> org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:86)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:850)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-14397) PrefixFilter doesn't filter all remaining rows if the prefix is longer than rowkey being compared
[
https://issues.apache.org/jira/browse/HBASE-14397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov resolved HBASE-14397.
-
Resolution: Fixed
> PrefixFilter doesn't filter all remaining rows if the prefix is longer than
> rowkey being compared
> -
>
> Key: HBASE-14397
> URL: https://issues.apache.org/jira/browse/HBASE-14397
> Project: HBase
> Issue Type: Improvement
> Components: Filters
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
>Assignee: Jianwei Cui
>Priority: Minor
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-14397-trunk-v1.patch
>
>
> The PrefixFilter will filter rowkey as:
> {code}
> public boolean filterRowKey(Cell firstRowCell) {
> ...
> int length = firstRowCell.getRowLength();
> if (length < prefix.length) return true; // ===> return directly if the
> prefix is longer
>
> if ((!isReversed() && cmp > 0) || (isReversed() && cmp < 0)) {
> passedPrefix = true;
> }
> filterRow = (cmp != 0);
> return filterRow;
> }
> {code}
> If the prefix is longer than the current rowkey, PrefixFilter#filterRowKey
> will filter the rowkey directly without comparing, so that won't set
> 'passedPrefix' flag even the current row is larger than the prefix.
> For example, if there are three rows 'a', 'b' and 'c' in the table, and we
> issue a scan request as:
> {code}
> hbase(main):001:0> scan 'test_table', {STARTROW => 'a', FILTER =>
> "(PrefixFilter ('aa'))"}
> {code}
> The region server will check the three rows before returning. In our
> production, the user issue a scan with a PrefixFilter. The prefix is longer
> than the rowkeys of following millions of rows, so the region server will
> continue to check rows until hit a rowkey longer than the prefix. This make
> the client easily timeout. To fix this case, it seems we need to compare the
> prefix with the rowkey every serveral rows even when the prefix is longer.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14397) PrefixFilter doesn't filter all remaining rows if the prefix is longer than rowkey being compared
[
https://issues.apache.org/jira/browse/HBASE-14397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-14397:
Fix Version/s: 1.3.0
2.0.0
> PrefixFilter doesn't filter all remaining rows if the prefix is longer than
> rowkey being compared
> -
>
> Key: HBASE-14397
> URL: https://issues.apache.org/jira/browse/HBASE-14397
> Project: HBase
> Issue Type: Improvement
> Components: Filters
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
>Assignee: Jianwei Cui
>Priority: Minor
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-14397-trunk-v1.patch
>
>
> The PrefixFilter will filter rowkey as:
> {code}
> public boolean filterRowKey(Cell firstRowCell) {
> ...
> int length = firstRowCell.getRowLength();
> if (length < prefix.length) return true; // ===> return directly if the
> prefix is longer
>
> if ((!isReversed() && cmp > 0) || (isReversed() && cmp < 0)) {
> passedPrefix = true;
> }
> filterRow = (cmp != 0);
> return filterRow;
> }
> {code}
> If the prefix is longer than the current rowkey, PrefixFilter#filterRowKey
> will filter the rowkey directly without comparing, so that won't set
> 'passedPrefix' flag even the current row is larger than the prefix.
> For example, if there are three rows 'a', 'b' and 'c' in the table, and we
> issue a scan request as:
> {code}
> hbase(main):001:0> scan 'test_table', {STARTROW => 'a', FILTER =>
> "(PrefixFilter ('aa'))"}
> {code}
> The region server will check the three rows before returning. In our
> production, the user issue a scan with a PrefixFilter. The prefix is longer
> than the rowkeys of following millions of rows, so the region server will
> continue to check rows until hit a rowkey longer than the prefix. This make
> the client easily timeout. To fix this case, it seems we need to compare the
> prefix with the rowkey every serveral rows even when the prefix is longer.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14397) PrefixFilter doesn't filter all remaining rows if the prefix is longer than rowkey being compared
[
https://issues.apache.org/jira/browse/HBASE-14397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340245#comment-15340245
]
Mikhail Antonov commented on HBASE-14397:
-
[~apurtell] re-resolving this issue as it hasn't been reverted or amended. Do
you want to open a follow-up issue?
> PrefixFilter doesn't filter all remaining rows if the prefix is longer than
> rowkey being compared
> -
>
> Key: HBASE-14397
> URL: https://issues.apache.org/jira/browse/HBASE-14397
> Project: HBase
> Issue Type: Improvement
> Components: Filters
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
>Assignee: Jianwei Cui
>Priority: Minor
> Attachments: HBASE-14397-trunk-v1.patch
>
>
> The PrefixFilter will filter rowkey as:
> {code}
> public boolean filterRowKey(Cell firstRowCell) {
> ...
> int length = firstRowCell.getRowLength();
> if (length < prefix.length) return true; // ===> return directly if the
> prefix is longer
>
> if ((!isReversed() && cmp > 0) || (isReversed() && cmp < 0)) {
> passedPrefix = true;
> }
> filterRow = (cmp != 0);
> return filterRow;
> }
> {code}
> If the prefix is longer than the current rowkey, PrefixFilter#filterRowKey
> will filter the rowkey directly without comparing, so that won't set
> 'passedPrefix' flag even the current row is larger than the prefix.
> For example, if there are three rows 'a', 'b' and 'c' in the table, and we
> issue a scan request as:
> {code}
> hbase(main):001:0> scan 'test_table', {STARTROW => 'a', FILTER =>
> "(PrefixFilter ('aa'))"}
> {code}
> The region server will check the three rows before returning. In our
> production, the user issue a scan with a PrefixFilter. The prefix is longer
> than the rowkeys of following millions of rows, so the region server will
> continue to check rows until hit a rowkey longer than the prefix. This make
> the client easily timeout. To fix this case, it seems we need to compare the
> prefix with the rowkey every serveral rows even when the prefix is longer.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14349) pre-commit zombie finder is overly broad
[
https://issues.apache.org/jira/browse/HBASE-14349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-14349:
Fix Version/s: 1.3.0
> pre-commit zombie finder is overly broad
>
>
> Key: HBASE-14349
> URL: https://issues.apache.org/jira/browse/HBASE-14349
> Project: HBase
> Issue Type: Bug
> Components: build
>Reporter: Sean Busbey
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0, 1.3.0
>
> Attachments: 14349.addendum.txt, 14349.txt, 14349v2.txt
>
>
> Zombie detector is flagging processes from builds that aren't ours.
> ex from HBASE-14337:
> {code}
> -1 core zombie tests. There are 4 zombie test(s): at
> org.apache.reef.io.network.DeprecatedNetworkConnectionServiceTest.testMultithreadedSharedConnMessagingNetworkConnServiceRate(DeprecatedNetworkConnectionServiceTest.java:343)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16073) update compatibility_checker for jacc dropping comma sep args
[ https://issues.apache.org/jira/browse/HBASE-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340239#comment-15340239 ] Dima Spivak commented on HBASE-16073: - Alright, we can circle back on longer-term stuff later. (Ideally, it'd be Python living in a Docker image for dependency management.) I'll try out the easier checkout-1.7-tag option now. > update compatibility_checker for jacc dropping comma sep args > - > > Key: HBASE-16073 > URL: https://issues.apache.org/jira/browse/HBASE-16073 > Project: HBase > Issue Type: Task > Components: build, documentation >Reporter: Sean Busbey >Assignee: Dima Spivak >Priority: Critical > > the japi-compliance-checker has a change in place (post the 1.7 release) that > removes the ability to give a comma separated list of jars on the cli. > we should switch to generating descriptor xml docs since that will still be > supported, or update to use the expanded tooling suggested in the issue: > https://github.com/lvc/japi-compliance-checker/issues/27 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-16073) update compatibility_checker for jacc dropping comma sep args
[ https://issues.apache.org/jira/browse/HBASE-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak reassigned HBASE-16073: --- Assignee: Dima Spivak > update compatibility_checker for jacc dropping comma sep args > - > > Key: HBASE-16073 > URL: https://issues.apache.org/jira/browse/HBASE-16073 > Project: HBase > Issue Type: Task > Components: build, documentation >Reporter: Sean Busbey >Assignee: Dima Spivak >Priority: Critical > > the japi-compliance-checker has a change in place (post the 1.7 release) that > removes the ability to give a comma separated list of jars on the cli. > we should switch to generating descriptor xml docs since that will still be > supported, or update to use the expanded tooling suggested in the issue: > https://github.com/lvc/japi-compliance-checker/issues/27 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13082) Coarsen StoreScanner locks to RegionScanner
[ https://issues.apache.org/jira/browse/HBASE-13082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-13082: Fix Version/s: 1.3.0 > Coarsen StoreScanner locks to RegionScanner > --- > > Key: HBASE-13082 > URL: https://issues.apache.org/jira/browse/HBASE-13082 > Project: HBase > Issue Type: Bug > Components: Performance, Scanners >Reporter: Lars Hofhansl >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0, 1.3.0 > > Attachments: 13082-test.txt, 13082-v2.txt, 13082-v3.txt, > 13082-v4.txt, 13082.txt, 13082.txt, CountDownLatch-0.98.txt, HBASE-13082.pdf, > HBASE-13082_1.pdf, HBASE-13082_12.patch, HBASE-13082_13.patch, > HBASE-13082_14.patch, HBASE-13082_15.patch, HBASE-13082_16.patch, > HBASE-13082_17.patch, HBASE-13082_18.patch, HBASE-13082_19.patch, > HBASE-13082_1_WIP.patch, HBASE-13082_2.pdf, HBASE-13082_2_WIP.patch, > HBASE-13082_3.patch, HBASE-13082_4.patch, HBASE-13082_9.patch, > HBASE-13082_9.patch, HBASE-13082_withoutpatch.jpg, HBASE-13082_withpatch.jpg, > LockVsSynchronized.java, gc.png, gc.png, gc.png, hits.png, next.png, next.png > > > Continuing where HBASE-10015 left of. > We can avoid locking (and memory fencing) inside StoreScanner by deferring to > the lock already held by the RegionScanner. > In tests this shows quite a scan improvement and reduced CPU (the fences make > the cores wait for memory fetches). > There are some drawbacks too: > * All calls to RegionScanner need to be remain synchronized > * Implementors of coprocessors need to be diligent in following the locking > contract. For example Phoenix does not lock RegionScanner.nextRaw() and > required in the documentation (not picking on Phoenix, this one is my fault > as I told them it's OK) > * possible starving of flushes and compaction with heavy read load. > RegionScanner operations would keep getting the locks and the > flushes/compactions would not be able finalize the set of files. > I'll have a patch soon. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16073) update compatibility_checker for jacc dropping comma sep args
[ https://issues.apache.org/jira/browse/HBASE-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340235#comment-15340235 ] Sean Busbey commented on HBASE-16073: - sure, if you like any of those solutions would be fine. one advantage of keeping it in bash is that I'd eventually like to adapt this into a precommit check, and that will be easier if it's bash (though it is doable if it's not). > update compatibility_checker for jacc dropping comma sep args > - > > Key: HBASE-16073 > URL: https://issues.apache.org/jira/browse/HBASE-16073 > Project: HBase > Issue Type: Task > Components: build, documentation >Reporter: Sean Busbey >Priority: Critical > > the japi-compliance-checker has a change in place (post the 1.7 release) that > removes the ability to give a comma separated list of jars on the cli. > we should switch to generating descriptor xml docs since that will still be > supported, or update to use the expanded tooling suggested in the issue: > https://github.com/lvc/japi-compliance-checker/issues/27 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12133) Add FastLongHistogram for metric computation
[ https://issues.apache.org/jira/browse/HBASE-12133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-12133: Fix Version/s: 1.3.0 > Add FastLongHistogram for metric computation > > > Key: HBASE-12133 > URL: https://issues.apache.org/jira/browse/HBASE-12133 > Project: HBase > Issue Type: New Feature > Components: metrics >Affects Versions: 0.98.8 >Reporter: Yi Deng >Assignee: Yi Deng >Priority: Minor > Labels: histogram, metrics > Fix For: 2.0.0, 0.99.1, 1.3.0 > > Attachments: > 0001-Add-FastLongHistogram-for-fast-histogram-estimation.patch, > 0001-Add-FastLongHistogram-for-fast-histogram-estimation.patch, > 0001-Add-FastLongHistogram-for-fast-histogram-estimation.patch, > 12133.addendum.txt > > > FastLongHistogram is a thread-safe class that estimate distribution of data > and computes the quantiles. It's useful for computing aggregated metrics like > P99/P95. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
