date:20150701


[ 
https://issues.apache.org/jira/browse/HBASE-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610746#comment-14610746
 ] 

stack commented on HBASE-13998:
---

In MetaCache.java, why is it safe to no longer check for meta table?

Else, patch looks good to me.

 Remove CellComparator#compareRows(byte[] left, int loffset, int llength, 
 byte[] right, int roffset, int rlength)
 

 Key: HBASE-13998
 URL: https://issues.apache.org/jira/browse/HBASE-13998
 Project: HBase
  Issue Type: Sub-task
Reporter: Anoop Sam John
Assignee: Anoop Sam John
 Fix For: 2.0.0

 Attachments: HBASE-13998.patch


 A public API in CellComparator which takes old style byte[], offset, length 
 alone is not correct.  CellComparator supposed to compare cell(s).  At least 
 one side param has to be a cell.. This is the agreement we discussed in 
 HBASE-10800.  Still we could not remove the above one method because it was 
 getting used from multiple places.  Now most of the usage is removed.  This 
 jira aims at removing it fully and replace the usage with other APIs.
 Note: The CellComparator is added in 2.0 only so removing the public API is 
 not creating any BC issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13978) Variable never assigned in SimpleTotalOrderPartitioner.getPartition()


 [ 
https://issues.apache.org/jira/browse/HBASE-13978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-13978:

   Resolution: Fixed
Fix Version/s: (was: 1.3.0)
   Status: Resolved  (was: Patch Available)

pushed to 1.2+

 Variable never assigned in SimpleTotalOrderPartitioner.getPartition() 
 --

 Key: HBASE-13978
 URL: https://issues.apache.org/jira/browse/HBASE-13978
 Project: HBase
  Issue Type: Bug
  Components: mapreduce
Affects Versions: 1.1.0.1
Reporter: Lars George
Assignee: Bhupendra Kumar Jain
  Labels: beginner
 Fix For: 2.0.0, 1.2.0

 Attachments: 
 0001-HBASE-13978-Variable-never-assigned-in-SimpleTotalOr.patch


 See 
 https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/SimpleTotalOrderPartitioner.java#L104,
  which has an {{if}} statement that tries to limit the code to run only once, 
 but since it does not assign {{this.lastReduces}} it will always trigger and 
 recompute the splits (and log them).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13014) Java Tool For Region Moving


 [ 
https://issues.apache.org/jira/browse/HBASE-13014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-13014:

Fix Version/s: (was: 1.2.0)
   1.3.0

 Java Tool For Region Moving 
 

 Key: HBASE-13014
 URL: https://issues.apache.org/jira/browse/HBASE-13014
 Project: HBase
  Issue Type: Improvement
Reporter: Abhishek Singh Chouhan
Assignee: Abhishek Singh Chouhan
 Fix For: 2.0.0, 0.98.14, 1.3.0

 Attachments: HBASE-13014-v2.patch, HBASE-13014-v3.patch, 
 HBASE-13014-v4.patch, HBASE-13014-v5.patch, HBASE-13014-v6.patch, 
 HBASE-13014.patch


 As per discussion on HBASE-12989 we should move the functionality of 
 region_mover.rb into a Java tool and use region_mover.rb only only as a 
 wrapper around it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13990) clean up remaining errors for maven site goal


[ 
https://issues.apache.org/jira/browse/HBASE-13990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610631#comment-14610631
 ] 

Hudson commented on HBASE-13990:


SUCCESS: Integrated in HBase-1.3-IT #15 (See 
[https://builds.apache.org/job/HBase-1.3-IT/15/])
HBASE-13990 make maven site generation work with jdk8 (busbey: rev 
a58848507a31f432785b2863f544aa6cca914a57)
* pom.xml
* 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/Result.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/filter/Filter.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/util/ZKDataMigrator.java
* 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/filter/FilterBase.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/util/ByteBloomFilter.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/filter/FilterList.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlockIndex.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValueUtil.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SimpleRegionNormalizer.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/coordination/SplitTransactionCoordination.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTablePool.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/filter/FilterWrapper.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/coordination/SplitLogWorkerCoordination.java
* 
hbase-prefix-tree/src/main/java/org/apache/hadoop/hbase/codec/prefixtree/PrefixTreeSeeker.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java


 clean up remaining errors for maven site goal
 -

 Key: HBASE-13990
 URL: https://issues.apache.org/jira/browse/HBASE-13990
 Project: HBase
  Issue Type: Sub-task
  Components: documentation
Affects Versions: 1.2.0
Reporter: Sean Busbey
Assignee: Sean Busbey
 Fix For: 2.0.0, 1.2.0, 1.3.0

 Attachments: HBASE-13990-branch-1.1.patch, 
 HBASE-13990-branch-1.v1.patch, HBASE-13990-branch-1.v1.patch, 
 HBASE-13990-branch-1.v2.patch, HBASE-13990.1.patch


 maven sit goal still fails with a problem about resolving mockito. work 
 through any remaining issues to get a successful site execution.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13895) DATALOSS: Double-assignment ITBLL'ing


[ 
https://issues.apache.org/jira/browse/HBASE-13895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610647#comment-14610647
 ] 

stack commented on HBASE-13895:
---

What you thinking on this one [~enis]? Thanks.

 DATALOSS: Double-assignment ITBLL'ing
 -

 Key: HBASE-13895
 URL: https://issues.apache.org/jira/browse/HBASE-13895
 Project: HBase
  Issue Type: Bug
Affects Versions: 1.2.0
Reporter: stack
Assignee: stack
Priority: Critical
 Fix For: 2.0.0, 1.1.2, 1.3.0, 1.2.1

 Attachments: hbase-13895_v1-branch-1.1.patch


 Opening a place holder till finish analysis.
 I have dataloss running ITBLL at 3B (testing HBASE-13877). Most obvious 
 culprit is the double-assignment that I can see.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-12148) Remove TimeRangeTracker as point of contention when many threads writing a Store


 [ 
https://issues.apache.org/jira/browse/HBASE-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-12148:

Fix Version/s: (was: 1.2.0)
   1.2.1
   1.3.0
   1.0.2
   Status: Open  (was: Patch Available)

 Remove TimeRangeTracker as point of contention when many threads writing a 
 Store
 

 Key: HBASE-12148
 URL: https://issues.apache.org/jira/browse/HBASE-12148
 Project: HBase
  Issue Type: Sub-task
  Components: Performance
Affects Versions: 0.99.1, 2.0.0
Reporter: stack
Assignee: John Leach
 Fix For: 2.0.0, 0.98.14, 1.0.2, 1.1.2, 1.3.0, 1.2.1

 Attachments: 
 0001-In-AtomicUtils-change-updateMin-and-updateMax-to-ret.patch, 
 12148.addendum.txt, 12148.txt, 12148.txt, 12148v2.txt, 12148v2.txt, 
 HBASE-12148.txt, HBASE-12148V2.txt, Screen Shot 2014-10-01 at 3.39.46 PM.png, 
 Screen Shot 2014-10-01 at 3.41.07 PM.png






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13965) Stochastic Load Balancer JMX Metrics

2015-07-01 Thread Lei Chen (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-13965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610601#comment-14610601
 ] 

Lei Chen commented on HBASE-13965:
--

I agree that the unused balancers should be purged or made into attributes of 
the stochastic load balancer. I think it may be better to do it in another 
Jira, since “One thing at a time”.

About the ever growing map, I’m thinking of two ways to solve this problem. 
1.  Besides updateStochasticCost, add another method (or add a boolean 
parameter) which should be called when the table is deleted. This will allow 
the map to contain only existing tables
2.  Use a fixed-size most recent used (MRU) cache to store the map. The 
size can be configurable. 

Any suggestion?


 Stochastic Load Balancer JMX Metrics
 

 Key: HBASE-13965
 URL: https://issues.apache.org/jira/browse/HBASE-13965
 Project: HBase
  Issue Type: Improvement
  Components: Balancer, metrics
Reporter: Lei Chen
Assignee: Lei Chen
 Attachments: HBase-13965-v1.patch, 
 stochasticloadbalancerclasses_v2.png


 Today’s default HBase load balancer (the Stochastic load balancer) is cost 
 function based. The cost function weights are tunable but no visibility into 
 those cost function results is directly provided.
 A driving example is a cluster we have been tuning which has skewed rack size 
 (one rack has half the nodes of the other few racks). We are tuning the 
 cluster for uniform response time from all region servers with the ability to 
 tolerate a rack failure. Balancing LocalityCost, RegionReplicaRack Cost and 
 RegionCountSkew Cost is difficult without a way to attribute each cost 
 function’s contribution to overall cost. 
 What this jira proposes is to provide visibility via JMX into each cost 
 function of the stochastic load balancer, as well as the overall cost of the 
 balancing plan.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13861) BucketCacheTmpl.jamon has wrong bucket free and used labels


 [ 
https://issues.apache.org/jira/browse/HBASE-13861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-13861:

Resolution: Fixed
Status: Resolved  (was: Patch Available)

pushed to 1.2+. thanks!

 BucketCacheTmpl.jamon has wrong bucket free and used labels
 ---

 Key: HBASE-13861
 URL: https://issues.apache.org/jira/browse/HBASE-13861
 Project: HBase
  Issue Type: Bug
  Components: regionserver, UI
Affects Versions: 1.1.0
Reporter: Lars George
Assignee: Matt Warhaftig
  Labels: beginner
 Fix For: 2.0.0, 1.2.0

 Attachments: hbase-13861-v1.patch


 See this from the template, and note the label and actual values for the last 
 two columns.
 {noformat}
 table class=table table-striped
 tr
 thBucket Offset/th
 thAllocation Size/th
 thFree Count/th
 thUsed Count/th
 /tr
 %for Bucket bucket: buckets %
 tr
 td% bucket.getBaseOffset() %/td
 td% bucket.getItemAllocationSize() %/td
 td% bucket.getFreeBytes() %/td
 td% bucket.getUsedBytes() %/td
 /tr
 {noformat}
 They are labeled counts but are bytes, duh.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13911) add 1.2 to prereq tables in ref guide


[ 
https://issues.apache.org/jira/browse/HBASE-13911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610747#comment-14610747
 ] 

Hudson commented on HBASE-13911:


FAILURE: Integrated in HBase-TRUNK #6619 (See 
[https://builds.apache.org/job/HBase-TRUNK/6619/])
HBASE-13911 update java/hadoop prereqs for 1.2 (busbey: rev 
348a11ad55dc93c40b7f6e3ff8cb71d2e7c6162c)
* src/main/asciidoc/_chapters/configuration.adoc


 add 1.2 to prereq tables in ref guide
 -

 Key: HBASE-13911
 URL: https://issues.apache.org/jira/browse/HBASE-13911
 Project: HBase
  Issue Type: Sub-task
  Components: documentation
Reporter: Sean Busbey
Assignee: Sean Busbey
Priority: Blocker
 Fix For: 2.0.0

 Attachments: HBASE-13911.1.patch, HBASE-13911.2.patch


 update the ref guide to have a column for 1.2.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13998) Remove CellComparator#compareRows(byte[] left, int loffset, int llength, byte[] right, int roffset, int rlength)

2015-07-01 Thread Anoop Sam John (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610779#comment-14610779
 ] 

Anoop Sam John commented on HBASE-13998:


We are trying to find a table region corresponding to give row.  The region's 
end key is compared against the given key. (to make sure this region is 
suitable for the row not the next region).  These will be normal table regions 
and so no issue in directly doing Bytes.compare() call.
{code}
byte[] endKey = possibleRegion.getRegionLocation().getRegionInfo().getEndKey();
if (Bytes.equals(endKey, HConstants.EMPTY_END_ROW) ||
getRowComparator(tableName).compareRows(
  endKey, 0, endKey.length, row, 0, row.length)  0) {
{code}
Even if the region is META table region,  the 1st condition itself will be 
true..  It is single region table and so the end key will be empty byte[ ].
Am I explaining it clear?


 Remove CellComparator#compareRows(byte[] left, int loffset, int llength, 
 byte[] right, int roffset, int rlength)
 

 Key: HBASE-13998
 URL: https://issues.apache.org/jira/browse/HBASE-13998
 Project: HBase
  Issue Type: Sub-task
Reporter: Anoop Sam John
Assignee: Anoop Sam John
 Fix For: 2.0.0

 Attachments: HBASE-13998.patch


 A public API in CellComparator which takes old style byte[], offset, length 
 alone is not correct.  CellComparator supposed to compare cell(s).  At least 
 one side param has to be a cell.. This is the agreement we discussed in 
 HBASE-10800.  Still we could not remove the above one method because it was 
 getting used from multiple places.  Now most of the usage is removed.  This 
 jira aims at removing it fully and replace the usage with other APIs.
 Note: The CellComparator is added in 2.0 only so removing the public API is 
 not creating any BC issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13984) Add option to allow caller to know the heartbeat and scanner position when scanner timeout


[ 
https://issues.apache.org/jira/browse/HBASE-13984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610796#comment-14610796
 ] 

stack commented on HBASE-13984:
---

[~jonathan.lawlor] FYI

So, idea is to pass back in the heartbeat, the 'next' row that would be 
encountered so the Scan can be picked up later at this point?

bq. 1. Allow client set a flag whether pass the heartbeat (a fake row) to the 
caller (via ResultScanner next)

Rather than a global config in hbase Configuration, why is this not an option 
you'd specify on Scan? (What is an 'explicit' heartbeat vs a heartbeat?)

It seems entangled with client not taking partial results? Does it have to be? 
Could someone want the next row AND want partial results?

These 'general' additions made just for the exotic case of an heartbeat 
carrying the next Cell from which to carry on the Scan seems like too much 
exposure on a pivotal class like ScannerCallable

+  /**
+   * @return Return the next cell when the most recent RPC response was a 
heartbeat message and
+   * the server is only allowed break at row boundary. If the next 
cell is beyond the
+   * scan range, null will be returned.
+   */
+  protected Cell getNextCell() {
+return nextCell;
+  }
+
+  protected void setNextCell(Cell nextCell) {
+this.nextCell = nextCell;
+  }

I think it important we get the flagging clean and clear so folks can easily 
discover the existence of this new facility. Would also like to make it so we 
can implement it w/o tarnishing meaning of current flags.

Thanks [~heliangliang]





 Add option to allow caller to know the heartbeat and scanner position when 
 scanner timeout
 --

 Key: HBASE-13984
 URL: https://issues.apache.org/jira/browse/HBASE-13984
 Project: HBase
  Issue Type: Improvement
  Components: Scanners
Reporter: He Liangliang
Assignee: He Liangliang
 Attachments: HBASE-13984-V1.diff


 HBASE-13090 introduced scanner heartbeat. However, there are still some 
 limitations (see HBASE-13215). In some application, for example, an operation 
 access hbase to scan table data, and there is strict limit that this call 
 must return in a fixed interval. At the same time, this call is stateless, so 
 the call must return the next position to continue the scan. This is typical 
 use case for online applications.
 Based on this requirement, some improvements are proposed:
 1. Allow client set a flag whether pass the heartbeat (a fake row) to the 
 caller (via ResultScanner next)
 2. Allow the client pass a timeout to the server, which can override the 
 server side default value
 3. When requested by the client, the server peek the next cell and return to 
 the client in the heartbeat message



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13981) Fix ImportTsv spelling and usage issues


 [ 
https://issues.apache.org/jira/browse/HBASE-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-13981:

Fix Version/s: (was: 1.2.0)
   1.3.0
   Status: Open  (was: Patch Available)

 Fix ImportTsv spelling and usage issues
 ---

 Key: HBASE-13981
 URL: https://issues.apache.org/jira/browse/HBASE-13981
 Project: HBase
  Issue Type: Bug
  Components: mapreduce
Affects Versions: 1.1.0.1
Reporter: Lars George
Assignee: Gabor Liptak
  Labels: beginner
 Fix For: 2.0.0, 1.3.0

 Attachments: HBASE-13981.1.patch


 The {{ImportTsv}} tool has various spelling and formatting issues. Fix those.
 In code:
 {noformat}
   public final static String ATTRIBUTE_SEPERATOR_CONF_KEY = 
 attributes.seperator;
 {noformat}
 It is separator.
 In usage text:
 {noformat}
 input data. Another special columnHBASE_TS_KEY designates that this column 
 should be
 {noformat}
 Space missing.
 {noformat}
 Record with invalid timestamps (blank, non-numeric) will be treated as bad 
 record.
 {noformat}
 Records ... as bad records - plural missing twice.
 {noformat}
 HBASE_ATTRIBUTES_KEY can be used to specify Operation Attributes per record.
  Should be specified as key=value where -1 is used 
  as the seperator.  Note that more than one OperationAttributes can be 
 specified.
 {noformat}
 - Remove line wraps and indentation. 
 - Fix separator.
 - Fix wrong separator being output, it is not -1 (wrong constant use in 
 code)
 - General wording/style could be better (eg. last sentence now uses 
 OperationAttributes without a space).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14006) Hbase Thrift - add num versions to rowWithColumns


 [ 
https://issues.apache.org/jira/browse/HBASE-14006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-14006:
---
Issue Type: Improvement  (was: Bug)

 Hbase Thrift - add num versions to rowWithColumns
 -

 Key: HBASE-14006
 URL: https://issues.apache.org/jira/browse/HBASE-14006
 Project: HBase
  Issue Type: Improvement
Reporter: navdeep agrawal
Priority: Critical

 in thrift if we want column map and also want to pass num versions ,we dont 
 have any function for that . ie no numversion parameter in rowWithColumn and 
 rowsWithColumn methods .
 we can have a function which have columns map and numbversions parameter



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13998) Remove CellComparator#compareRows(byte[] left, int loffset, int llength, byte[] right, int roffset, int rlength)


[ 
https://issues.apache.org/jira/browse/HBASE-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610807#comment-14610807
 ] 

stack commented on HBASE-13998:
---

bq. It is single region table and so the end key will be empty byte[ ].

This will not be always true (hopefully) but it is the case currently. +1 on 
patch. Thanks [~anoop.hbase] Add the above explanation to code on commit.

 Remove CellComparator#compareRows(byte[] left, int loffset, int llength, 
 byte[] right, int roffset, int rlength)
 

 Key: HBASE-13998
 URL: https://issues.apache.org/jira/browse/HBASE-13998
 Project: HBase
  Issue Type: Sub-task
Reporter: Anoop Sam John
Assignee: Anoop Sam John
 Fix For: 2.0.0

 Attachments: HBASE-13998.patch


 A public API in CellComparator which takes old style byte[], offset, length 
 alone is not correct.  CellComparator supposed to compare cell(s).  At least 
 one side param has to be a cell.. This is the agreement we discussed in 
 HBASE-10800.  Still we could not remove the above one method because it was 
 getting used from multiple places.  Now most of the usage is removed.  This 
 jira aims at removing it fully and replace the usage with other APIs.
 Note: The CellComparator is added in 2.0 only so removing the public API is 
 not creating any BC issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13769) Some ZK ACLs are unnecessarily permissive


[ 
https://issues.apache.org/jira/browse/HBASE-13769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610814#comment-14610814
 ] 

Sean Busbey commented on HBASE-13769:
-

did this ticket get covered by HBASE-13768? if not, can we set a target version?

 Some ZK ACLs are unnecessarily permissive
 -

 Key: HBASE-13769
 URL: https://issues.apache.org/jira/browse/HBASE-13769
 Project: HBase
  Issue Type: Bug
Reporter: Andrew Purtell
Priority: Critical

 Some ZK ACLs are unnecessarily permissive. We can remove permissions for 
 'world' on backup-masters/, region-in-transition/, rs/, and table/.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14004) Inconsistency between Memstore and WAL

[
https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

He Liangliang updated HBASE-14004:
--
Description:
Looks like the current write path can cause inconsistency between
memstore/hfile and WAL which cause the slave cluster has more data than the
master cluster.

The simplified write path looks like:
1. insert record into Memstore
2. write record to WAL
3. sync WAL
4. rollback Memstore if 3 fails

It's possible that the HDFS sync RPC call fails, but the data is already (may
partially) transported to the DNs which finally get persisted. As a result, the
handler will rollback the Memstore and the later flushed HFile will also skip
this record.

was:
We encountered data inconsistency issue between master and slave clusters.
Looks like the current write path can cause inconsistency between
memstore/hfile and WAL which cause the slave cluster has more data than the
master cluster.

The simplified write path looks like:
1. insert record into Memstore
2. write record to WAL
3. sync WAL
4. rollback Memstore if 3 fails

In step 3, the sync operation can fail in two case:
1. The caller disconnected the socket due to timeout or any other client crashes
2. The underlying HDFS sync operation can fails

For case 1, the caller may disconnect the socket (for example, timeout due to
slow sync), but the sync can finally succeed because it's an asynchronous
operation. As a result, the handler will rollback the Memstore and the later
flushed HFile will also skip this record.
For case 2, it's possible that the HDFS sync RPC call fails, but the data is
already (may partially) transported to the DNs which finally get persisted.

0.94 code fails in both cases, and looks like the trunk code suffered only case
2.

Inconsistency between Memstore and WAL
--

Key: HBASE-14004
URL: https://issues.apache.org/jira/browse/HBASE-14004
Project: HBase
Issue Type: Bug
Components: regionserver
Reporter: He Liangliang
Priority: Critical

Looks like the current write path can cause inconsistency between
memstore/hfile and WAL which cause the slave cluster has more data than the
master cluster.
The simplified write path looks like:
1. insert record into Memstore
2. write record to WAL
3. sync WAL
4. rollback Memstore if 3 fails
It's possible that the HDFS sync RPC call fails, but the data is already
(may partially) transported to the DNs which finally get persisted. As a
result, the handler will rollback the Memstore and the later flushed HFile
will also skip this record.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (HBASE-13788) Shell comands do not support column qualifiers containing colon (:)

2015-07-01 Thread Pankaj Kumar (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-13788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pankaj Kumar reassigned HBASE-13788:


Assignee: Pankaj Kumar

 Shell comands do not support column qualifiers containing colon (:)
 ---

 Key: HBASE-13788
 URL: https://issues.apache.org/jira/browse/HBASE-13788
 Project: HBase
  Issue Type: Bug
  Components: shell
Affects Versions: 0.98.0, 0.96.0, 1.0.0, 1.1.0
Reporter: Dave Latham
Assignee: Pankaj Kumar

 The shell interprets the colon within the qualifier as a delimiter to a 
 FORMATTER instead of part of the qualifier itself.
 Example from the mailing list:
 Hmph, I may have spoken too soon. I know I tested this at one point and
 it worked, but now I'm getting different results:
 On the new cluster, I created a duplicate test table:
 hbase(main):043:0 create 'content3', {NAME = 'x', BLOOMFILTER =
 'NONE', REPLICATION_SCOPE = '0', VERSIONS = '3', COMPRESSION =
 'NONE', MIN_VERSIONS = '0', TTL = '2147483647', BLOCKSIZE = '65536',
 IN_MEMORY = 'false', BLOCKCACHE = 'true'}
 Then I pull some data from the imported table:
 hbase(main):045:0 scan 'content', {LIMIT=1,
 STARTROW='A:9223370612089311807:twtr:57013379'}
 ROW  COLUMN+CELL
 
 A:9223370612089311807:twtr:570133798827921408
 column=x:twitter:username, timestamp=1424775595345, value=BERITA 
 INFORMASI!
 Then put it:
 hbase(main):046:0 put
 'content3','A:9223370612089311807:twtr:570133798827921408',
 'x:twitter:username', 'BERITA  INFORMASI!'
 But then when I query it, I see that I've lost the column qualifier
 :username:
 hbase(main):046:0 scan 'content3'
 ROW  COLUMN+CELL
  A:9223370612089311807:twtr:570133798827921408 column=x:twitter,
  timestamp=1432745301788, value=BERITA  INFORMASI!
 Even though I'm missing one of the qualifiers, I can at least filter on
 columns in this sample table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-14005) Set permission to .top hfile in LoadIncrementalHFiles

Francesco MDE created HBASE-14005:
-

 Summary: Set permission to .top hfile in LoadIncrementalHFiles
 Key: HBASE-14005
 URL: https://issues.apache.org/jira/browse/HBASE-14005
 Project: HBase
  Issue Type: Bug
Reporter: Francesco MDE
Priority: Trivial


Set the same -rwxrwxrwx permission to .top file as .bottom and _tmp



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14005) Set permission to .top hfile in LoadIncrementalHFiles


 [ 
https://issues.apache.org/jira/browse/HBASE-14005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Francesco MDE updated HBASE-14005:
--
Status: Patch Available  (was: Open)

 Set permission to .top hfile in LoadIncrementalHFiles
 -

 Key: HBASE-14005
 URL: https://issues.apache.org/jira/browse/HBASE-14005
 Project: HBase
  Issue Type: Bug
Reporter: Francesco MDE
Priority: Trivial
 Attachments: HBASE-14005.patch


 Set the same -rwxrwxrwx permission to .top file as .bottom and _tmp



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13975) add 1.2 RM to docs


[ 
https://issues.apache.org/jira/browse/HBASE-13975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610232#comment-14610232
 ] 

Sean Busbey commented on HBASE-13975:
-

bump. test failure is unrelated.

 add 1.2 RM to docs
 --

 Key: HBASE-13975
 URL: https://issues.apache.org/jira/browse/HBASE-13975
 Project: HBase
  Issue Type: Sub-task
  Components: documentation
Affects Versions: 1.2.0
Reporter: Sean Busbey
Assignee: Sean Busbey
 Fix For: 2.0.0, 1.2.0, 1.3.0

 Attachments: HBASE-13975.1.patch






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13990) clean up remaining errors for maven site goal


 [ 
https://issues.apache.org/jira/browse/HBASE-13990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-13990:

Resolution: Fixed
Status: Resolved  (was: Patch Available)

 clean up remaining errors for maven site goal
 -

 Key: HBASE-13990
 URL: https://issues.apache.org/jira/browse/HBASE-13990
 Project: HBase
  Issue Type: Sub-task
  Components: documentation
Affects Versions: 1.2.0
Reporter: Sean Busbey
Assignee: Sean Busbey
 Fix For: 2.0.0, 1.2.0, 1.3.0

 Attachments: HBASE-13990-branch-1.1.patch, 
 HBASE-13990-branch-1.v1.patch, HBASE-13990-branch-1.v1.patch, 
 HBASE-13990-branch-1.v2.patch, HBASE-13990.1.patch


 maven sit goal still fails with a problem about resolving mockito. work 
 through any remaining issues to get a successful site execution.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13747) Promote Java 8 to yes in support matrix


 [ 
https://issues.apache.org/jira/browse/HBASE-13747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-13747:

Fix Version/s: 1.3.0
   2.0.0

 Promote Java 8 to yes in support matrix
 -

 Key: HBASE-13747
 URL: https://issues.apache.org/jira/browse/HBASE-13747
 Project: HBase
  Issue Type: Umbrella
Affects Versions: 1.2.0
Reporter: Sean Busbey
Assignee: Sean Busbey
Priority: Critical
 Fix For: 2.0.0, 1.2.0, 1.3.0


 Now that Java 7 is EOL, we need to move to formally supporting Java 8. Let's 
 use this ticket to track efforts needed to stop caveating our support table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HBASE-13747) Promote Java 8 to yes in support matrix


 [ 
https://issues.apache.org/jira/browse/HBASE-13747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey resolved HBASE-13747.
-
  Resolution: Fixed
Release Note: Java 8 is considered supported and tested as of HBase 1.2+

It's now possible to get through an entire make_rc.sh run with JDK8 as the 
target JVM, so I'm closing this out. Master and branch-1 are likely to drift 
from that due to the tighter javadoc requirements until we can get multi-jdk 
precommit tests in (probably late July), so something to watch out for on later 
releases.

 Promote Java 8 to yes in support matrix
 -

 Key: HBASE-13747
 URL: https://issues.apache.org/jira/browse/HBASE-13747
 Project: HBase
  Issue Type: Umbrella
Affects Versions: 1.2.0
Reporter: Sean Busbey
Assignee: Sean Busbey
Priority: Critical
 Fix For: 1.2.0


 Now that Java 7 is EOL, we need to move to formally supporting Java 8. Let's 
 use this ticket to track efforts needed to stop caveating our support table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12865) WALs may be deleted before they are replicated to peers


[ 
https://issues.apache.org/jira/browse/HBASE-12865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610294#comment-14610294
 ] 

Hadoop QA commented on HBASE-12865:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12743047/HBASE-12865-V2.diff
  against master branch at commit 85c278a6a8b25ff86e22c254ffec35e945cd7c66.
  ATTACHMENT ID: 12743047

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 6 new 
or modified tests.

{color:green}+1 hadoop versions{color}. The patch compiles with all 
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 protoc{color}.  The applied patch does not increase the 
total number of protoc compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 checkstyle{color}.  The applied patch does not increase the 
total number of checkstyle errors

{color:green}+1 findbugs{color}.  The patch does not introduce any  new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 lineLengths{color}.  The patch does not introduce lines 
longer than 100

  {color:green}+1 site{color}.  The mvn post-site goal succeeds with this patch.

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
   org.apache.hadoop.hbase.util.TestProcessBasedCluster
  org.apache.hadoop.hbase.mapreduce.TestImportExport
  org.apache.hadoop.hbase.TestRegionRebalancing

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/14639//testReport/
Release Findbugs (version 2.0.3)warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/14639//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors: 
https://builds.apache.org/job/PreCommit-HBASE-Build/14639//artifact/patchprocess/checkstyle-aggregate.html

  Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/14639//console

This message is automatically generated.

 WALs may be deleted before they are replicated to peers
 ---

 Key: HBASE-12865
 URL: https://issues.apache.org/jira/browse/HBASE-12865
 Project: HBase
  Issue Type: Bug
  Components: Replication
Reporter: Liu Shaohui
Assignee: He Liangliang
Priority: Critical
 Attachments: HBASE-12865-V1.diff, HBASE-12865-V2.diff


 By design, ReplicationLogCleaner guarantee that the WALs  being in 
 replication queue can't been deleted by the HMaster. The 
 ReplicationLogCleaner gets the WAL set from zookeeper by scanning the 
 replication zk node. But it may get uncompleted WAL set during replication 
 failover for the scan operation is not atomic.
 For example: There are three region servers: rs1, rs2, rs3, and peer id 10.  
 The layout of replication zookeeper nodes is:
 {code}
 /hbase/replication/rs/rs1/10/wals
  /rs2/10/wals
  /rs3/10/wals
 {code}
 - t1: the ReplicationLogCleaner finished scanning the replication queue of 
 rs1, and start to scan the queue of rs2.
 - t2: region server rs3 is down, and rs1 take over rs3's replication queue. 
 The new layout is
 {code}
 /hbase/replication/rs/rs1/10/wals
  /rs1/10-rs3/wals
  /rs2/10/wals
  /rs3
 {code}
 - t3, the ReplicationLogCleaner finished scanning the queue of rs2, and start 
 to scan the node of rs3. But the the queue has been moved to  
 replication/rs1/10-rs3/WALS
 So the  ReplicationLogCleaner will miss the WALs of rs3 in peer 10 and the 
 hmaster may delete these WALs before they are replicated to peer clusters.
 We encountered this problem in our cluster and I think it's a serious bug for 
 replication.
 Suggestions are welcomed to fix this bug. thx~



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14005) Set permission to .top hfile in LoadIncrementalHFiles


 [ 
https://issues.apache.org/jira/browse/HBASE-14005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Francesco MDE updated HBASE-14005:
--
Flags:   (was: Patch)

 Set permission to .top hfile in LoadIncrementalHFiles
 -

 Key: HBASE-14005
 URL: https://issues.apache.org/jira/browse/HBASE-14005
 Project: HBase
  Issue Type: Bug
Reporter: Francesco MDE
Priority: Trivial

 Set the same -rwxrwxrwx permission to .top file as .bottom and _tmp



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13990) clean up remaining errors for maven site goal


 [ 
https://issues.apache.org/jira/browse/HBASE-13990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-13990:

Attachment: HBASE-13990-branch-1.v2.patch

attaching the patch that was actualy applied to branch-1. It's a squash of the 
patches in v1 plus 2 additional lines that were incorrect backports from 
HBASE-13898.

 clean up remaining errors for maven site goal
 -

 Key: HBASE-13990
 URL: https://issues.apache.org/jira/browse/HBASE-13990
 Project: HBase
  Issue Type: Sub-task
  Components: documentation
Affects Versions: 1.2.0
Reporter: Sean Busbey
Assignee: Sean Busbey
 Fix For: 2.0.0, 1.2.0, 1.3.0

 Attachments: HBASE-13990-branch-1.1.patch, 
 HBASE-13990-branch-1.v1.patch, HBASE-13990-branch-1.v1.patch, 
 HBASE-13990-branch-1.v2.patch, HBASE-13990.1.patch


 maven sit goal still fails with a problem about resolving mockito. work 
 through any remaining issues to get a successful site execution.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14005) Set permission to .top hfile in LoadIncrementalHFiles


 [ 
https://issues.apache.org/jira/browse/HBASE-14005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Francesco MDE updated HBASE-14005:
--
Attachment: HBASE-14005.patch

 Set permission to .top hfile in LoadIncrementalHFiles
 -

 Key: HBASE-14005
 URL: https://issues.apache.org/jira/browse/HBASE-14005
 Project: HBase
  Issue Type: Bug
Reporter: Francesco MDE
Priority: Trivial
 Attachments: HBASE-14005.patch


 Set the same -rwxrwxrwx permission to .top file as .bottom and _tmp



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13911) add 1.2 to prereq tables in ref guide


 [ 
https://issues.apache.org/jira/browse/HBASE-13911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-13911:

   Resolution: Fixed
Fix Version/s: (was: 1.2.0)
   2.0.0
   Status: Resolved  (was: Patch Available)

 add 1.2 to prereq tables in ref guide
 -

 Key: HBASE-13911
 URL: https://issues.apache.org/jira/browse/HBASE-13911
 Project: HBase
  Issue Type: Sub-task
  Components: documentation
Reporter: Sean Busbey
Assignee: Sean Busbey
Priority: Blocker
 Fix For: 2.0.0

 Attachments: HBASE-13911.1.patch, HBASE-13911.2.patch


 update the ref guide to have a column for 1.2.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-12452) Add regular expression based split policy


 [ 
https://issues.apache.org/jira/browse/HBASE-12452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

He Liangliang updated HBASE-12452:
--
Attachment: HBASE-12452-V4.patch

retry

 Add regular expression based split policy
 -

 Key: HBASE-12452
 URL: https://issues.apache.org/jira/browse/HBASE-12452
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Reporter: He Liangliang
Assignee: He Liangliang
Priority: Minor
 Attachments: HBASE-12452-V2.patch, HBASE-12452-V2.patch, 
 HBASE-12452-V3.patch, HBASE-12452-V3.patch, HBASE-12452-V4.patch, 
 HBASE-12452-V4.patch, HBASE-12452.patch


 The current DelimitedKeyPrefixRegionSplitPolicy split policy is not flexible 
 enough to describe the split point prefix in some case. A regex based split 
 policy is proposed, for example:
 ^[^\x00]+\x00[^\x00]+\x00
 means the split point will always be truncated to a prefix at the second \0 
 char.
 The binary string support is quite useful when the rowkey encoded by a common 
 data access library instead of concatenated manually by application developer.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-8642) [Snapshot] List and delete snapshot by table


[ 
https://issues.apache.org/jira/browse/HBASE-8642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14609849#comment-14609849
 ] 

Hadoop QA commented on HBASE-8642:
--

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12743016/HBASE-8642-v3.patch
  against master branch at commit 85c278a6a8b25ff86e22c254ffec35e945cd7c66.
  ATTACHMENT ID: 12743016

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 4 new 
or modified tests.

{color:green}+1 hadoop versions{color}. The patch compiles with all 
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 protoc{color}.  The applied patch does not increase the 
total number of protoc compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 checkstyle{color}.  The applied patch does not increase the 
total number of checkstyle errors

{color:green}+1 findbugs{color}.  The patch does not introduce any  new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 lineLengths{color}.  The patch introduces the following lines 
longer than 100:
+puts No snapshots matched the table name regular expression 
#{tableNameregex.to_s} and the snapshot name regular expression 
#{snapshotNameRegex.to_s} if count == 0
+puts #{successfullyDeleted} snapshots successfully deleted. unless 
successfullyDeleted == 0

  {color:green}+1 site{color}.  The mvn post-site goal succeeds with this patch.

{color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/14637//testReport/
Release Findbugs (version 2.0.3)warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/14637//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors: 
https://builds.apache.org/job/PreCommit-HBASE-Build/14637//artifact/patchprocess/checkstyle-aggregate.html

  Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/14637//console

This message is automatically generated.

 [Snapshot] List and delete snapshot by table
 

 Key: HBASE-8642
 URL: https://issues.apache.org/jira/browse/HBASE-8642
 Project: HBase
  Issue Type: Improvement
  Components: snapshots
Affects Versions: 0.98.0, 0.95.0, 0.95.1, 0.95.2
Reporter: Julian Zhou
Assignee: Ashish Singhi
 Fix For: 2.0.0

 Attachments: 8642-trunk-0.95-v0.patch, 8642-trunk-0.95-v1.patch, 
 8642-trunk-0.95-v2.patch, HBASE-8642-v1.patch, HBASE-8642-v2.patch, 
 HBASE-8642-v3.patch, HBASE-8642.patch


 Support list and delete snapshots by table names.
 User scenario:
 A user wants to delete all the snapshots which were taken in January month 
 for a table 't' where snapshot names starts with 'Jan'.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-14004) Inconsistency between Memstore and WAL

He Liangliang created HBASE-14004:
-

 Summary: Inconsistency between Memstore and WAL
 Key: HBASE-14004
 URL: https://issues.apache.org/jira/browse/HBASE-14004
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Reporter: He Liangliang
Priority: Critical


We encountered data inconsistency issue between master and slave clusters. 
Looks like the current write path can cause inconsistency between 
memstore/hfile and WAL which cause the slave cluster has more data than the 
master cluster.

The simplified write path looks like:
1. insert record into Memstore
2. write record to WAL
3. sync WAL
4. rollback Memstore if 3 fails

In step 3, the sync operation can fail in two case:
1. The caller disconnected the socket due to timeout or any other client crashes
2. The underlying HDFS sync operation can fails

For case 1, the caller may disconnect the socket (for example, timeout due to 
slow sync), but the sync can finally succeed because it's an asynchronous 
operation. As a result, the handler will rollback the Memstore and the later 
flushed HFile will also skip this record.
For case 2, it's possible that the HDFS sync RPC call fails, but the data is 
already  (may partially) transported to the DNs which finally get persisted.

0.94 code fails in both cases, and looks like the trunk code suffered only case 
2.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-12865) WALs may be deleted before they are replicated to peers

[
https://issues.apache.org/jira/browse/HBASE-12865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

He Liangliang updated HBASE-12865:
--
Attachment: HBASE-12865-V2.diff

updated and retry

WALs may be deleted before they are replicated to peers
---

Key: HBASE-12865
URL: https://issues.apache.org/jira/browse/HBASE-12865
Project: HBase
Issue Type: Bug
Components: Replication
Reporter: Liu Shaohui
Assignee: He Liangliang
Priority: Critical
Attachments: HBASE-12865-V1.diff, HBASE-12865-V2.diff

By design, ReplicationLogCleaner guarantee that the WALs being in
replication queue can't been deleted by the HMaster. The
ReplicationLogCleaner gets the WAL set from zookeeper by scanning the
replication zk node. But it may get uncompleted WAL set during replication
failover for the scan operation is not atomic.
For example: There are three region servers: rs1, rs2, rs3, and peer id 10.
The layout of replication zookeeper nodes is:
{code}
/hbase/replication/rs/rs1/10/wals
/rs2/10/wals
/rs3/10/wals
{code}
- t1: the ReplicationLogCleaner finished scanning the replication queue of
rs1, and start to scan the queue of rs2.
- t2: region server rs3 is down, and rs1 take over rs3's replication queue.
The new layout is
{code}
/hbase/replication/rs/rs1/10/wals
/rs1/10-rs3/wals
/rs2/10/wals
/rs3
{code}
- t3, the ReplicationLogCleaner finished scanning the queue of rs2, and start
to scan the node of rs3. But the the queue has been moved to
replication/rs1/10-rs3/WALS
So the ReplicationLogCleaner will miss the WALs of rs3 in peer 10 and the
hmaster may delete these WALs before they are replicated to peer clusters.
We encountered this problem in our cluster and I think it's a serious bug for
replication.
Suggestions are welcomed to fix this bug. thx~

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12865) WALs may be deleted before they are replicated to peers

[
https://issues.apache.org/jira/browse/HBASE-12865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14609959#comment-14609959
]

He Liangliang commented on HBASE-12865:
---

Thanks for the comments, updated the patch.

WALs may be deleted before they are replicated to peers
---

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12452) Add regular expression based split policy


[ 
https://issues.apache.org/jira/browse/HBASE-12452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610033#comment-14610033
 ] 

Hadoop QA commented on HBASE-12452:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12743033/HBASE-12452-V4.patch
  against master branch at commit 85c278a6a8b25ff86e22c254ffec35e945cd7c66.
  ATTACHMENT ID: 12743033

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 3 new 
or modified tests.

{color:green}+1 hadoop versions{color}. The patch compiles with all 
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 protoc{color}.  The applied patch does not increase the 
total number of protoc compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 checkstyle{color}.  The applied patch does not increase the 
total number of checkstyle errors

{color:green}+1 findbugs{color}.  The patch does not introduce any  new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 lineLengths{color}.  The patch does not introduce lines 
longer than 100

  {color:green}+1 site{color}.  The mvn post-site goal succeeds with this patch.

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
 

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/14638//testReport/
Release Findbugs (version 2.0.3)warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/14638//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors: 
https://builds.apache.org/job/PreCommit-HBASE-Build/14638//artifact/patchprocess/checkstyle-aggregate.html

  Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/14638//console

This message is automatically generated.

 Add regular expression based split policy
 -

 Key: HBASE-12452
 URL: https://issues.apache.org/jira/browse/HBASE-12452
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Reporter: He Liangliang
Assignee: He Liangliang
Priority: Minor
 Attachments: HBASE-12452-V2.patch, HBASE-12452-V2.patch, 
 HBASE-12452-V3.patch, HBASE-12452-V3.patch, HBASE-12452-V4.patch, 
 HBASE-12452-V4.patch, HBASE-12452.patch


 The current DelimitedKeyPrefixRegionSplitPolicy split policy is not flexible 
 enough to describe the split point prefix in some case. A regex based split 
 policy is proposed, for example:
 ^[^\x00]+\x00[^\x00]+\x00
 means the split point will always be truncated to a prefix at the second \0 
 char.
 The binary string support is quite useful when the rowkey encoded by a common 
 data access library instead of concatenated manually by application developer.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12865) WALs may be deleted before they are replicated to peers

[
https://issues.apache.org/jira/browse/HBASE-12865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610381#comment-14610381
]

Ted Yu commented on HBASE-12865:

[~lhofhansl]:
What do you think of the latest patch ?

WALs may be deleted before they are replicated to peers
---

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-12943) Set sun.net.inetaddr.ttl in HBase


 [ 
https://issues.apache.org/jira/browse/HBASE-12943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-12943:

Fix Version/s: (was: 1.2.0)
   1.2.1
   1.3.0
   Status: Open  (was: Patch Available)

moving out of patch available status, since the patch is for the issue fixedi n 
the subtask and not the subject of this jira.

 Set sun.net.inetaddr.ttl in HBase
 -

 Key: HBASE-12943
 URL: https://issues.apache.org/jira/browse/HBASE-12943
 Project: HBase
  Issue Type: Bug
Reporter: Liu Shaohui
Assignee: Liu Shaohui
 Fix For: 2.0.0, 1.3.0, 1.2.1

 Attachments: 12943-1-master.txt


 The default value of config: sun.net.inetaddr.ttl is -1 and the java 
 processes will cache the mapping of hostname to ip address  forever, See: 
 http://docs.oracle.com/javase/7/docs/technotes/guides/net/properties.html
 But things go wrong when a regionserver with same hostname and different ip 
 address rejoins the hbase cluster. The HMaster will get wrong ip address of 
 the regionserver from this cache and every region assignment to this 
 regionserver will be blocked for a time because the HMaster can't communicate 
 with the regionserver.
 A tradeoff is to set the sun.net.inetaddr.ttl to 10m or 1h and make the wrong 
 cache expired.
 Suggestions are welcomed. Thanks~



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13452) HRegion warning about memstore size miscalculation is not actionable


 [ 
https://issues.apache.org/jira/browse/HBASE-13452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-13452:

Fix Version/s: (was: 1.2.0)
   1.2.1

bumping out of 1.2.0.

 HRegion warning about memstore size miscalculation is not actionable
 

 Key: HBASE-13452
 URL: https://issues.apache.org/jira/browse/HBASE-13452
 Project: HBase
  Issue Type: Bug
Affects Versions: 1.0.0
Reporter: Dev Lakhani
Assignee: Mikhail Antonov
Priority: Critical
 Fix For: 2.0.0, 1.0.2, 1.1.2, 1.2.1


 During normal operation the HRegion class reports a message related to 
 memstore flushing in HRegion.class :
   if (!canFlush) {
 addAndGetGlobalMemstoreSize(-memstoreSize.get());
   } else if (memstoreSize.get() != 0) {
 LOG.error(Memstore size is  + memstoreSize.get());
   }
 The log file is filled with lots of 
 Memstore size is 558744
 Memstore size is 4390632
 Memstore size is 558744 
 ...
 These message are uninformative, clog up the logs and offers no root cause 
 nor solution. Maybe the message needs to be more informative, changed to WARN 
 or some further information provided.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13844) Move static helper methods from KeyValue into CellUtils


 [ 
https://issues.apache.org/jira/browse/HBASE-13844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-13844:

 Assignee: Gabor Liptak
Fix Version/s: (was: 1.2.0)
   1.3.0
   Status: Open  (was: Patch Available)

 Move static helper methods from KeyValue into CellUtils
 ---

 Key: HBASE-13844
 URL: https://issues.apache.org/jira/browse/HBASE-13844
 Project: HBase
  Issue Type: Bug
Affects Versions: 1.1.0
Reporter: Lars George
Assignee: Gabor Liptak
  Labels: beginner
 Fix For: 2.0.0, 1.3.0

 Attachments: HBASE-13844.1.patch, HBASE-13844.2.patch


 Add KeyValue.parseColumn() to CellUtils (also any other public static helper)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13996) Add write sniffing in canary


 [ 
https://issues.apache.org/jira/browse/HBASE-13996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-13996:

Fix Version/s: (was: 1.1.1)

 Add write sniffing in canary
 

 Key: HBASE-13996
 URL: https://issues.apache.org/jira/browse/HBASE-13996
 Project: HBase
  Issue Type: New Feature
  Components: canary
Affects Versions: 0.98.13, 1.1.0.1
Reporter: Liu Shaohui
Assignee: Liu Shaohui
 Fix For: 2.0.0, 0.98.14, 1.1.2

 Attachments: HBASE-13996-v001.diff


 Currently the canary tool only sniff the read operations, it's hard to 
 finding the problem in write path. 
 To support the write sniffing, we create a system table named '_canary_'  in 
 the canary tool. And the tool will make sure that the region number is large 
 than the number of the regionserver and the regions will be distributed onto 
 all regionservers.
 Periodically, the tool will put data to these regions to calculate the write 
 availability of HBase and send alerts if needed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13990) clean up remaining errors for maven site goal


[ 
https://issues.apache.org/jira/browse/HBASE-13990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610472#comment-14610472
 ] 

Hudson commented on HBASE-13990:


FAILURE: Integrated in HBase-TRUNK #6618 (See 
[https://builds.apache.org/job/HBase-TRUNK/6618/])
HBASE-13990 make maven site generation work with jdk8 (busbey: rev 
6ca337cb3eb7496077285c7d9c7ab14c73735ac0)
* hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SimpleRegionNormalizer.java
* pom.xml


 clean up remaining errors for maven site goal
 -

 Key: HBASE-13990
 URL: https://issues.apache.org/jira/browse/HBASE-13990
 Project: HBase
  Issue Type: Sub-task
  Components: documentation
Affects Versions: 1.2.0
Reporter: Sean Busbey
Assignee: Sean Busbey
 Fix For: 2.0.0, 1.2.0, 1.3.0

 Attachments: HBASE-13990-branch-1.1.patch, 
 HBASE-13990-branch-1.v1.patch, HBASE-13990-branch-1.v1.patch, 
 HBASE-13990-branch-1.v2.patch, HBASE-13990.1.patch


 maven sit goal still fails with a problem about resolving mockito. work 
 through any remaining issues to get a successful site execution.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13339) Update default Hadoop version to 2.6.0


[ 
https://issues.apache.org/jira/browse/HBASE-13339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610488#comment-14610488
 ] 

Sean Busbey commented on HBASE-13339:
-

I'd prefer not to kick a Blocker issue over another minor version number. 
[~eclark] do you want this in 1.2 or can I drop to Critical when bumping it to 
1.3?

 Update default Hadoop version to 2.6.0
 --

 Key: HBASE-13339
 URL: https://issues.apache.org/jira/browse/HBASE-13339
 Project: HBase
  Issue Type: Bug
  Components: build
Affects Versions: 2.0.0, 1.1.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Priority: Blocker
 Fix For: 2.0.0, 1.2.0

 Attachments: HBASE-13339-v1.patch, HBASE-13339.patch


 Current default Hadoop version is getting a little long in the tooth. We 
 should update to the latest version. The latest version is backwards 
 compatible with 2.5.1's dfs and mr so this should be painless.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13997) ScannerCallableWithReplicas cause Infinitely blocking


 [ 
https://issues.apache.org/jira/browse/HBASE-13997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-13997:
---
Status: Open  (was: Patch Available)

 ScannerCallableWithReplicas cause Infinitely blocking
 -

 Key: HBASE-13997
 URL: https://issues.apache.org/jira/browse/HBASE-13997
 Project: HBase
  Issue Type: Bug
  Components: Client
Affects Versions: 1.0.1.1
Reporter: Zephyr Guo
Assignee: Zephyr Guo
Priority: Minor

 Bug in ScannerCallableWithReplicas.addCallsForOtherReplicas method  
 {code:title=code in ScannerCallableWithReplicas.addCallsForOtherReplicas 
 |borderStyle=solid}
 private int addCallsForOtherReplicas(
   BoundedCompletionServicePairResult[], ScannerCallable cs, 
 RegionLocations rl, int min,
   int max) {
 if (scan.getConsistency() == Consistency.STRONG) {
   return 0; // not scheduling on other replicas for strong consistency
 }
 for (int id = min; id = max; id++) {
   if (currentScannerCallable.getHRegionInfo().getReplicaId() == id) {
 continue; //this was already scheduled earlier
   }
   ScannerCallable s = 
 currentScannerCallable.getScannerCallableForReplica(id);
   if (this.lastResult != null) {
 s.getScan().setStartRow(this.lastResult.getRow());
   }
   outstandingCallables.add(s);
   RetryingRPC retryingOnReplica = new RetryingRPC(s);
   cs.submit(retryingOnReplica);
 }
 return max - min + 1; //bug? should be max - min,because continue
 //always happen once
   }
 {code}
 It can cause completed  submitted always so that the following code will be 
 infinitely blocked.
 {code:title=code in ScannerCallableWithReplicas.call|borderStyle=solid}
 // submitted larger than the actual one
  submitted += addCallsForOtherReplicas(cs, rl, 0, rl.size() - 1);
 try {
   //here will be affected
   while (completed  submitted) {
 try {
   FuturePairResult[], ScannerCallable f = cs.take();
   PairResult[], ScannerCallable r = f.get();
   if (r != null  r.getSecond() != null) {
 updateCurrentlyServingReplica(r.getSecond(), r.getFirst(), done, 
 pool);
   }
   return r == null ? null : r.getFirst(); // great we got an answer
 } catch (ExecutionException e) {
   // if not cancel or interrupt, wait until all RPC's are done
   // one of the tasks failed. Save the exception for later.
   if (exceptions == null) exceptions = new 
 ArrayListExecutionException(rl.size());
   exceptions.add(e);
   completed++;
 }
   }
 } catch (CancellationException e) {
   throw new InterruptedIOException(e.getMessage());
 } catch (InterruptedException e) {
   throw new InterruptedIOException(e.getMessage());
 } finally {
   // We get there because we were interrupted or because one or more of 
 the
   // calls succeeded or failed. In all case, we stop all our tasks.
   cs.cancelAll(true);
 }
 {code}
 If all replica-RS occur ExecutionException ,it will be infinitely blocked in  
 cs.take()



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13895) DATALOSS: Double-assignment ITBLL'ing


 [ 
https://issues.apache.org/jira/browse/HBASE-13895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-13895:

Fix Version/s: (was: 1.2.0)
   1.2.1
   1.3.0

bumping out of 1.2.0

 DATALOSS: Double-assignment ITBLL'ing
 -

 Key: HBASE-13895
 URL: https://issues.apache.org/jira/browse/HBASE-13895
 Project: HBase
  Issue Type: Bug
Affects Versions: 1.2.0
Reporter: stack
Assignee: stack
Priority: Critical
 Fix For: 2.0.0, 1.1.2, 1.3.0, 1.2.1

 Attachments: hbase-13895_v1-branch-1.1.patch


 Opening a place holder till finish analysis.
 I have dataloss running ITBLL at 3B (testing HBASE-13877). Most obvious 
 culprit is the double-assignment that I can see.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-8073) HFileOutputFormat support for offline operation

[
https://issues.apache.org/jira/browse/HBASE-8073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sean Busbey updated HBASE-8073:
---
Fix Version/s: (was: 1.2.0)
1.3.0
Status: Open (was: Patch Available)

HFileOutputFormat support for offline operation
---

Key: HBASE-8073
URL: https://issues.apache.org/jira/browse/HBASE-8073
Project: HBase
Issue Type: Sub-task
Components: mapreduce
Reporter: Nick Dimiduk
Fix For: 2.0.0, 1.3.0

Attachments: HBASE-8073-trunk-v0.patch, HBASE-8073-trunk-v1.patch

When using HFileOutputFormat to generate HFiles, it inspects the region
topology of the target table. The split points from that table are used to
guide the TotalOrderPartitioner. If the target table does not exist, it is
first created. This imposes an unnecessary dependence on an online HBase and
existing table.
If the table exists, it can be used. However, the job can be smarter. For
example, if there's far more data going into the HFiles than the table
currently contains, the table regions aren't very useful for data split
points. Instead, the input data can be sampled to produce split points more
meaningful to the dataset. LoadIncrementalHFiles is already capable of
handling divergence between HFile boundaries and table regions, so this
should not pose any additional burdon at load time.
The proper method of sampling the data likely requires a custom input format
and an additional map-reduce job perform the sampling. See a relevant
implementation:
https://github.com/alexholmes/hadoop-book/blob/master/src/main/java/com/manning/hip/ch4/sampler/ReservoirSamplerInputFormat.java

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13995) ServerName is not fully case insensitive


 [ 
https://issues.apache.org/jira/browse/HBASE-13995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-13995:

Fix Version/s: (was: 1.1.1)
   1.1.2

 ServerName is not fully case insensitive
 

 Key: HBASE-13995
 URL: https://issues.apache.org/jira/browse/HBASE-13995
 Project: HBase
  Issue Type: Bug
  Components: Region Assignment
Affects Versions: 2.0.0, 1.2.0, 0.98.12.1, 1.0.1.1, 1.1.0.1
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: 2.0.0, 0.98.14, 1.0.2, 1.2.0, 1.1.2

 Attachments: HBASE-13995-v0.patch, HBASE-13995-v0.patch


 we ended up with two ServerName with different cases, AAA and aaa.
 Trying to create a table, every once in a while, we ended up with the region 
 lost and not assigned. 
 BaseLoadBalancer.roundRobinAssignment() goes through each server and create a 
 map with what to assign to them.
 We had to server on the list AAA and aaa which are the same machine, the 
 problem is that the round robin now is assigning an empty list to one of the 
 two. so depending on the order we ended up with a region not assigned.
 ServerName equals() does the case insensitive comparison but the hashCode() 
 is done on a case sensitive server name, so the Map in ServerManager will 
 never hit the item and compare it using equals, so we end up with two entries 
 that are the same server. similar thing for 
 ServerName.isSameHostnameAndPort() where we don't check for cases



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14005) Set permission to .top hfile in LoadIncrementalHFiles


[ 
https://issues.apache.org/jira/browse/HBASE-14005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610483#comment-14610483
 ] 

Ted Yu commented on HBASE-14005:


lgtm

 Set permission to .top hfile in LoadIncrementalHFiles
 -

 Key: HBASE-14005
 URL: https://issues.apache.org/jira/browse/HBASE-14005
 Project: HBase
  Issue Type: Bug
Reporter: Francesco MDE
Priority: Trivial
 Attachments: HBASE-14005.patch


 Set the same -rwxrwxrwx permission to .top file as .bottom and _tmp



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13844) Move static helper methods from KeyValue into CellUtils


 [ 
https://issues.apache.org/jira/browse/HBASE-13844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-13844:

Status: Patch Available  (was: Open)

 Move static helper methods from KeyValue into CellUtils
 ---

 Key: HBASE-13844
 URL: https://issues.apache.org/jira/browse/HBASE-13844
 Project: HBase
  Issue Type: Bug
Affects Versions: 1.1.0
Reporter: Lars George
Assignee: Gabor Liptak
  Labels: beginner
 Fix For: 2.0.0, 1.3.0

 Attachments: HBASE-13844.1.patch, HBASE-13844.2.patch


 Add KeyValue.parseColumn() to CellUtils (also any other public static helper)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13980) Distinguish blockedFlushCount vs unblockedFlushCount when tuning heap memory


[ 
https://issues.apache.org/jira/browse/HBASE-13980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610827#comment-14610827
 ] 

Hadoop QA commented on HBASE-13980:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12743110/HBASE-13980-v1.patch
  against master branch at commit 348a11ad55dc93c40b7f6e3ff8cb71d2e7c6162c.
  ATTACHMENT ID: 12743110

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 3 new 
or modified tests.

{color:green}+1 hadoop versions{color}. The patch compiles with all 
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 protoc{color}.  The applied patch does not increase the 
total number of protoc compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 checkstyle{color}.  The applied patch does not increase the 
total number of checkstyle errors

{color:green}+1 findbugs{color}.  The patch does not introduce any  new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 lineLengths{color}.  The patch does not introduce lines 
longer than 100

  {color:green}+1 site{color}.  The mvn post-site goal succeeds with this patch.

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
 

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/14641//testReport/
Release Findbugs (version 2.0.3)warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/14641//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors: 
https://builds.apache.org/job/PreCommit-HBASE-Build/14641//artifact/patchprocess/checkstyle-aggregate.html

  Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/14641//console

This message is automatically generated.

 Distinguish blockedFlushCount vs unblockedFlushCount when tuning heap memory
 

 Key: HBASE-13980
 URL: https://issues.apache.org/jira/browse/HBASE-13980
 Project: HBase
  Issue Type: Improvement
Reporter: Ted Yu
Assignee: Abhilash
Priority: Minor
 Fix For: 2.0.0, 1.3.0

 Attachments: HBASE-13980-v1.patch, HBASE-13980-v1.patch, 
 HBASE-13980.patch, HBASE-13980.patch


 Currently DefaultHeapMemoryTuner doesn't distinguish blockedFlushCount vs 
 unblockedFlushCount.
 In its tune() method:
 {code}
 long totalFlushCount = blockedFlushCount+unblockedFlushCount;
 rollingStatsForCacheMisses.insertDataValue(cacheMissCount);
 rollingStatsForFlushes.insertDataValue(totalFlushCount);
 {code}
 Occurrence of blocked flush indicates that upper limit for memstore is not 
 sufficient.
 We should either give blockedFlushCount more weight or, take tuning action 
 based on blockedFlushCount directly.
 See discussion from tail of HBASE-13876.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13963) avoid leaking jdk.tools


 [ 
https://issues.apache.org/jira/browse/HBASE-13963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-13963:
---
Fix Version/s: 0.98.14

Picked back to 0.98. It is a bug that we are leaking a dependency on jdk.tools.

 avoid leaking jdk.tools
 ---

 Key: HBASE-13963
 URL: https://issues.apache.org/jira/browse/HBASE-13963
 Project: HBase
  Issue Type: Sub-task
  Components: build, documentation
Reporter: Sean Busbey
Assignee: Gabor Liptak
Priority: Critical
 Fix For: 2.0.0, 0.98.14, 1.2.0, 1.3.0

 Attachments: HBASE-13963.1.patch, HBASE-13963.2.patch


 Right now hbase-annotations uses jdk7 jdk.tools and exposes that to 
 downstream via hbase-client. We need it for building and using our custom 
 doclet, but can improve a couple of things: 
 -1) We should be using a jdk.tools version based on our java version (use jdk 
 activated profiles to set it)-
 2) We should not be including any jdk.tools version in our hbase-client 
 transitive dependencies (or other downstream-facing artifacts). 
 Unfortunately, system dependencies are included in transitive resolution, so 
 we'll need to exclude it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (HBASE-13861) BucketCacheTmpl.jamon has wrong bucket free and used labels


 [ 
https://issues.apache.org/jira/browse/HBASE-13861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey reopened HBASE-13861:
-

pulling back to 1.1  and 1.0 now

 BucketCacheTmpl.jamon has wrong bucket free and used labels
 ---

 Key: HBASE-13861
 URL: https://issues.apache.org/jira/browse/HBASE-13861
 Project: HBase
  Issue Type: Bug
  Components: regionserver, UI
Affects Versions: 1.1.0
Reporter: Lars George
Assignee: Matt Warhaftig
  Labels: beginner
 Fix For: 2.0.0, 0.98.14, 1.2.0

 Attachments: hbase-13861-v1.patch


 See this from the template, and note the label and actual values for the last 
 two columns.
 {noformat}
 table class=table table-striped
 tr
 thBucket Offset/th
 thAllocation Size/th
 thFree Count/th
 thUsed Count/th
 /tr
 %for Bucket bucket: buckets %
 tr
 td% bucket.getBaseOffset() %/td
 td% bucket.getItemAllocationSize() %/td
 td% bucket.getFreeBytes() %/td
 td% bucket.getUsedBytes() %/td
 /tr
 {noformat}
 They are labeled counts but are bytes, duh.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13970) NPE during compaction in trunk


 [ 
https://issues.apache.org/jira/browse/HBASE-13970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-13970:

Fix Version/s: (was: 1.2.0)
   1.2.1
   1.3.0
   1.0.2

 NPE during compaction in trunk
 --

 Key: HBASE-13970
 URL: https://issues.apache.org/jira/browse/HBASE-13970
 Project: HBase
  Issue Type: Bug
Affects Versions: 2.0.0, 0.98.13, 1.2.0, 1.1.1
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 2.0.0, 0.98.14, 1.0.2, 1.1.2, 1.3.0, 1.2.1

 Attachments: HBASE-13970-v1.patch, HBASE-13970.patch


 Updated the trunk.. Loaded the table with PE tool.  Trigger a flush to ensure 
 all data is flushed out to disk. When the first compaction is triggered we 
 get an NPE and this is very easy to reproduce
 {code}
 015-06-25 21:33:46,041 INFO  [main-EventThread] 
 procedure.ZKProcedureMemberRpcs: Received procedure start children changed 
 event: /hbase/flush-table-proc/acquired
 2015-06-25 21:33:46,051 INFO  
 [rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1] 
 regionserver.HRegion: Flushing 1/1 column families, memstore=76.91 MB
 2015-06-25 21:33:46,159 ERROR 
 [regionserver/stobdtserver3/10.224.54.70:16040-longCompactions-1435248183945] 
 regionserver.CompactSplitThread: Compaction failed Request = 
 regionName=TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.,
  storeName=info, fileCount=3, fileSize=343.4 M (114.5 M, 114.5 M, 114.5 M), 
 priority=3, time=7536968291719985
 java.lang.NullPointerException
 at 
 org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController$ActiveCompaction.access$700(PressureAwareCompactionThroughputController.java:79)
 at 
 org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController.finish(PressureAwareCompactionThroughputController.java:238)
 at 
 org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:306)
 at 
 org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:106)
 at 
 org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:112)
 at 
 org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1202)
 at 
 org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1792)
 at 
 org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:524)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:745)
 2015-06-25 21:33:46,745 INFO  
 [rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1] 
 regionserver.DefaultStoreFlusher: Flushed, sequenceid=1534, memsize=76.9 M, 
 hasBloomFilter=true, into tmp file 
 hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/.tmp/942ba0831a0047a08987439e34361a0c
 2015-06-25 21:33:46,772 INFO  
 [rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1] 
 regionserver.HStore: Added 
 hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/info/942ba0831a0047a08987439e34361a0c,
  entries=68116, sequenceid=1534, filesize=68.7 M
 2015-06-25 21:33:46,773 INFO  
 [rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1] 
 regionserver.HRegion: Finished memstore flush of ~76.91 MB/80649344, 
 currentsize=0 B/0 for region 
 TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4.
  in 723ms, sequenceid=1534, compaction requested=true
 2015-06-25 21:33:46,780 INFO  [main-EventThread] 
 procedure.ZKProcedureMemberRpcs: Received created 
 event:/hbase/flush-table-proc/reached/TestTable
 2015-06-25 21:33:46,790 INFO  [main-EventThread] 
 procedure.ZKProcedureMemberRpcs: Received created 
 event:/hbase/flush-table-proc/abort/TestTable
 2015-06-25 21:33:46,791 INFO  [main-EventThread] 
 procedure.ZKProcedureMemberRpcs: Received procedure abort children changed 
 event: /hbase/flush-table-proc/abort
 2015-06-25 21:33:46,803 INFO  [main-EventThread] 
 procedure.ZKProcedureMemberRpcs: Received procedure start children changed 
 event: /hbase/flush-table-proc/acquired
 2015-06-25 21:33:46,818 INFO  [main-EventThread] 
 procedure.ZKProcedureMemberRpcs: Received procedure abort children changed 
 event: /hbase/flush-table-proc/abort
 {code}
 Will check this on what is the reason behind it. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13372) Unit tests for SplitTransaction and RegionMergeTransaction listeners


 [ 
https://issues.apache.org/jira/browse/HBASE-13372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-13372:

Fix Version/s: (was: 1.2.0)
   1.3.0

 Unit tests for SplitTransaction and RegionMergeTransaction listeners
 

 Key: HBASE-13372
 URL: https://issues.apache.org/jira/browse/HBASE-13372
 Project: HBase
  Issue Type: Test
Affects Versions: 2.0.0, 1.1.0
Reporter: Andrew Purtell
  Labels: beginner
 Fix For: 2.0.0, 1.3.0


 We have new Listener interfaces in SplitTransaction and 
 RegionMergeTransaction. There are no use cases for these yet, nor unit tests. 
 We should have unit tests for these that do something just a bit nontrivial 
 so as to provide a useful example.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13839) Fix AssgnmentManagerTmpl.jamon issues (coloring, content etc.)

[
https://issues.apache.org/jira/browse/HBASE-13839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sean Busbey updated HBASE-13839:

Fix Version/s: (was: 1.2.0)
1.3.0

Fix AssgnmentManagerTmpl.jamon issues (coloring, content etc.)
--

Key: HBASE-13839
URL: https://issues.apache.org/jira/browse/HBASE-13839
Project: HBase
Issue Type: Bug
Components: master, UI
Affects Versions: 1.1.0
Reporter: Lars George
Labels: beginner
Fix For: 2.0.0, 1.3.0

The template for the RIT in the Master status page,
AssignmentManagerTmpl.jamon) has a few issues:
- The oldest RIT should not be _red_, looks like a failed entry
The RIT entries should be for example yellow/amber when over the threshold
time, and red if 2x the threshold - or red for the oldest once over the
threshold.
- Region count over RIT threshold should only be colored if 0
The summary line (first of two) should not be colored unless there is a value
0 in it.
- Color is overriden by table-stripped CSS style!
The Bootstrap stylesheet cancels out the hardcoded coloring! The
table-stripped resets the conditional coloring and should be fixed. Best is
to use alert-warning etc. that come from the Bootstrap theme stylesheet.
That should maybe already work in combination with the table-stripped from
the same.
- Should sort descending by time
Currently the list of regions is sorted by encoded region name. Better is to
have the table sorted by RIT time descending.
We should also think about a pagination option for the currently hardcoded
100 entries max. Maybe a separate issue?

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13838) Fix shared TaskStatusTmpl.jamon issues (coloring, content, etc.)


 [ 
https://issues.apache.org/jira/browse/HBASE-13838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-13838:

Fix Version/s: (was: 1.2.0)
   1.3.0

 Fix shared TaskStatusTmpl.jamon issues (coloring, content, etc.)
 

 Key: HBASE-13838
 URL: https://issues.apache.org/jira/browse/HBASE-13838
 Project: HBase
  Issue Type: Bug
  Components: UI
Affects Versions: 1.1.0
Reporter: Lars George
  Labels: beginner
 Fix For: 2.0.0, 1.3.0


 There are a few issues with the shared TaskStatusTmpl:
 - Client operations tab is always empty 
 For Master this is expected, but for RegionServers there is never anything 
 listed either. Fix for RS status page (probably caused by params not 
 containing Operation subclass anymore, but some PB generated classes?)
 - Hide “Client Operations” tab for master UI
 Since operations are RS only. Or we fix this and make other calls show here.
 - The alert-error for aborted tasks is not set in CSS at all
 When a task was aborted it should be amber or red, but the assigned style is 
 not in any of the linked stylesheets (abort-error). Add.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13994) Backport HBASE-13917 (Remove string comparison to identify request priority) to 0.98


[ 
https://issues.apache.org/jira/browse/HBASE-13994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610909#comment-14610909
 ] 

stack commented on HBASE-13994:
---

LGTM +1

 Backport HBASE-13917 (Remove string comparison to identify request priority) 
 to 0.98
 

 Key: HBASE-13994
 URL: https://issues.apache.org/jira/browse/HBASE-13994
 Project: HBase
  Issue Type: Task
Reporter: Andrew Purtell
Assignee: Andrew Purtell
 Fix For: 0.98.14

 Attachments: HBASE-13994-0.98.patch






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13841) Master UI: Region server table compactions percentage less useful over time


 [ 
https://issues.apache.org/jira/browse/HBASE-13841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-13841:

Fix Version/s: (was: 1.2.0)
   1.3.0

 Master UI: Region server table compactions percentage less useful over time
 ---

 Key: HBASE-13841
 URL: https://issues.apache.org/jira/browse/HBASE-13841
 Project: HBase
  Issue Type: Bug
  Components: master, UI
Affects Versions: 1.1.0
Reporter: Lars George
  Labels: beginner
 Fix For: 2.0.0, 1.3.0


 The problem is that the percentage is computed as current KVs to 
 compact/total KVs to compact, and the template accumulating the counts 
 over its lifetime. Initially it is OK (for the very first run), but after a 
 short amount of time the percentage idles around 99-100% always.
 Not sure how to fix, since we do not have the previews value (unless we save 
 it somewhere in the page, which is messy during reloads etc.). We may have to 
 simply drop the percentage? Or better would be not to accumulate the counts, 
 but keep the current/total number as well, which makes the percentage 
 computation always correct. I vote for the latter.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13986) HMaster instance always returns false for isAborted() check.


[ 
https://issues.apache.org/jira/browse/HBASE-13986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610916#comment-14610916
 ] 

stack commented on HBASE-13986:
---

Lets see a patch [~a72877] Sounds fine. Master for a while was a RegionServer. 
We are climbing back some from this classification. Would be good to see a 
patch to see to see where we are doing this abort (abort of the Master is not 
an op I recall us needing... lets see)

 HMaster instance always returns false for isAborted() check.
 

 Key: HBASE-13986
 URL: https://issues.apache.org/jira/browse/HBASE-13986
 Project: HBase
  Issue Type: Bug
Reporter: Abhishek Kumar
Assignee: Abhishek Kumar
Priority: Minor

 It seems that HMaster never set abortRequested flag to true as done by 
 HRegionServer in its abort() method.We can see isAborted method being used in 
 few places for HMaster instance (like in HMasterCommandLine.startMaster) 
 where code flow being determined based on the result of isAborted() call.
 We can set this abortRequested flag in Hmaster's abort() method as well like 
 in HRegionServer's abort method, let me know if it seems ok. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11747) ClusterStatus is too bulky


[ 
https://issues.apache.org/jira/browse/HBASE-11747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610932#comment-14610932
 ] 

Thiruvel Thirumoolan commented on HBASE-11747:
--

We are also exploring the option of compressing the status and sending from the 
server.

 ClusterStatus is too bulky 
 ---

 Key: HBASE-11747
 URL: https://issues.apache.org/jira/browse/HBASE-11747
 Project: HBase
  Issue Type: Sub-task
Reporter: Virag Kothari
 Attachments: exceptiontrace


 Following exception on 0.98 with 1M regions on cluster with 160 region servers
 {code}
 Caused by: java.io.IOException: Call to regionserverhost:port failed on local 
 exception: com.google.protobuf.InvalidProtocolBufferException: Protocol 
 message was too large.  May be malicious.  Use 
 CodedInputStream.setSizeLimit() to increase the size limit.
   at 
 org.apache.hadoop.hbase.ipc.RpcClient.wrapException(RpcClient.java:1482)
   at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1454)
   at 
 org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1654)
   at 
 org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1712)
   at 
 org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.getClusterStatus(MasterProtos.java:42555)
   at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$5.getClusterStatus(HConnectionManager.java:2132)
   at 
 org.apache.hadoop.hbase.client.HBaseAdmin$16.call(HBaseAdmin.java:2166)
   at 
 org.apache.hadoop.hbase.client.HBaseAdmin$16.call(HBaseAdmin.java:2162)
   at 
 org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:114)
   ... 43 more
 Caused by: com.google.protobuf.InvalidProtocolBufferException: Protocol 
 message was too large.  May be malicious.  Use 
 CodedInputStream.setSizeLimit() to increase the size limit.
   at 
 com.google.protobuf.InvalidProtocolBufferException.sizeLimitExceeded(InvalidProtocolBufferException.java:110)
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13994) Backport HBASE-13917 (Remove string comparison to identify request priority) to 0.98


 [ 
https://issues.apache.org/jira/browse/HBASE-13994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-13994:
---
Status: Open  (was: Patch Available)

 Backport HBASE-13917 (Remove string comparison to identify request priority) 
 to 0.98
 

 Key: HBASE-13994
 URL: https://issues.apache.org/jira/browse/HBASE-13994
 Project: HBase
  Issue Type: Task
Reporter: Andrew Purtell
Assignee: Andrew Purtell
 Fix For: 0.98.14

 Attachments: HBASE-13994-0.98.patch






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13994) Backport HBASE-13917 (Remove string comparison to identify request priority) to 0.98


 [ 
https://issues.apache.org/jira/browse/HBASE-13994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-13994:
---
Status: Patch Available  (was: Open)

Can I get a quickie review [~lhofhansl] or [~jesse_yates] ?

 Backport HBASE-13917 (Remove string comparison to identify request priority) 
 to 0.98
 

 Key: HBASE-13994
 URL: https://issues.apache.org/jira/browse/HBASE-13994
 Project: HBase
  Issue Type: Task
Reporter: Andrew Purtell
Assignee: Andrew Purtell
 Fix For: 0.98.14

 Attachments: HBASE-13994-0.98.patch






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13339) Update default Hadoop version to 2.6.0


[ 
https://issues.apache.org/jira/browse/HBASE-13339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610875#comment-14610875
 ] 

Sean Busbey commented on HBASE-13339:
-

I thought there were plans to have a 2.6.1 release? (see Planning Hadoop 2.6.1 
release on common-dev@hadoop from Apr - Jun 2015).

I believe [~ndimiduk] is supposed to be filing a ticket about starting to 
address the 2.7.z line. That's going to be contentious because of the 
non-production label for 2.7.0. I'd personally have an easier time voting in 
favor of adding better 2.7.z support if there were a Hadoop 2.6.1 release that 
included HADOOP-11674 and HADOOP-11710.

 Update default Hadoop version to 2.6.0
 --

 Key: HBASE-13339
 URL: https://issues.apache.org/jira/browse/HBASE-13339
 Project: HBase
  Issue Type: Bug
  Components: build
Affects Versions: 2.0.0, 1.1.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Priority: Blocker
 Fix For: 2.0.0, 1.2.0

 Attachments: HBASE-13339-v1.patch, HBASE-13339.patch


 Current default Hadoop version is getting a little long in the tooth. We 
 should update to the latest version. The latest version is backwards 
 compatible with 2.5.1's dfs and mr so this should be painless.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13861) BucketCacheTmpl.jamon has wrong bucket free and used labels


[ 
https://issues.apache.org/jira/browse/HBASE-13861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610884#comment-14610884
 ] 

Sean Busbey commented on HBASE-13861:
-

actually, does this break operational compatibility? or is the presumption that 
folks parsing out the table won't be broken by changing a column label?

 BucketCacheTmpl.jamon has wrong bucket free and used labels
 ---

 Key: HBASE-13861
 URL: https://issues.apache.org/jira/browse/HBASE-13861
 Project: HBase
  Issue Type: Bug
  Components: regionserver, UI
Affects Versions: 1.1.0
Reporter: Lars George
Assignee: Matt Warhaftig
  Labels: beginner
 Fix For: 2.0.0, 0.98.14, 1.2.0

 Attachments: hbase-13861-v1.patch


 See this from the template, and note the label and actual values for the last 
 two columns.
 {noformat}
 table class=table table-striped
 tr
 thBucket Offset/th
 thAllocation Size/th
 thFree Count/th
 thUsed Count/th
 /tr
 %for Bucket bucket: buckets %
 tr
 td% bucket.getBaseOffset() %/td
 td% bucket.getItemAllocationSize() %/td
 td% bucket.getFreeBytes() %/td
 td% bucket.getUsedBytes() %/td
 /tr
 {noformat}
 They are labeled counts but are bytes, duh.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13983) Doc how the oddball HTable methods getStartKey, getEndKey, etc. will be removed in 2.0.0


 [ 
https://issues.apache.org/jira/browse/HBASE-13983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-13983:
--
Description: See the mail thread on dev Semantic Versioning Worksheet for 
background discussion. A few methods in HTable were NOT deprecated when we cut 
1.0.0: getStartKeys, getEndKeys This issue is about doc'ing that even 
though they were not deprecated, they will be removed in hbase 2.0.0.  (was: 
See the mail thread on dev Semantic Versioning Worksheet for background 
discussion. A few methods in HTable were NOT deprecated when we cut 1.0.0: 
getStartKeys, getEndKeys This issue is about doc'ing that even though they 
were not deprecated, they will be removed in hbase 1.0.0.)

 Doc how the oddball HTable methods getStartKey, getEndKey, etc. will be 
 removed in 2.0.0
 

 Key: HBASE-13983
 URL: https://issues.apache.org/jira/browse/HBASE-13983
 Project: HBase
  Issue Type: Task
  Components: documentation
Reporter: stack
Assignee: stack
Priority: Minor
 Attachments: 13983.branch-1.txt, 13983.txt


 See the mail thread on dev Semantic Versioning Worksheet for background 
 discussion. A few methods in HTable were NOT deprecated when we cut 1.0.0: 
 getStartKeys, getEndKeys This issue is about doc'ing that even though 
 they were not deprecated, they will be removed in hbase 2.0.0.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13923) Loaded region coprocessors are not reported in shell status command


 [ 
https://issues.apache.org/jira/browse/HBASE-13923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-13923:
---
Fix Version/s: 0.98.14

Picked to 0.98

 Loaded region coprocessors are not reported in shell status command
 ---

 Key: HBASE-13923
 URL: https://issues.apache.org/jira/browse/HBASE-13923
 Project: HBase
  Issue Type: Bug
  Components: regionserver, shell
Affects Versions: 1.1.0.1
Reporter: Lars George
Assignee: Ashish Singhi
 Fix For: 2.0.0, 0.98.14, 1.0.2, 1.1.2, 1.3.0, 1.2.1

 Attachments: 13923-addendum.txt, HBASE-13923-branch-1.0.patch, 
 HBASE-13923-v1.patch, HBASE-13923-v2.patch, HBASE-13923.patch


 I added a CP to a table using the shell's alter command. Now I tried to check 
 if it was loaded (short of resorting to parsing the logs). I recalled the 
 refguide mentioned the {{status 'detailed'}} command, and tried that to no 
 avail.
 The UI shows the loaded class in the Software Attributes section, so the info 
 is there. But a shell status command (even after waiting 12+ hours shows 
 nothing. Here an example of a server that has it loaded according to 
 {{describe}} and the UI, but the shell lists this:
 {noformat}
 slave-1.internal.larsgeorge.com:16020 1434486031598
 requestsPerSecond=0.0, numberOfOnlineRegions=5, usedHeapMB=278, 
 maxHeapMB=941, numberOfStores=5, numberOfStorefiles=3, 
 storefileUncompressedSizeMB=2454, storefileSizeMB=2454, 
 compressionRatio=1., memstoreSizeMB=0, storefileIndexSizeMB=0, 
 readRequestsCount=32070, writeRequestsCount=0, rootIndexSizeKB=0, 
 totalStaticIndexSizeKB=2086, totalStaticBloomSizeKB=480, 
 totalCompactingKVs=0, currentCompactedKVs=0, compactionProgressPct=NaN, 
 coprocessors=[]
 testqauat:usertable,,1433747062257.4db0d7d73cbaac45cb8568d5b185e1f2.
 numberOfStores=1, numberOfStorefiles=0, 
 storefileUncompressedSizeMB=0, lastMajorCompactionTimestamp=0, 
 storefileSizeMB=0, memstoreSizeMB=0, storefileIndexSizeMB=0, 
 readRequestsCount=0, writeRequestsCount=0, rootIndexSizeKB=0, 
 totalStaticIndexSizeKB=0, totalStaticBloomSizeKB=0, totalCompactingKVs=0, 
 currentCompactedKVs=0, compactionProgressPct=NaN, completeSequenceId=-1, 
 dataLocality=0.0
 
 testqauat:usertable,user0,1433747062257.f7c7fe3c7d26910010f40101b20f8d06.
 numberOfStores=1, numberOfStorefiles=0, 
 storefileUncompressedSizeMB=0, lastMajorCompactionTimestamp=0, 
 storefileSizeMB=0, memstoreSizeMB=0, storefileIndexSizeMB=0, 
 readRequestsCount=0, writeRequestsCount=0, rootIndexSizeKB=0, 
 totalStaticIndexSizeKB=0, totalStaticBloomSizeKB=0, totalCompactingKVs=0, 
 currentCompactedKVs=0, compactionProgressPct=NaN, completeSequenceId=-1, 
 dataLocality=0.0
 
 testqauat:usertable,user1,1433747062257.dcd5395044732242dfed39b09aa05c36.
 numberOfStores=1, numberOfStorefiles=1, 
 storefileUncompressedSizeMB=820, lastMajorCompactionTimestamp=1434173025593, 
 storefileSizeMB=820, compressionRatio=1., memstoreSizeMB=0, 
 storefileIndexSizeMB=0, readRequestsCount=32070, writeRequestsCount=0, 
 rootIndexSizeKB=0, totalStaticIndexSizeKB=699, totalStaticBloomSizeKB=160, 
 totalCompactingKVs=0, currentCompactedKVs=0, compactionProgressPct=NaN, 
 completeSequenceId=-1, dataLocality=1.0
 
 testqauat:usertable,user7,1433747062257.9277fd1d34909b0cb150707cbd7a3907.
 numberOfStores=1, numberOfStorefiles=1, 
 storefileUncompressedSizeMB=816, lastMajorCompactionTimestamp=1434283025585, 
 storefileSizeMB=816, compressionRatio=1., memstoreSizeMB=0, 
 storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=0, 
 rootIndexSizeKB=0, totalStaticIndexSizeKB=690, totalStaticBloomSizeKB=160, 
 totalCompactingKVs=0, currentCompactedKVs=0, compactionProgressPct=NaN, 
 completeSequenceId=-1, dataLocality=1.0
 
 testqauat:usertable,user8,1433747062257.d930b52db8c7f07f3c3ab3e12e61a085.
 numberOfStores=1, numberOfStorefiles=1, 
 storefileUncompressedSizeMB=818, lastMajorCompactionTimestamp=1433771950960, 
 storefileSizeMB=818, compressionRatio=1., memstoreSizeMB=0, 
 storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=0, 
 rootIndexSizeKB=0, totalStaticIndexSizeKB=697, totalStaticBloomSizeKB=160, 
 totalCompactingKVs=0, currentCompactedKVs=0, compactionProgressPct=NaN, 
 completeSequenceId=-1, dataLocality=1.0
 {noformat}
 The refguide shows an example of an older HBase version that has the CP class 
 listed properly. Something is broken.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11747) ClusterStatus is too bulky

2015-07-01 Thread Mikhail Antonov (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-11747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610877#comment-14610877
 ] 

Mikhail Antonov commented on HBASE-11747:
-

Wondering about next steps/directions here. Do we want to just bump 
CodeInputStream#limit to higher numbers and see if that addresses problem at 
hands (I think it should), or do we want to optimize protocol? Ive seen 3 
options here - 1)streaming instead of single message 2) decouple region/RS load 
info from cluster status itself 3) try to make data pieces themselves more 
compact, region names etc.

 ClusterStatus is too bulky 
 ---

 Key: HBASE-11747
 URL: https://issues.apache.org/jira/browse/HBASE-11747
 Project: HBase
  Issue Type: Sub-task
Reporter: Virag Kothari
 Attachments: exceptiontrace


 Following exception on 0.98 with 1M regions on cluster with 160 region servers
 {code}
 Caused by: java.io.IOException: Call to regionserverhost:port failed on local 
 exception: com.google.protobuf.InvalidProtocolBufferException: Protocol 
 message was too large.  May be malicious.  Use 
 CodedInputStream.setSizeLimit() to increase the size limit.
   at 
 org.apache.hadoop.hbase.ipc.RpcClient.wrapException(RpcClient.java:1482)
   at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1454)
   at 
 org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1654)
   at 
 org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1712)
   at 
 org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.getClusterStatus(MasterProtos.java:42555)
   at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$5.getClusterStatus(HConnectionManager.java:2132)
   at 
 org.apache.hadoop.hbase.client.HBaseAdmin$16.call(HBaseAdmin.java:2166)
   at 
 org.apache.hadoop.hbase.client.HBaseAdmin$16.call(HBaseAdmin.java:2162)
   at 
 org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:114)
   ... 43 more
 Caused by: com.google.protobuf.InvalidProtocolBufferException: Protocol 
 message was too large.  May be malicious.  Use 
 CodedInputStream.setSizeLimit() to increase the size limit.
   at 
 com.google.protobuf.InvalidProtocolBufferException.sizeLimitExceeded(InvalidProtocolBufferException.java:110)
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11747) ClusterStatus is too bulky


[ 
https://issues.apache.org/jira/browse/HBASE-11747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610905#comment-14610905
 ] 

stack commented on HBASE-11747:
---

Lets up CIS#limit for sure.

In new JIRA optimize protocol. There are a few already if you search 'hbase 
clusterstatus'. I like #2 and #3 from your list. For #2, was looking at 
exporting jmx so say the Master could read cluster metrics instead of getting 
metrics recast and served on the heartbeat. Was looking at https://jolokia.org/ 
Seems more sensible than JMX federation (Seems like its possible to hook up as 
src for D3 graphing). Do we poll rather than have the stuff pushed? What 
happens in a big cluster?

Good on you [~mantonov]

 ClusterStatus is too bulky 
 ---

 Key: HBASE-11747
 URL: https://issues.apache.org/jira/browse/HBASE-11747
 Project: HBase
  Issue Type: Sub-task
Reporter: Virag Kothari
 Attachments: exceptiontrace


 Following exception on 0.98 with 1M regions on cluster with 160 region servers
 {code}
 Caused by: java.io.IOException: Call to regionserverhost:port failed on local 
 exception: com.google.protobuf.InvalidProtocolBufferException: Protocol 
 message was too large.  May be malicious.  Use 
 CodedInputStream.setSizeLimit() to increase the size limit.
   at 
 org.apache.hadoop.hbase.ipc.RpcClient.wrapException(RpcClient.java:1482)
   at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1454)
   at 
 org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1654)
   at 
 org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1712)
   at 
 org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.getClusterStatus(MasterProtos.java:42555)
   at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$5.getClusterStatus(HConnectionManager.java:2132)
   at 
 org.apache.hadoop.hbase.client.HBaseAdmin$16.call(HBaseAdmin.java:2166)
   at 
 org.apache.hadoop.hbase.client.HBaseAdmin$16.call(HBaseAdmin.java:2162)
   at 
 org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:114)
   ... 43 more
 Caused by: com.google.protobuf.InvalidProtocolBufferException: Protocol 
 message was too large.  May be malicious.  Use 
 CodedInputStream.setSizeLimit() to increase the size limit.
   at 
 com.google.protobuf.InvalidProtocolBufferException.sizeLimitExceeded(InvalidProtocolBufferException.java:110)
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13557) Special WAL handling for system tables


 [ 
https://issues.apache.org/jira/browse/HBASE-13557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-13557:

Fix Version/s: (was: 1.2.0)
   1.3.0

 Special WAL handling for system tables
 --

 Key: HBASE-13557
 URL: https://issues.apache.org/jira/browse/HBASE-13557
 Project: HBase
  Issue Type: Sub-task
  Components: wal
Affects Versions: 0.98.0
Reporter: Sean Busbey
 Fix For: 2.0.0, 1.3.0


 We need to be able to recover system tables after meta and before user 
 tables, since they block things like setting a new active master.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13857) Slow WAL Append count in ServerMetricsTmpl.jamon is hardcoded to zero


 [ 
https://issues.apache.org/jira/browse/HBASE-13857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-13857:

Fix Version/s: (was: 1.2.0)
   1.2.1
   1.3.0
   1.1.2
   1.0.2
   0.98.14

 Slow WAL Append count in ServerMetricsTmpl.jamon is hardcoded to zero
 -

 Key: HBASE-13857
 URL: https://issues.apache.org/jira/browse/HBASE-13857
 Project: HBase
  Issue Type: Bug
  Components: regionserver, UI
Affects Versions: 1.1.0
Reporter: Lars George
  Labels: beginner
 Fix For: 2.0.0, 0.98.14, 1.0.2, 1.1.2, 1.3.0, 1.2.1


 The template has this:
 {noformat}
  tr
 ...
 thSlow WAL Append Count/th
 /tr
 tr
 
 td% 0 %/td
 /tr
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13983) Doc how the oddball HTable methods getStartKey, getEndKey, etc. will be removed in 2.0.0


 [ 
https://issues.apache.org/jira/browse/HBASE-13983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-13983:
--
Issue Type: Sub-task  (was: Task)
Parent: HBASE-13197

 Doc how the oddball HTable methods getStartKey, getEndKey, etc. will be 
 removed in 2.0.0
 

 Key: HBASE-13983
 URL: https://issues.apache.org/jira/browse/HBASE-13983
 Project: HBase
  Issue Type: Sub-task
  Components: documentation
Reporter: stack
Assignee: stack
Priority: Minor
 Attachments: 13983.branch-1.txt, 13983.txt


 See the mail thread on dev Semantic Versioning Worksheet for background 
 discussion. A few methods in HTable were NOT deprecated when we cut 1.0.0: 
 getStartKeys, getEndKeys This issue is about doc'ing that even though 
 they were not deprecated, they will be removed in hbase 2.0.0.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13347) RowCounter using special filter is broken


 [ 
https://issues.apache.org/jira/browse/HBASE-13347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-13347:

Fix Version/s: (was: 1.2.0)
   1.2.1
   1.3.0

 RowCounter using special filter is broken
 -

 Key: HBASE-13347
 URL: https://issues.apache.org/jira/browse/HBASE-13347
 Project: HBase
  Issue Type: Bug
  Components: mapreduce
Affects Versions: 1.0.0
Reporter: Lars George
 Fix For: 2.0.0, 1.0.2, 1.1.2, 1.3.0, 1.2.1


 The {{RowCounter}} in the {{mapreduce}} package is supposed to check if the 
 row count scan has a column selection added to it, and if so, use a different 
 filter that finds the row and counts it. But the {{qualifier.add()}} call is 
 missing in the {{for}} loop. See 
 https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/RowCounter.java#L165
 Needs fixing or row count might be wrong when using {{--range}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13587) TestSnapshotCloneIndependence.testOnlineSnapshotDeleteIndependent is flakey


 [ 
https://issues.apache.org/jira/browse/HBASE-13587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-13587:

Fix Version/s: (was: 1.2.0)
   1.2.1

 TestSnapshotCloneIndependence.testOnlineSnapshotDeleteIndependent is flakey
 ---

 Key: HBASE-13587
 URL: https://issues.apache.org/jira/browse/HBASE-13587
 Project: HBase
  Issue Type: Test
Reporter: Nick Dimiduk
 Fix For: 2.0.0, 1.1.2, 1.2.1


 Looking at our [build 
 history|https://builds.apache.org/job/HBase-1.1/buildTimeTrend], it seems 
 this test is flakey. See builds 428, 431, 432, 433.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13861) BucketCacheTmpl.jamon has wrong bucket free and used labels


[ 
https://issues.apache.org/jira/browse/HBASE-13861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610950#comment-14610950
 ] 

Andrew Purtell commented on HBASE-13861:


It's debatable. I think operational tooling that scrapes UIs is brittle and not 
advisable (broken, really) and this is a clearly misleading _typo_ that should 
be fixed. Other branch RMs may have a different take. 

 BucketCacheTmpl.jamon has wrong bucket free and used labels
 ---

 Key: HBASE-13861
 URL: https://issues.apache.org/jira/browse/HBASE-13861
 Project: HBase
  Issue Type: Bug
  Components: regionserver, UI
Affects Versions: 1.1.0
Reporter: Lars George
Assignee: Matt Warhaftig
  Labels: beginner
 Fix For: 2.0.0, 0.98.14, 1.2.0

 Attachments: hbase-13861-v1.patch


 See this from the template, and note the label and actual values for the last 
 two columns.
 {noformat}
 table class=table table-striped
 tr
 thBucket Offset/th
 thAllocation Size/th
 thFree Count/th
 thUsed Count/th
 /tr
 %for Bucket bucket: buckets %
 tr
 td% bucket.getBaseOffset() %/td
 td% bucket.getItemAllocationSize() %/td
 td% bucket.getFreeBytes() %/td
 td% bucket.getUsedBytes() %/td
 /tr
 {noformat}
 They are labeled counts but are bytes, duh.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13995) ServerName is not fully case insensitive


[ 
https://issues.apache.org/jira/browse/HBASE-13995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610834#comment-14610834
 ] 

Hudson commented on HBASE-13995:


FAILURE: Integrated in HBase-1.2 #43 (See 
[https://builds.apache.org/job/HBase-1.2/43/])
HBASE-13995 ServerName is not fully case insensitive (matteo.bertozzi: rev 
012b1d426e1c4a84e5932cb8fbdcc8bfc32f6e63)
* hbase-server/src/test/java/org/apache/hadoop/hbase/TestServerName.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/ServerName.java


 ServerName is not fully case insensitive
 

 Key: HBASE-13995
 URL: https://issues.apache.org/jira/browse/HBASE-13995
 Project: HBase
  Issue Type: Bug
  Components: Region Assignment
Affects Versions: 2.0.0, 1.2.0, 0.98.12.1, 1.0.1.1, 1.1.0.1
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: 2.0.0, 0.98.14, 1.0.2, 1.2.0, 1.1.2

 Attachments: HBASE-13995-v0.patch, HBASE-13995-v0.patch


 we ended up with two ServerName with different cases, AAA and aaa.
 Trying to create a table, every once in a while, we ended up with the region 
 lost and not assigned. 
 BaseLoadBalancer.roundRobinAssignment() goes through each server and create a 
 map with what to assign to them.
 We had to server on the list AAA and aaa which are the same machine, the 
 problem is that the round robin now is assigning an empty list to one of the 
 two. so depending on the order we ended up with a region not assigned.
 ServerName equals() does the case insensitive comparison but the hashCode() 
 is done on a case sensitive server name, so the Map in ServerManager will 
 never hit the item and compare it using equals, so we end up with two entries 
 that are the same server. similar thing for 
 ServerName.isSameHostnameAndPort() where we don't check for cases



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13994) Backport HBASE-13917 (Remove string comparison to identify request priority) to 0.98


[ 
https://issues.apache.org/jira/browse/HBASE-13994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610836#comment-14610836
 ] 

Andrew Purtell commented on HBASE-13994:


Looks like a bad precommit run:
{noformat}
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-surefire-plugin:2.18:test 
(secondPartTestsExecution) on project hbase-server: ExecutionException: 
java.lang.RuntimeException: The forked VM terminated without properly saying 
goodbye. VM crash or System.exit called?
[ERROR] Command was /bin/sh -c cd 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server  
/home/jenkins/jenkins-slave/tools/hudson.model.JDK/jdk-1.7u51/jre/bin/java 
-enableassertions -XX:MaxDirectMemorySize=1G -Xmx2800m -XX:MaxPermSize=256m 
-Djava.security.egd=file:/dev/./urandom -Djava.net.preferIPv4Stack=true 
-Djava.awt.headless=true -jar 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/target/surefire/surefirebooter6668024873047921134.jar
 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/target/surefire/surefire4054841610365071419tmp
 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/target/surefire/surefire_10802012333607148549769tmp
[ERROR] - [Help 1]
{noformat}

Let me resubmit.

 Backport HBASE-13917 (Remove string comparison to identify request priority) 
 to 0.98
 

 Key: HBASE-13994
 URL: https://issues.apache.org/jira/browse/HBASE-13994
 Project: HBase
  Issue Type: Task
Reporter: Andrew Purtell
Assignee: Andrew Purtell
 Fix For: 0.98.14

 Attachments: HBASE-13994-0.98.patch






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13861) BucketCacheTmpl.jamon has wrong bucket free and used labels


 [ 
https://issues.apache.org/jira/browse/HBASE-13861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-13861:
---
Fix Version/s: 0.98.14

Picked to 0.98. Showing users the wrong unit is a bug. Should go into 1.1 and 
1.0 too.

 BucketCacheTmpl.jamon has wrong bucket free and used labels
 ---

 Key: HBASE-13861
 URL: https://issues.apache.org/jira/browse/HBASE-13861
 Project: HBase
  Issue Type: Bug
  Components: regionserver, UI
Affects Versions: 1.1.0
Reporter: Lars George
Assignee: Matt Warhaftig
  Labels: beginner
 Fix For: 2.0.0, 0.98.14, 1.2.0

 Attachments: hbase-13861-v1.patch


 See this from the template, and note the label and actual values for the last 
 two columns.
 {noformat}
 table class=table table-striped
 tr
 thBucket Offset/th
 thAllocation Size/th
 thFree Count/th
 thUsed Count/th
 /tr
 %for Bucket bucket: buckets %
 tr
 td% bucket.getBaseOffset() %/td
 td% bucket.getItemAllocationSize() %/td
 td% bucket.getFreeBytes() %/td
 td% bucket.getUsedBytes() %/td
 /tr
 {noformat}
 They are labeled counts but are bytes, duh.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HBASE-13983) Doc how the oddball HTable methods getStartKey, getEndKey, etc. will be removed in 2.0.0


 [ 
https://issues.apache.org/jira/browse/HBASE-13983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-13983.
---
   Resolution: Fixed
 Hadoop Flags: Incompatible change,Reviewed
Fix Version/s: 1.1.2
   1.2.0
   1.0.2
   2.0.0
 Release Note: 
Adds extra doc on getStartKeys, getEndKeys, and getStartEndKeys in HTable 
explaining that they will be removed in 2.0.0 (these methods did not get the 
proper full major version deprecation cycle).

In this issue, we actually also remove these methods in master/2.0.0 branch.

Thanks for reviews lads (Fixed the issue you fingered in opening comment 
[~anoop.hbase] -- thanks... and moved this as subissue as you suggested 
[~mantonov])

 Doc how the oddball HTable methods getStartKey, getEndKey, etc. will be 
 removed in 2.0.0
 

 Key: HBASE-13983
 URL: https://issues.apache.org/jira/browse/HBASE-13983
 Project: HBase
  Issue Type: Sub-task
  Components: documentation
Reporter: stack
Assignee: stack
Priority: Minor
 Fix For: 2.0.0, 1.0.2, 1.2.0, 1.1.2

 Attachments: 13983.branch-1.txt, 13983.txt


 See the mail thread on dev Semantic Versioning Worksheet for background 
 discussion. A few methods in HTable were NOT deprecated when we cut 1.0.0: 
 getStartKeys, getEndKeys This issue is about doc'ing that even though 
 they were not deprecated, they will be removed in hbase 2.0.0.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13339) Update default Hadoop version to 2.6.0

2015-07-01 Thread Arpit Agarwal (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-13339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610945#comment-14610945
 ] 

Arpit Agarwal commented on HBASE-13339:
---

There is no further progress towards 2.6.1 since the brief discussion in April. 
This came up at the HDFS BoF at the Hadoop Summit and the informal consensus 
was to stabilize 2.7.x. But like I said if there is an RM willing to drive 
2.6.1 I don't see any grounds for vetoing the release either and no I am not 
volunteering. :-)

 Update default Hadoop version to 2.6.0
 --

 Key: HBASE-13339
 URL: https://issues.apache.org/jira/browse/HBASE-13339
 Project: HBase
  Issue Type: Bug
  Components: build
Affects Versions: 2.0.0, 1.1.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Priority: Blocker
 Fix For: 2.0.0, 1.2.0

 Attachments: HBASE-13339-v1.patch, HBASE-13339.patch


 Current default Hadoop version is getting a little long in the tooth. We 
 should update to the latest version. The latest version is backwards 
 compatible with 2.5.1's dfs and mr so this should be painless.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13995) ServerName is not fully case insensitive


[ 
https://issues.apache.org/jira/browse/HBASE-13995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610832#comment-14610832
 ] 

Hudson commented on HBASE-13995:


FAILURE: Integrated in HBase-1.0 #982 (See 
[https://builds.apache.org/job/HBase-1.0/982/])
HBASE-13995 ServerName is not fully case insensitive (matteo.bertozzi: rev 
ac6a1c1d99351001c15f254660454312d711f0c2)
* hbase-client/src/main/java/org/apache/hadoop/hbase/ServerName.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/TestServerName.java


 ServerName is not fully case insensitive
 

 Key: HBASE-13995
 URL: https://issues.apache.org/jira/browse/HBASE-13995
 Project: HBase
  Issue Type: Bug
  Components: Region Assignment
Affects Versions: 2.0.0, 1.2.0, 0.98.12.1, 1.0.1.1, 1.1.0.1
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: 2.0.0, 0.98.14, 1.0.2, 1.2.0, 1.1.2

 Attachments: HBASE-13995-v0.patch, HBASE-13995-v0.patch


 we ended up with two ServerName with different cases, AAA and aaa.
 Trying to create a table, every once in a while, we ended up with the region 
 lost and not assigned. 
 BaseLoadBalancer.roundRobinAssignment() goes through each server and create a 
 map with what to assign to them.
 We had to server on the list AAA and aaa which are the same machine, the 
 problem is that the round robin now is assigning an empty list to one of the 
 two. so depending on the order we ended up with a region not assigned.
 ServerName equals() does the case insensitive comparison but the hashCode() 
 is done on a case sensitive server name, so the Map in ServerManager will 
 never hit the item and compare it using equals, so we end up with two entries 
 that are the same server. similar thing for 
 ServerName.isSameHostnameAndPort() where we don't check for cases



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13339) Update default Hadoop version to 2.6.0

2015-07-01 Thread Arpit Agarwal (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-13339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610851#comment-14610851
 ] 

Arpit Agarwal commented on HBASE-13339:
---

FWIW Hadoop 2.7.1 RC0 is out. This fixes a large number of stability, 
performance and functionality issues since 2.6.0. (2.7.0 was not a 
production-ready release).
http://markmail.org/message/5o7oxk4lywpcdnw2

There is momentum in the Hadoop community around stabilizing 2.7.x. There are 
no plans for a 2.6.1 release, however if anyone wants to RM 2.6.1 I don't think 
there would be any objections.

 Update default Hadoop version to 2.6.0
 --

 Key: HBASE-13339
 URL: https://issues.apache.org/jira/browse/HBASE-13339
 Project: HBase
  Issue Type: Bug
  Components: build
Affects Versions: 2.0.0, 1.1.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Priority: Blocker
 Fix For: 2.0.0, 1.2.0

 Attachments: HBASE-13339-v1.patch, HBASE-13339.patch


 Current default Hadoop version is getting a little long in the tooth. We 
 should update to the latest version. The latest version is backwards 
 compatible with 2.5.1's dfs and mr so this should be painless.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13988) Add exception handler for lease thread


[ 
https://issues.apache.org/jira/browse/HBASE-13988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610855#comment-14610855
 ] 

stack commented on HBASE-13988:
---

Loosing the lease checker is pretty bad. Should we add an exception handler 
that logs at ERROR rather than abort the regionserver [~liushaohui]?

 Add exception handler for lease thread
 --

 Key: HBASE-13988
 URL: https://issues.apache.org/jira/browse/HBASE-13988
 Project: HBase
  Issue Type: Bug
Affects Versions: 2.0.0
Reporter: Liu Shaohui
Assignee: Liu Shaohui
Priority: Minor
 Fix For: 2.0.0, 1.0.2, 1.1.2, 0.98.15

 Attachments: HBASE-13988-v001.diff


 In a prod cluster, a region server exited for some important 
 threads were not alive. After excluding other threads from the log, we 
 doubted the lease thread was the root. 
 So we need to add an exception handler to the lease thread to debug why it 
 exited in future.
  
 {quote}
 2015-06-29,12:46:09,222 INFO 
 org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: One or more 
 threads are no longer alive -- stop
 2015-06-29,12:46:09,223 INFO org.apache.hadoop.ipc.HBaseServer: Stopping 
 server on 21600
 ...
 2015-06-29,12:46:09,330 INFO org.apache.hadoop.hbase.regionserver.LogRoller: 
 LogRoller exiting.
 2015-06-29,12:46:09,330 INFO 
 org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Thread-37 exiting
 2015-06-29,12:46:09,330 INFO 
 org.apache.hadoop.hbase.regionserver.HRegionServer$CompactionChecker: 
 regionserver21600.compactionChecker exiting
 2015-06-29,12:46:12,403 INFO 
 org.apache.hadoop.hbase.regionserver.HRegionServer$PeriodicMemstoreFlusher: 
 regionserver21600.periodicFlusher exiting
 {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13995) ServerName is not fully case insensitive


[ 
https://issues.apache.org/jira/browse/HBASE-13995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610886#comment-14610886
 ] 

Hudson commented on HBASE-13995:


FAILURE: Integrated in HBase-1.3 #29 (See 
[https://builds.apache.org/job/HBase-1.3/29/])
HBASE-13995 ServerName is not fully case insensitive (matteo.bertozzi: rev 
7ece46a6075cb660336ed06e3f4394389ffbc69a)
* hbase-server/src/test/java/org/apache/hadoop/hbase/TestServerName.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/ServerName.java


 ServerName is not fully case insensitive
 

 Key: HBASE-13995
 URL: https://issues.apache.org/jira/browse/HBASE-13995
 Project: HBase
  Issue Type: Bug
  Components: Region Assignment
Affects Versions: 2.0.0, 1.2.0, 0.98.12.1, 1.0.1.1, 1.1.0.1
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: 2.0.0, 0.98.14, 1.0.2, 1.2.0, 1.1.2

 Attachments: HBASE-13995-v0.patch, HBASE-13995-v0.patch


 we ended up with two ServerName with different cases, AAA and aaa.
 Trying to create a table, every once in a while, we ended up with the region 
 lost and not assigned. 
 BaseLoadBalancer.roundRobinAssignment() goes through each server and create a 
 map with what to assign to them.
 We had to server on the list AAA and aaa which are the same machine, the 
 problem is that the round robin now is assigning an empty list to one of the 
 two. so depending on the order we ended up with a region not assigned.
 ServerName equals() does the case insensitive comparison but the hashCode() 
 is done on a case sensitive server name, so the Map in ServerManager will 
 never hit the item and compare it using equals, so we end up with two entries 
 that are the same server. similar thing for 
 ServerName.isSameHostnameAndPort() where we don't check for cases



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HBASE-13842) [shell] clear_auths.rb description should be improved


 [ 
https://issues.apache.org/jira/browse/HBASE-13842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey resolved HBASE-13842.
-
   Resolution: Duplicate
Fix Version/s: (was: 1.2.0)
   (was: 2.0.0)

 [shell] clear_auths.rb description should be improved
 -

 Key: HBASE-13842
 URL: https://issues.apache.org/jira/browse/HBASE-13842
 Project: HBase
  Issue Type: Bug
  Components: shell
Affects Versions: 1.1.0
Reporter: Lars George
  Labels: beginner

 Currently: “Add a set of visibility labels for an user that has to removed
 Hard to parse, rephrase to something meaningful.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-14007) Writing to table through MR should fail upfront if table does not exist/disabled

Thiruvel Thirumoolan created HBASE-14007:


 Summary: Writing to table through MR should fail upfront if table 
does not exist/disabled
 Key: HBASE-14007
 URL: https://issues.apache.org/jira/browse/HBASE-14007
 Project: HBase
  Issue Type: Bug
Reporter: Thiruvel Thirumoolan
Assignee: Thiruvel Thirumoolan
Priority: Minor






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13995) ServerName is not fully case insensitive


[ 
https://issues.apache.org/jira/browse/HBASE-13995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610947#comment-14610947
 ] 

Hudson commented on HBASE-13995:


FAILURE: Integrated in HBase-TRUNK #6620 (See 
[https://builds.apache.org/job/HBase-TRUNK/6620/])
HBASE-13995 ServerName is not fully case insensitive (matteo.bertozzi: rev 
5ee11840f61c519fedbe99ef589313be6a9a9657)
* hbase-common/src/main/java/org/apache/hadoop/hbase/ServerName.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/TestServerName.java


 ServerName is not fully case insensitive
 

 Key: HBASE-13995
 URL: https://issues.apache.org/jira/browse/HBASE-13995
 Project: HBase
  Issue Type: Bug
  Components: Region Assignment
Affects Versions: 2.0.0, 1.2.0, 0.98.12.1, 1.0.1.1, 1.1.0.1
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: 2.0.0, 0.98.14, 1.0.2, 1.2.0, 1.1.2

 Attachments: HBASE-13995-v0.patch, HBASE-13995-v0.patch


 we ended up with two ServerName with different cases, AAA and aaa.
 Trying to create a table, every once in a while, we ended up with the region 
 lost and not assigned. 
 BaseLoadBalancer.roundRobinAssignment() goes through each server and create a 
 map with what to assign to them.
 We had to server on the list AAA and aaa which are the same machine, the 
 problem is that the round robin now is assigning an empty list to one of the 
 two. so depending on the order we ended up with a region not assigned.
 ServerName equals() does the case insensitive comparison but the hashCode() 
 is done on a case sensitive server name, so the Map in ServerManager will 
 never hit the item and compare it using equals, so we end up with two entries 
 that are the same server. similar thing for 
 ServerName.isSameHostnameAndPort() where we don't check for cases



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13861) BucketCacheTmpl.jamon has wrong bucket free and used labels


[ 
https://issues.apache.org/jira/browse/HBASE-13861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610946#comment-14610946
 ] 

Hudson commented on HBASE-13861:


FAILURE: Integrated in HBase-TRUNK #6620 (See 
[https://builds.apache.org/job/HBase-TRUNK/6620/])
HBASE-13861 Fix UI label for BucketCache free  used bytes. (busbey: rev 
364205da25eb949751342c5aac834ce5434d7212)
* 
hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/BlockCacheTmpl.jamon


 BucketCacheTmpl.jamon has wrong bucket free and used labels
 ---

 Key: HBASE-13861
 URL: https://issues.apache.org/jira/browse/HBASE-13861
 Project: HBase
  Issue Type: Bug
  Components: regionserver, UI
Affects Versions: 1.1.0
Reporter: Lars George
Assignee: Matt Warhaftig
  Labels: beginner
 Fix For: 2.0.0, 0.98.14, 1.2.0

 Attachments: hbase-13861-v1.patch


 See this from the template, and note the label and actual values for the last 
 two columns.
 {noformat}
 table class=table table-striped
 tr
 thBucket Offset/th
 thAllocation Size/th
 thFree Count/th
 thUsed Count/th
 /tr
 %for Bucket bucket: buckets %
 tr
 td% bucket.getBaseOffset() %/td
 td% bucket.getItemAllocationSize() %/td
 td% bucket.getFreeBytes() %/td
 td% bucket.getUsedBytes() %/td
 /tr
 {noformat}
 They are labeled counts but are bytes, duh.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13861) BucketCacheTmpl.jamon has wrong bucket free and used labels


[ 
https://issues.apache.org/jira/browse/HBASE-13861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14611004#comment-14611004
 ] 

Sean Busbey commented on HBASE-13861:
-

aight. ping [~enis] and [~ndimiduk] for 1.0 and 1.1.

 BucketCacheTmpl.jamon has wrong bucket free and used labels
 ---

 Key: HBASE-13861
 URL: https://issues.apache.org/jira/browse/HBASE-13861
 Project: HBase
  Issue Type: Bug
  Components: regionserver, UI
Affects Versions: 1.1.0
Reporter: Lars George
Assignee: Matt Warhaftig
  Labels: beginner
 Fix For: 2.0.0, 0.98.14, 1.2.0

 Attachments: hbase-13861-v1.patch


 See this from the template, and note the label and actual values for the last 
 two columns.
 {noformat}
 table class=table table-striped
 tr
 thBucket Offset/th
 thAllocation Size/th
 thFree Count/th
 thUsed Count/th
 /tr
 %for Bucket bucket: buckets %
 tr
 td% bucket.getBaseOffset() %/td
 td% bucket.getItemAllocationSize() %/td
 td% bucket.getFreeBytes() %/td
 td% bucket.getUsedBytes() %/td
 /tr
 {noformat}
 They are labeled counts but are bytes, duh.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13995) ServerName is not fully case insensitive


[ 
https://issues.apache.org/jira/browse/HBASE-13995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14611033#comment-14611033
 ] 

Hudson commented on HBASE-13995:


SUCCESS: Integrated in HBase-0.98 #1044 (See 
[https://builds.apache.org/job/HBase-0.98/1044/])
HBASE-13995 ServerName is not fully case insensitive (matteo.bertozzi: rev 
4384daea2531d05d0ca16d9016b5316beb4e3fb5)
* hbase-server/src/test/java/org/apache/hadoop/hbase/TestServerName.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/ServerName.java


 ServerName is not fully case insensitive
 

 Key: HBASE-13995
 URL: https://issues.apache.org/jira/browse/HBASE-13995
 Project: HBase
  Issue Type: Bug
  Components: Region Assignment
Affects Versions: 2.0.0, 1.2.0, 0.98.12.1, 1.0.1.1, 1.1.0.1
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: 2.0.0, 0.98.14, 1.0.2, 1.2.0, 1.1.2

 Attachments: HBASE-13995-v0.patch, HBASE-13995-v0.patch


 we ended up with two ServerName with different cases, AAA and aaa.
 Trying to create a table, every once in a while, we ended up with the region 
 lost and not assigned. 
 BaseLoadBalancer.roundRobinAssignment() goes through each server and create a 
 map with what to assign to them.
 We had to server on the list AAA and aaa which are the same machine, the 
 problem is that the round robin now is assigning an empty list to one of the 
 two. so depending on the order we ended up with a region not assigned.
 ServerName equals() does the case insensitive comparison but the hashCode() 
 is done on a case sensitive server name, so the Map in ServerManager will 
 never hit the item and compare it using equals, so we end up with two entries 
 that are the same server. similar thing for 
 ServerName.isSameHostnameAndPort() where we don't check for cases



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14007) Writing to table through MR should fail upfront if table does not exist/disabled


 [ 
https://issues.apache.org/jira/browse/HBASE-14007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thiruvel Thirumoolan updated HBASE-14007:
-
   Labels: mapreduce  (was: )
Fix Version/s: 1.1.2
   1.2.0
Affects Version/s: 1.1.1
   Status: Patch Available  (was: Open)

 Writing to table through MR should fail upfront if table does not 
 exist/disabled
 

 Key: HBASE-14007
 URL: https://issues.apache.org/jira/browse/HBASE-14007
 Project: HBase
  Issue Type: Bug
  Components: mapreduce
Affects Versions: 1.1.1
Reporter: Thiruvel Thirumoolan
Assignee: Thiruvel Thirumoolan
Priority: Minor
  Labels: mapreduce
 Fix For: 1.2.0, 1.1.2

 Attachments: HBASE-14007.patch


 TableOutputFormat.checkOutputSpecs() needs to be fixed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14008) REST - Throw an appropriate error during schema POST


 [ 
https://issues.apache.org/jira/browse/HBASE-14008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thiruvel Thirumoolan updated HBASE-14008:
-
Description: When an update is done on the schema through REST and an error 
occurs, the actual reason is not thrown back to the client. Right now we get a 
javax.ws.rs.WebApplicationException instead of the actual error message.  
(was: Right now we get a javax.ws.rs.WebApplicationException instead of the 
actual error message.)

 REST - Throw an appropriate error during schema POST
 

 Key: HBASE-14008
 URL: https://issues.apache.org/jira/browse/HBASE-14008
 Project: HBase
  Issue Type: Bug
  Components: REST
Affects Versions: 1.1.1
Reporter: Thiruvel Thirumoolan
Assignee: Thiruvel Thirumoolan
Priority: Minor
  Labels: REST
 Fix For: 1.2.0, 1.1.2

 Attachments: HBASE-14008.patch


 When an update is done on the schema through REST and an error occurs, the 
 actual reason is not thrown back to the client. Right now we get a 
 javax.ws.rs.WebApplicationException instead of the actual error message.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11747) ClusterStatus is too bulky

2015-07-01 Thread Mikhail Antonov (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-11747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14611054#comment-14611054
 ] 

Mikhail Antonov commented on HBASE-11747:
-

[~thiruvel] 

bq. We are also exploring the option of compressing the status and sending from 
the server.

Could you please describe you case little more? You're facing this error, or 
just trying to optimize the traffic or something else? Would be interested to 
know the size of cluster/ # of regions you're serving.

 ClusterStatus is too bulky 
 ---

 Key: HBASE-11747
 URL: https://issues.apache.org/jira/browse/HBASE-11747
 Project: HBase
  Issue Type: Sub-task
Reporter: Virag Kothari
 Attachments: exceptiontrace


 Following exception on 0.98 with 1M regions on cluster with 160 region servers
 {code}
 Caused by: java.io.IOException: Call to regionserverhost:port failed on local 
 exception: com.google.protobuf.InvalidProtocolBufferException: Protocol 
 message was too large.  May be malicious.  Use 
 CodedInputStream.setSizeLimit() to increase the size limit.
   at 
 org.apache.hadoop.hbase.ipc.RpcClient.wrapException(RpcClient.java:1482)
   at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1454)
   at 
 org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1654)
   at 
 org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1712)
   at 
 org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.getClusterStatus(MasterProtos.java:42555)
   at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$5.getClusterStatus(HConnectionManager.java:2132)
   at 
 org.apache.hadoop.hbase.client.HBaseAdmin$16.call(HBaseAdmin.java:2166)
   at 
 org.apache.hadoop.hbase.client.HBaseAdmin$16.call(HBaseAdmin.java:2162)
   at 
 org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:114)
   ... 43 more
 Caused by: com.google.protobuf.InvalidProtocolBufferException: Protocol 
 message was too large.  May be malicious.  Use 
 CodedInputStream.setSizeLimit() to increase the size limit.
   at 
 com.google.protobuf.InvalidProtocolBufferException.sizeLimitExceeded(InvalidProtocolBufferException.java:110)
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13980) Distinguish blockedFlushCount vs unblockedFlushCount when tuning heap memory


 [ 
https://issues.apache.org/jira/browse/HBASE-13980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-13980:
---
Resolution: Fixed
Status: Resolved  (was: Patch Available)

 Distinguish blockedFlushCount vs unblockedFlushCount when tuning heap memory
 

 Key: HBASE-13980
 URL: https://issues.apache.org/jira/browse/HBASE-13980
 Project: HBase
  Issue Type: Improvement
Reporter: Ted Yu
Assignee: Abhilash
Priority: Minor
 Fix For: 2.0.0, 1.3.0

 Attachments: HBASE-13980-v1.patch, HBASE-13980-v1.patch, 
 HBASE-13980.patch, HBASE-13980.patch


 Currently DefaultHeapMemoryTuner doesn't distinguish blockedFlushCount vs 
 unblockedFlushCount.
 In its tune() method:
 {code}
 long totalFlushCount = blockedFlushCount+unblockedFlushCount;
 rollingStatsForCacheMisses.insertDataValue(cacheMissCount);
 rollingStatsForFlushes.insertDataValue(totalFlushCount);
 {code}
 Occurrence of blocked flush indicates that upper limit for memstore is not 
 sufficient.
 We should either give blockedFlushCount more weight or, take tuning action 
 based on blockedFlushCount directly.
 See discussion from tail of HBASE-13876.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13861) BucketCacheTmpl.jamon has wrong bucket free and used labels


[ 
https://issues.apache.org/jira/browse/HBASE-13861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14611023#comment-14611023
 ] 

Hudson commented on HBASE-13861:


FAILURE: Integrated in HBase-1.2 #44 (See 
[https://builds.apache.org/job/HBase-1.2/44/])
HBASE-13861 Fix UI label for BucketCache free  used bytes. (busbey: rev 
580495d30fb89d2f749725989fc0985e532aa565)
* 
hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/BlockCacheTmpl.jamon


 BucketCacheTmpl.jamon has wrong bucket free and used labels
 ---

 Key: HBASE-13861
 URL: https://issues.apache.org/jira/browse/HBASE-13861
 Project: HBase
  Issue Type: Bug
  Components: regionserver, UI
Affects Versions: 1.1.0
Reporter: Lars George
Assignee: Matt Warhaftig
  Labels: beginner
 Fix For: 2.0.0, 0.98.14, 1.2.0

 Attachments: hbase-13861-v1.patch


 See this from the template, and note the label and actual values for the last 
 two columns.
 {noformat}
 table class=table table-striped
 tr
 thBucket Offset/th
 thAllocation Size/th
 thFree Count/th
 thUsed Count/th
 /tr
 %for Bucket bucket: buckets %
 tr
 td% bucket.getBaseOffset() %/td
 td% bucket.getItemAllocationSize() %/td
 td% bucket.getFreeBytes() %/td
 td% bucket.getUsedBytes() %/td
 /tr
 {noformat}
 They are labeled counts but are bytes, duh.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13994) Backport HBASE-13917 (Remove string comparison to identify request priority) to 0.98


[ 
https://issues.apache.org/jira/browse/HBASE-13994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14611032#comment-14611032
 ] 

Andrew Purtell commented on HBASE-13994:


Thank you [~stack] you are a saint.


 Backport HBASE-13917 (Remove string comparison to identify request priority) 
 to 0.98
 

 Key: HBASE-13994
 URL: https://issues.apache.org/jira/browse/HBASE-13994
 Project: HBase
  Issue Type: Task
Reporter: Andrew Purtell
Assignee: Andrew Purtell
 Fix For: 0.98.14

 Attachments: HBASE-13994-0.98.patch






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14007) Writing to table through MR should fail upfront if table does not exist/disabled


 [ 
https://issues.apache.org/jira/browse/HBASE-14007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thiruvel Thirumoolan updated HBASE-14007:
-
Attachment: HBASE-14007.patch

 Writing to table through MR should fail upfront if table does not 
 exist/disabled
 

 Key: HBASE-14007
 URL: https://issues.apache.org/jira/browse/HBASE-14007
 Project: HBase
  Issue Type: Bug
Reporter: Thiruvel Thirumoolan
Assignee: Thiruvel Thirumoolan
Priority: Minor
 Attachments: HBASE-14007.patch


 TableOutputFormat.checkOutputSpecs() needs to be fixed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14007) Writing to table through MR should fail upfront if table does not exist/disabled


 [ 
https://issues.apache.org/jira/browse/HBASE-14007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thiruvel Thirumoolan updated HBASE-14007:
-
Component/s: mapreduce

 Writing to table through MR should fail upfront if table does not 
 exist/disabled
 

 Key: HBASE-14007
 URL: https://issues.apache.org/jira/browse/HBASE-14007
 Project: HBase
  Issue Type: Bug
  Components: mapreduce
Reporter: Thiruvel Thirumoolan
Assignee: Thiruvel Thirumoolan
Priority: Minor
 Attachments: HBASE-14007.patch


 TableOutputFormat.checkOutputSpecs() needs to be fixed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14008) REST - Throw an appropriate error during schema POST


 [ 
https://issues.apache.org/jira/browse/HBASE-14008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thiruvel Thirumoolan updated HBASE-14008:
-
Attachment: HBASE-14008.patch

 REST - Throw an appropriate error during schema POST
 

 Key: HBASE-14008
 URL: https://issues.apache.org/jira/browse/HBASE-14008
 Project: HBase
  Issue Type: Bug
  Components: REST
Reporter: Thiruvel Thirumoolan
Assignee: Thiruvel Thirumoolan
Priority: Minor
 Attachments: HBASE-14008.patch


 Right now we get a javax.ws.rs.WebApplicationException instead of the 
 actual error message.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14008) REST - Throw an appropriate error during schema POST


 [ 
https://issues.apache.org/jira/browse/HBASE-14008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thiruvel Thirumoolan updated HBASE-14008:
-
   Labels: REST  (was: )
Fix Version/s: 1.1.2
   1.2.0
Affects Version/s: 1.1.1
   Status: Patch Available  (was: Open)

 REST - Throw an appropriate error during schema POST
 

 Key: HBASE-14008
 URL: https://issues.apache.org/jira/browse/HBASE-14008
 Project: HBase
  Issue Type: Bug
  Components: REST
Affects Versions: 1.1.1
Reporter: Thiruvel Thirumoolan
Assignee: Thiruvel Thirumoolan
Priority: Minor
  Labels: REST
 Fix For: 1.2.0, 1.1.2

 Attachments: HBASE-14008.patch


 Right now we get a javax.ws.rs.WebApplicationException instead of the 
 actual error message.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13339) Update default Hadoop version to 2.6.0


[ 
https://issues.apache.org/jira/browse/HBASE-13339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14611008#comment-14611008
 ] 

Sean Busbey commented on HBASE-13339:
-

Could y'all make sure to post things like that to common-dev@hadoop? Not to put 
to fine a point on it, but if it didn't happen on the mailing list it didn't 
happen.

 Update default Hadoop version to 2.6.0
 --

 Key: HBASE-13339
 URL: https://issues.apache.org/jira/browse/HBASE-13339
 Project: HBase
  Issue Type: Bug
  Components: build
Affects Versions: 2.0.0, 1.1.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Priority: Blocker
 Fix For: 2.0.0, 1.2.0

 Attachments: HBASE-13339-v1.patch, HBASE-13339.patch


 Current default Hadoop version is getting a little long in the tooth. We 
 should update to the latest version. The latest version is backwards 
 compatible with 2.5.1's dfs and mr so this should be painless.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13978) Variable never assigned in SimpleTotalOrderPartitioner.getPartition()