date:20121106

[
https://issues.apache.org/jira/browse/HBASE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491291#comment-13491291
]

stack commented on HBASE-2645:
--

Been playing w/ this one. I can't reproduce in hdfs on trunk or on 1.0 branch
(rough patches attached). If after calling recoverFileLease, I open the file
and keep reading to the end, counting edits, usually I do it once and the
number of edits is constant, even if I open the file again and read. Other
times I see the number of edits climb w/ each opening of the file, often over
seconds.

HLog writer can do 1-2 sync operations after lease has been recovered for
split process.

Key: HBASE-2645
URL: https://issues.apache.org/jira/browse/HBASE-2645
Project: HBase
Issue Type: Bug
Components: Filters
Affects Versions: 0.90.4
Reporter: Cosmin Lehene
Assignee: Todd Lipcon
Priority: Blocker
Fix For: 0.96.0

Attachments: 2645.txt, 2645v2.txt, 2645v3.txt,
hdfs_1.0_editswriter_recoverlease.txt,
hdfs_trunk_editswriter_recoverlease.txt,
org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit-output.txt

TestHLogSplit.testLogCannotBeWrittenOnceParsed is failing.
This test starts a thread that writes one edit to the log, syncs and counts.
During this, a HLog.splitLog operation is started. splitLog recovers the log
lease before reading the log, so that the original regionserver could not
wake up and write after the split process started.
The test compares the number of edits reported by the split process and by
the writer thread. Writer thread (called zombie in the test) should report =
than the splitLog (sync() might raise after the last edit gets written and
the edit won't get counted by zombie thread). However it appears that the
zombie counts 1-2 more edits. So it looks like it can sync without a lease.
This might be a hdfs-0.20 related issue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-2645) HLog writer can do 1-2 sync operations after lease has been recovered for split process.

[
https://issues.apache.org/jira/browse/HBASE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

stack updated HBASE-2645:
-

Attachment: hdfs_trunk_editswriter_recoverlease.txt

Messy patch I was using on hdfs trunk.

HLog writer can do 1-2 sync operations after lease has been recovered for
split process.

Attachments: 2645.txt, 2645v2.txt, 2645v3.txt,
hdfs_1.0_editswriter_recoverlease.txt,
hdfs_trunk_editswriter_recoverlease.txt,
org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit-output.txt

[jira] [Updated] (HBASE-2645) HLog writer can do 1-2 sync operations after lease has been recovered for split process.

[
https://issues.apache.org/jira/browse/HBASE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

stack updated HBASE-2645:
-

Attachment: hdfs_1.0_editswriter_recoverlease.txt

hdfs 1.0 branch version.

HLog writer can do 1-2 sync operations after lease has been recovered for
split process.

Attachments: 2645.txt, 2645v2.txt, 2645v3.txt,
hdfs_1.0_editswriter_recoverlease.txt,
hdfs_trunk_editswriter_recoverlease.txt,
org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit-output.txt

[jira] [Commented] (HBASE-7101) HBase stuck in Region SPLIT

2012-11-06 Thread ramkrishna.s.vasudevan (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-7101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491357#comment-13491357
 ] 

ramkrishna.s.vasudevan commented on HBASE-7101:
---

May be this and HBASE-7103 are different.

 HBase stuck in Region SPLIT 
 

 Key: HBASE-7101
 URL: https://issues.apache.org/jira/browse/HBASE-7101
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.1
Reporter: Bing Jiang
 Fix For: 0.96.0, 0.94.4


 I found this issue from a zknode which has existed for a long time in the 
 unassigned parent.And HMaster report warnning log increasingly.The loop log 
 is at below. 
 WARN org.apache.hadoop.hbase.master.AssignmentManager: Region 
 1a1c950ad45812d7b4b9b90ebf268468 not found on server 
 sev0040,60020,1350378314041; failed processing
 WARN org.apache.hadoop.hbase.master.AssignmentManager: Received SPLIT for 
 region 1a1c950ad45812d7b4b9b90ebf268468 from server 
 sev0040,60020,1350378314041 but it doesn't exist anymore, probably already 
 processed its split
 WARN org.apache.hadoop.hbase.master.AssignmentManager: Region 
 1a1c950ad45812d7b4b9b90ebf268468 not found on server 
 gs-dpo-sev0040,60020,1350378314041; failed processing
 WARN org.apache.hadoop.hbase.master.AssignmentManager: Received SPLIT for 
 region 1a1c950ad45812d7b4b9b90ebf268468 from server 
 sev0040,60020,1350378314041 but it doesn't exist anymore, probably already 
 processed its split
 we use Hbase-0.92.1, and I trace back to the source code. HMaster 
 AssignmentManager have already deleted the SPLIT_Region in its memory 
 structure,but HRegionServer SplitTransaction has found the 
 unassigned/parent-node existed in a transient state, precisely 
 SplitTransaction executes tickleNodeSplit to update a new version a little 
 later than  AssignmentManager deleting unassigned/parent-znode. After 
 updating a version of the znode, it will intrigue the handleRegion operation 
 again, however, AssignmentManager assert that the RegionState in Memory has 
 been deleted, and transaction goes into a retry loop.
 In the SplitTransaction, transitionZKNode will retry tickleNodeSplit after 
 sleeping 100ms. In my opinion, if the time is much longger than 100ms, all 
 the operation from AssignmentManagement will finish off completely.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7103) Need to fail split if SPLIT znode is deleted even before the split is completed.

2012-11-06 Thread ramkrishna.s.vasudevan (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-7103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491358#comment-13491358
]

ramkrishna.s.vasudevan commented on HBASE-7103:
---

Will try to come up with a patch for this.

Need to fail split if SPLIT znode is deleted even before the split is
completed.

Key: HBASE-7103
URL: https://issues.apache.org/jira/browse/HBASE-7103
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan

This came up after the following mail in dev list
'infinite loop of RS_ZK_REGION_SPLIT on .94.2'.
The following is the reason for the problem
The following steps happen
- Initially the parent region P1 starts splitting.
- The split is going on normally.
- Another split starts at the same time for the same region P1. (Not sure
why this started).
- Rollback happens seeing an already existing node.
- This node gets deleted in rollback and nodeDeleted Event starts.
- In nodeDeleted event the RIT for the region P1 gets deleted.
- Because of this there is no region in RIT.
- Now the first split gets over. Here the problem is we try to transit the
node to SPLITTING to SPLIT. But the node even does not exist.
But we don take any action on this. We think it is successful.
- Because of this SplitRegionHandler never gets invoked.

[jira] [Commented] (HBASE-6956) Do not return back to HTablePool closed connections

2012-11-06 Thread shengjie min (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-6956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491365#comment-13491365
 ] 

shengjie min commented on HBASE-6956:
-

[~amuraru]what version you are using?

 Do not return back to HTablePool closed connections
 ---

 Key: HBASE-6956
 URL: https://issues.apache.org/jira/browse/HBASE-6956
 Project: HBase
  Issue Type: Bug
  Components: Client
Affects Versions: 0.90.6
Reporter: Igor Yurinok

 Sometimes we see a lot of Exception about closed connections:
 {code}
  
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@553fd068
  closed
 org.apache.hadoop.hbase.client.ClosedConnectionException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@553fd068
  closed
 {code}
 After investigation we assumed that it occurs because closed connection 
 returns back into HTablePool. 
 For our opinion best solution is  check whether the table is closed in method 
 HTablePool.putTable and if true don't add it into the queue and release such 
 HTableInterface.
 But unfortunatly right now there are no access to HTable#closed field through 
 HTableInterface

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6956) Do not return back to HTablePool closed connections

2012-11-06 Thread Adrian Muraru (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-6956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491367#comment-13491367
 ] 

Adrian Muraru commented on HBASE-6956:
--

[~shengjie_min] 0.92.2

 Do not return back to HTablePool closed connections
 ---

 Key: HBASE-6956
 URL: https://issues.apache.org/jira/browse/HBASE-6956
 Project: HBase
  Issue Type: Bug
  Components: Client
Affects Versions: 0.90.6
Reporter: Igor Yurinok

 Sometimes we see a lot of Exception about closed connections:
 {code}
  
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@553fd068
  closed
 org.apache.hadoop.hbase.client.ClosedConnectionException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@553fd068
  closed
 {code}
 After investigation we assumed that it occurs because closed connection 
 returns back into HTablePool. 
 For our opinion best solution is  check whether the table is closed in method 
 HTablePool.putTable and if true don't add it into the queue and release such 
 HTableInterface.
 But unfortunatly right now there are no access to HTable#closed field through 
 HTableInterface

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7101) HBase stuck in Region SPLIT

2012-11-06 Thread Bing Jiang (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-7101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491386#comment-13491386
 ] 

Bing Jiang commented on HBASE-7101:
---

If znode change state from SPLITTING to SPLIT, please assert HRegionServer 
SplitTransaction wait a long time that HMaster-AssignmentManager has finished 
the clean-up work.Maybe it can add a ZooKeeperWatcher in the SplitTransaction.
  

 HBase stuck in Region SPLIT 
 

 Key: HBASE-7101
 URL: https://issues.apache.org/jira/browse/HBASE-7101
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.1
Reporter: Bing Jiang
 Fix For: 0.96.0, 0.94.4


 I found this issue from a zknode which has existed for a long time in the 
 unassigned parent.And HMaster report warnning log increasingly.The loop log 
 is at below. 
 WARN org.apache.hadoop.hbase.master.AssignmentManager: Region 
 1a1c950ad45812d7b4b9b90ebf268468 not found on server 
 sev0040,60020,1350378314041; failed processing
 WARN org.apache.hadoop.hbase.master.AssignmentManager: Received SPLIT for 
 region 1a1c950ad45812d7b4b9b90ebf268468 from server 
 sev0040,60020,1350378314041 but it doesn't exist anymore, probably already 
 processed its split
 WARN org.apache.hadoop.hbase.master.AssignmentManager: Region 
 1a1c950ad45812d7b4b9b90ebf268468 not found on server 
 gs-dpo-sev0040,60020,1350378314041; failed processing
 WARN org.apache.hadoop.hbase.master.AssignmentManager: Received SPLIT for 
 region 1a1c950ad45812d7b4b9b90ebf268468 from server 
 sev0040,60020,1350378314041 but it doesn't exist anymore, probably already 
 processed its split
 we use Hbase-0.92.1, and I trace back to the source code. HMaster 
 AssignmentManager have already deleted the SPLIT_Region in its memory 
 structure,but HRegionServer SplitTransaction has found the 
 unassigned/parent-node existed in a transient state, precisely 
 SplitTransaction executes tickleNodeSplit to update a new version a little 
 later than  AssignmentManager deleting unassigned/parent-znode. After 
 updating a version of the znode, it will intrigue the handleRegion operation 
 again, however, AssignmentManager assert that the RegionState in Memory has 
 been deleted, and transaction goes into a retry loop.
 In the SplitTransaction, transitionZKNode will retry tickleNodeSplit after 
 sleeping 100ms. In my opinion, if the time is much longger than 100ms, all 
 the operation from AssignmentManagement will finish off completely.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-7104) HBase includes multiple versions of netty: 3.5.0; 3.2.4; 3.2.2

nkeywal created HBASE-7104:
--

 Summary: HBase includes multiple versions of netty: 3.5.0; 3.2.4; 
3.2.2
 Key: HBASE-7104
 URL: https://issues.apache.org/jira/browse/HBASE-7104
 Project: HBase
  Issue Type: Bug
  Components: build
Affects Versions: 0.96.0
Reporter: nkeywal
Assignee: nkeywal
Priority: Minor
 Fix For: 0.96.0


We've got 3 of them on trunk.

[INFO] org.apache.hbase:hbase-server:jar:0.95-SNAPSHOT
[INFO] +- io.netty:netty:jar:3.5.0.Final:compile
[INFO] +- org.apache.zookeeper:zookeeper:jar:3.4.3:compile
[INFO] |  \- org.jboss.netty:netty:jar:3.2.2.Final:compile

[INFO] org.apache.hbase:hbase-hadoop2-compat:jar:0.95-SNAPSHOT
[INFO] +- org.apache.hadoop:hadoop-client:jar:2.0.2-alpha:compile
[INFO] |  +- 
org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.0.2-alpha:compile
[INFO] |  |  \- org.jboss.netty:netty:jar:3.2.4.Final:compile


The patch attached:
- fixes this for hadoop 1 profile
- bump the netty version to 3.5.9
- does not fix it for hadoop 2. I don't know why, but I haven't investigate: as 
it's still alpha may be they will change the version on hadoop side anyway.

Tests are ok.

I haven't really investigated the differences between netty 3.2 and 3.5. A 
quick search seems to say it's ok, but don't hesitate to raise a warning...

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7104) HBase includes multiple versions of netty: 3.5.0; 3.2.4; 3.2.2


 [ 
https://issues.apache.org/jira/browse/HBASE-7104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal updated HBASE-7104:
---

Attachment: 7104.v1.patch

 HBase includes multiple versions of netty: 3.5.0; 3.2.4; 3.2.2
 --

 Key: HBASE-7104
 URL: https://issues.apache.org/jira/browse/HBASE-7104
 Project: HBase
  Issue Type: Bug
  Components: build
Affects Versions: 0.96.0
Reporter: nkeywal
Assignee: nkeywal
Priority: Minor
 Fix For: 0.96.0

 Attachments: 7104.v1.patch


 We've got 3 of them on trunk.
 [INFO] org.apache.hbase:hbase-server:jar:0.95-SNAPSHOT
 [INFO] +- io.netty:netty:jar:3.5.0.Final:compile
 [INFO] +- org.apache.zookeeper:zookeeper:jar:3.4.3:compile
 [INFO] |  \- org.jboss.netty:netty:jar:3.2.2.Final:compile
 [INFO] org.apache.hbase:hbase-hadoop2-compat:jar:0.95-SNAPSHOT
 [INFO] +- org.apache.hadoop:hadoop-client:jar:2.0.2-alpha:compile
 [INFO] |  +- 
 org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.0.2-alpha:compile
 [INFO] |  |  \- org.jboss.netty:netty:jar:3.2.4.Final:compile
 The patch attached:
 - fixes this for hadoop 1 profile
 - bump the netty version to 3.5.9
 - does not fix it for hadoop 2. I don't know why, but I haven't investigate: 
 as it's still alpha may be they will change the version on hadoop side anyway.
 Tests are ok.
 I haven't really investigated the differences between netty 3.2 and 3.5. A 
 quick search seems to say it's ok, but don't hesitate to raise a warning...

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7104) HBase includes multiple versions of netty: 3.5.0; 3.2.4; 3.2.2


 [ 
https://issues.apache.org/jira/browse/HBASE-7104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal updated HBASE-7104:
---

Status: Patch Available  (was: Open)

 HBase includes multiple versions of netty: 3.5.0; 3.2.4; 3.2.2
 --

 Key: HBASE-7104
 URL: https://issues.apache.org/jira/browse/HBASE-7104
 Project: HBase
  Issue Type: Bug
  Components: build
Affects Versions: 0.96.0
Reporter: nkeywal
Assignee: nkeywal
Priority: Minor
 Fix For: 0.96.0

 Attachments: 7104.v1.patch


 We've got 3 of them on trunk.
 [INFO] org.apache.hbase:hbase-server:jar:0.95-SNAPSHOT
 [INFO] +- io.netty:netty:jar:3.5.0.Final:compile
 [INFO] +- org.apache.zookeeper:zookeeper:jar:3.4.3:compile
 [INFO] |  \- org.jboss.netty:netty:jar:3.2.2.Final:compile
 [INFO] org.apache.hbase:hbase-hadoop2-compat:jar:0.95-SNAPSHOT
 [INFO] +- org.apache.hadoop:hadoop-client:jar:2.0.2-alpha:compile
 [INFO] |  +- 
 org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.0.2-alpha:compile
 [INFO] |  |  \- org.jboss.netty:netty:jar:3.2.4.Final:compile
 The patch attached:
 - fixes this for hadoop 1 profile
 - bump the netty version to 3.5.9
 - does not fix it for hadoop 2. I don't know why, but I haven't investigate: 
 as it's still alpha may be they will change the version on hadoop side anyway.
 Tests are ok.
 I haven't really investigated the differences between netty 3.2 and 3.5. A 
 quick search seems to say it's ok, but don't hesitate to raise a warning...

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7104) HBase includes multiple versions of netty: 3.5.0; 3.2.4; 3.2.2


[ 
https://issues.apache.org/jira/browse/HBASE-7104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491402#comment-13491402
 ] 

Hadoop QA commented on HBASE-7104:
--

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12552262/7104.v1.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 hadoop2.0{color}.  The patch compiles against the hadoop 
2.0 profile.

{color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 
77 warning messages.

{color:red}-1 javac{color}.  The patch appears to cause mvn compile goal to 
fail.

{color:red}-1 findbugs{color}.  The patch appears to cause Findbugs 
(version 1.3.9) to fail.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
   org.apache.hadoop.hbase.regionserver.TestSplitTransaction

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3232//testReport/
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3232//console

This message is automatically generated.

 HBase includes multiple versions of netty: 3.5.0; 3.2.4; 3.2.2
 --

 Key: HBASE-7104
 URL: https://issues.apache.org/jira/browse/HBASE-7104
 Project: HBase
  Issue Type: Bug
  Components: build
Affects Versions: 0.96.0
Reporter: nkeywal
Assignee: nkeywal
Priority: Minor
 Fix For: 0.96.0

 Attachments: 7104.v1.patch


 We've got 3 of them on trunk.
 [INFO] org.apache.hbase:hbase-server:jar:0.95-SNAPSHOT
 [INFO] +- io.netty:netty:jar:3.5.0.Final:compile
 [INFO] +- org.apache.zookeeper:zookeeper:jar:3.4.3:compile
 [INFO] |  \- org.jboss.netty:netty:jar:3.2.2.Final:compile
 [INFO] org.apache.hbase:hbase-hadoop2-compat:jar:0.95-SNAPSHOT
 [INFO] +- org.apache.hadoop:hadoop-client:jar:2.0.2-alpha:compile
 [INFO] |  +- 
 org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.0.2-alpha:compile
 [INFO] |  |  \- org.jboss.netty:netty:jar:3.2.4.Final:compile
 The patch attached:
 - fixes this for hadoop 1 profile
 - bump the netty version to 3.5.9
 - does not fix it for hadoop 2. I don't know why, but I haven't investigate: 
 as it's still alpha may be they will change the version on hadoop side anyway.
 Tests are ok.
 I haven't really investigated the differences between netty 3.2 and 3.5. A 
 quick search seems to say it's ok, but don't hesitate to raise a warning...

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6356) printStackTrace in FSUtils

2012-11-06 Thread Gustavo Anatoly (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-6356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gustavo Anatoly updated HBASE-6356:
---

Attachment: (was: HBASE-6356.patch)

 printStackTrace in FSUtils
 --

 Key: HBASE-6356
 URL: https://issues.apache.org/jira/browse/HBASE-6356
 Project: HBase
  Issue Type: Bug
  Components: Client, master, regionserver
Affects Versions: 0.96.0
Reporter: nkeywal
Priority: Trivial
  Labels: noob

 This is bad...
 {noformat}
 public boolean accept(Path p) {
   boolean isValid = false;
   try {
 if (HConstants.HBASE_NON_USER_TABLE_DIRS.contains(p.toString())) {
   isValid = false;
 } else {
 isValid = this.fs.getFileStatus(p).isDir();
 }
   } catch (IOException e) {
 e.printStackTrace();   
   }
   return isValid;
 }
   }
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6356) printStackTrace in FSUtils

2012-11-06 Thread Gustavo Anatoly (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-6356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491463#comment-13491463
 ] 

Gustavo Anatoly commented on HBASE-6356:


Hi, nkeywal.

Could you verify if you are agree with this message to LOG.warn?  

{noformat}
public boolean accept(Path p) {
  boolean isValid = false;
  try {
if (HConstants.HBASE_NON_USER_TABLE_DIRS.contains(p.toString())) {
  isValid = false;
} else {
  isValid = this.fs.getFileStatus(p).isDir();
}
  } catch (IOException e) {
LOG.warn(An error occured while verify [ + p.toString() + ] if is a 
valid directory., e);
  }
  return isValid;
}
{noformat}

If you are agree, I will submit a new patch.

Thanks.

 printStackTrace in FSUtils
 --

 Key: HBASE-6356
 URL: https://issues.apache.org/jira/browse/HBASE-6356
 Project: HBase
  Issue Type: Bug
  Components: Client, master, regionserver
Affects Versions: 0.96.0
Reporter: nkeywal
Priority: Trivial
  Labels: noob

 This is bad...
 {noformat}
 public boolean accept(Path p) {
   boolean isValid = false;
   try {
 if (HConstants.HBASE_NON_USER_TABLE_DIRS.contains(p.toString())) {
   isValid = false;
 } else {
 isValid = this.fs.getFileStatus(p).isDir();
 }
   } catch (IOException e) {
 e.printStackTrace();   
   }
   return isValid;
 }
   }
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6356) printStackTrace in FSUtils


[ 
https://issues.apache.org/jira/browse/HBASE-6356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491488#comment-13491488
 ] 

nkeywal commented on HBASE-6356:


Hi Gustavo,

What do you think of 
LOG.warn(An error occured while verifying [ + p.toString() + ] if is a 
valid directory. Returning 'not valid' and continuing., e);

It's worth checking what Path#toString() says as well (is it really a name?)

Thanks!

Nicolas


 printStackTrace in FSUtils
 --

 Key: HBASE-6356
 URL: https://issues.apache.org/jira/browse/HBASE-6356
 Project: HBase
  Issue Type: Bug
  Components: Client, master, regionserver
Affects Versions: 0.96.0
Reporter: nkeywal
Priority: Trivial
  Labels: noob

 This is bad...
 {noformat}
 public boolean accept(Path p) {
   boolean isValid = false;
   try {
 if (HConstants.HBASE_NON_USER_TABLE_DIRS.contains(p.toString())) {
   isValid = false;
 } else {
 isValid = this.fs.getFileStatus(p).isDir();
 }
   } catch (IOException e) {
 e.printStackTrace();   
   }
   return isValid;
 }
   }
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7103) Need to fail split if SPLIT znode is deleted even before the split is completed.

[
https://issues.apache.org/jira/browse/HBASE-7103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Lars Hofhansl updated HBASE-7103:
-

Fix Version/s: 0.96.0
0.94.3

Need to fail split if SPLIT znode is deleted even before the split is
completed.

Key: HBASE-7103
URL: https://issues.apache.org/jira/browse/HBASE-7103
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.94.3, 0.96.0

[jira] [Commented] (HBASE-7099) [89-fb] Create compaction hooks

2012-11-06 Thread Kannan Muthukkaruppan (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-7099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491565#comment-13491565
 ] 

Kannan Muthukkaruppan commented on HBASE-7099:
--

We were thinking of this as an emit/write time compaction hook. It won't be 
in the general read path, but in compactStores(). By the time a value gets 
here, TTL purging etc. would have already been done. Whatever survives, and is 
about to be written to the output (of compaction) file-- this hook will give a 
chance to intercept  transform/skip the value portion of the KV.


 [89-fb] Create compaction hooks 
 

 Key: HBASE-7099
 URL: https://issues.apache.org/jira/browse/HBASE-7099
 Project: HBase
  Issue Type: New Feature
Reporter: Adela Maznikar
Assignee: Adela Maznikar

 Create compaction hooks which should be used to modify the value of the KV 
 during compaction. The possible modifications would include: skipping the KV, 
 modifying the value of the KV in the desired format, leaving the value as it 
 was.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7103) Need to fail split if SPLIT znode is deleted even before the split is completed.

2012-11-06 Thread ramkrishna.s.vasudevan (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-7103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491653#comment-13491653
]

ramkrishna.s.vasudevan commented on HBASE-7103:
---

Trying to write a testcase for this? Can we fail the split if the znode is not
present? But my doubt is if a split is currently going on for the region A and
if another split is called for the same region how should we handle it?
Ideally this prob is caused because of the rollback that is done by the second
split.
Thinking on this. Any suggestions?

Need to fail split if SPLIT znode is deleted even before the split is
completed.

Key: HBASE-7103
URL: https://issues.apache.org/jira/browse/HBASE-7103
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.94.3, 0.96.0

[jira] [Commented] (HBASE-4583) Integrate RWCC with Append and Increment operations

2012-11-06 Thread Jean-Daniel Cryans (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491654#comment-13491654
 ] 

Jean-Daniel Cryans commented on HBASE-4583:
---

In my latest comment I meant to say that I'm +1 on the more radical patch for 
0.96

 Integrate RWCC with Append and Increment operations
 ---

 Key: HBASE-4583
 URL: https://issues.apache.org/jira/browse/HBASE-4583
 Project: HBase
  Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.96.0

 Attachments: 4583-trunk-less-radical.txt, 
 4583-trunk-less-radical-v2.txt, 4583-trunk-less-radical-v3.txt, 
 4583-trunk-less-radical-v4.txt, 4583-trunk-less-radical-v5.txt, 
 4583-trunk-less-radical-v6.txt, 4583-trunk-radical.txt, 
 4583-trunk-radical_v2.txt, 4583-trunk-v3.txt, 4583.txt, 4583-v2.txt, 
 4583-v3.txt, 4583-v4.txt


 Currently Increment and Append operations do not work with RWCC and hence a 
 client could see the results of multiple such operation mixed in the same 
 Get/Scan.
 The semantics might be a bit more interesting here as upsert adds and removes 
 to and from the memstore.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7103) Need to fail split if SPLIT znode is deleted even before the split is completed.

2012-11-06 Thread Jean-Daniel Cryans (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-7103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491656#comment-13491656
]

Jean-Daniel Cryans commented on HBASE-7103:
---

I wonder which jira introduced this issue as it seems that it wasn't present
before 0.94.2

Need to fail split if SPLIT znode is deleted even before the split is
completed.

Key: HBASE-7103
URL: https://issues.apache.org/jira/browse/HBASE-7103
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.94.3, 0.96.0

[jira] [Commented] (HBASE-7103) Need to fail split if SPLIT znode is deleted even before the split is completed.

2012-11-06 Thread Matt Corgan (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-7103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491669#comment-13491669
]

Matt Corgan commented on HBASE-7103:

From step 3, do you think the double-splitting is a new phenomenon? It
doesn't sound like something that should happen very often. Maybe that would
explain why i didn't get this error in .94.0.

Also, please note I went straight from .94.0 to .94.2, so I don't know if it
was present in .94.1.

Need to fail split if SPLIT znode is deleted even before the split is
completed.

Key: HBASE-7103
URL: https://issues.apache.org/jira/browse/HBASE-7103
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.94.3, 0.96.0

[jira] [Commented] (HBASE-6874) Implement prefetching for scanners

2012-11-06 Thread Karthik Ranganathan (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-6874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491695#comment-13491695
]

Karthik Ranganathan commented on HBASE-6874:

Lars - the dependency on HBASE-6770 is more to make the code simpler.
Currently, the HRegionServer loops over numRows, and the RegionScanner loops
over the columns in the various CF's but for one row. HBASE-6770 will move the
looping on the numRows into the RegionScanner itself, because we need to track
both memory size and number of rows - in order to respect the more restrictive
of the two. Once that happens, we can implement prefetching in the
RegionScanner itself, instead of spreading the logic in HRegionServer also. So
more of a code-simplicity and not having to resolve conflicts thing.

Implement prefetching for scanners
--

Key: HBASE-6874
URL: https://issues.apache.org/jira/browse/HBASE-6874
Project: HBase
Issue Type: Sub-task
Reporter: Karthik Ranganathan
Assignee: Karthik Ranganathan

I did some quick experiments by scanning data that should be completely in
memory and found that adding pre-fetching increases the throughput by about
50% from 26MB/s to 39MB/s.
The idea is to perform the next in a background thread, and keep the result
ready. When the scanner's next comes in, return the pre-computed result and
issue another background read.

[jira] [Commented] (HBASE-3743) Throttle major compaction

2012-11-06 Thread Otis Gospodnetic (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-3743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491694#comment-13491694
 ] 

Otis Gospodnetic commented on HBASE-3743:
-

Is this issue still needed or did HBASE-5867 take care of compaction throttling?


 Throttle major compaction
 -

 Key: HBASE-3743
 URL: https://issues.apache.org/jira/browse/HBASE-3743
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Reporter: Joep Rottinghuis

 Add the ability to throttle major compaction.
 For those use cases when a stop-the-world approach is not practical, it is 
 useful to be able to throttle the impact that major compaction has on the 
 cluster.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6775) Use ZK.multi when available for HBASE-6710 0.92/0.94 compatibility fix

2012-11-06 Thread Gregory Chanan (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-6775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491701#comment-13491701
 ] 

Gregory Chanan commented on HBASE-6775:
---

Review Board: https://issues.apache.org/jira/browse/HBASE-6775

 Use ZK.multi when available for HBASE-6710 0.92/0.94 compatibility fix
 --

 Key: HBASE-6775
 URL: https://issues.apache.org/jira/browse/HBASE-6775
 Project: HBase
  Issue Type: Improvement
  Components: Zookeeper
Affects Versions: 0.94.2
Reporter: Gregory Chanan
Assignee: Gregory Chanan
Priority: Minor
 Fix For: 0.94.4


 HBASE-6710 fixed a 0.92/0.94 compatibility issue by writing two znodes in 
 different formats.
 If a ZK failure occurs between the writing of the two znodes, strange 
 behavior can result.
 This issue is a reminder to change these two ZK writes to use ZK.multi when 
 we require ZK 3.4+. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6775) Use ZK.multi when available for HBASE-6710 0.92/0.94 compatibility fix

2012-11-06 Thread Gregory Chanan (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-6775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491702#comment-13491702
 ] 

Gregory Chanan commented on HBASE-6775:
---

Woops, wrong copy-paste: https://reviews.apache.org/r/7888/

 Use ZK.multi when available for HBASE-6710 0.92/0.94 compatibility fix
 --

 Key: HBASE-6775
 URL: https://issues.apache.org/jira/browse/HBASE-6775
 Project: HBase
  Issue Type: Improvement
  Components: Zookeeper
Affects Versions: 0.94.2
Reporter: Gregory Chanan
Assignee: Gregory Chanan
Priority: Minor
 Fix For: 0.94.4


 HBASE-6710 fixed a 0.92/0.94 compatibility issue by writing two znodes in 
 different formats.
 If a ZK failure occurs between the writing of the two znodes, strange 
 behavior can result.
 This issue is a reminder to change these two ZK writes to use ZK.multi when 
 we require ZK 3.4+. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6371) [89-fb] Tier based compaction

[
https://issues.apache.org/jira/browse/HBASE-6371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Liyin Tang updated HBASE-6371:
--

Attachment: (was: HBase_Tier_Base_Compaction.pdf)

[89-fb] Tier based compaction
-

Key: HBASE-6371
URL: https://issues.apache.org/jira/browse/HBASE-6371
Project: HBase
Issue Type: Improvement
Reporter: Akashnil
Assignee: Liyin Tang
Labels: noob
Attachments: HBASE-6371-089fb-commit.patch,
HBase_Tier_Base_Compaction.pdf

Currently, the compaction selection is not very flexible and is not sensitive
to the hotness of the data. Very old data is likely to be accessed less, and
very recent data is likely to be in the block cache. Both of these
considerations make it inefficient to compact these files as aggressively as
other files. In some use-cases, the access-pattern is particularly obvious
even though there is no way to control the compaction algorithm in those
cases.
In the new compaction selection algorithm, we plan to divide the candidate
files into different levels according to oldness of the data that is present
in those files. For each level, parameters like compaction ratio, minimum
number of store-files in each compaction may be different. Number of levels,
time-ranges, and parameters for each level will be configurable online on a
per-column family basis.

[jira] [Updated] (HBASE-6371) [89-fb] Tier based compaction

[
https://issues.apache.org/jira/browse/HBASE-6371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Liyin Tang updated HBASE-6371:
--

Attachment: HBase_Tier_Base_Compaction.pdf

[89-fb] Tier based compaction
-

[jira] [Commented] (HBASE-6894) Adding metadata to a table in the shell is both arcane and painful

[
https://issues.apache.org/jira/browse/HBASE-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491711#comment-13491711
]

stack commented on HBASE-6894:
--

MS Ill fix the below comments on help screens on commit (They are not your
issues -- I just saw them reviewing this):

In alter help, this example does not illustrate the point that you can alter
multiple familes at once:

hbase alter 't1', 'f1', {NAME = 'f2', IN_MEMORY = true}, 'f3'

Lets not have two ways to do something:

hbase alter 't1', NAME = 'f1', METHOD = 'delete'
hbase alter 't1', 'delete' = 'f1'

Should we explain CONFIG? Its not explained in create command.

Question for you Sergey:

How do I remove something from CONFIG? Maybe you can't just now. For another
issue.

Table attributes are currently anonymous... They look the same as a CF only w/o
a NAME = attribute. Should they be named as in
TABLE_ATTRIBUTES = {CONFIG {'x' = 'y', 'OWNER' = 'SOMEOWNER'}}

METHOD = 'table_att' as analog for CF NAME = 'name' was ugly. I'm glad you
removed that... just wondering if we should add something back to say what they
are.

I'm up for commit. This fixes a bunch of stuff. It unearths the fact that
there is a load more work to do in here but that is not your fault.

Adding metadata to a table in the shell is both arcane and painful
--

Key: HBASE-6894
URL: https://issues.apache.org/jira/browse/HBASE-6894
Project: HBase
Issue Type: Bug
Components: shell
Affects Versions: 0.96.0
Reporter: stack
Assignee: Sergey Shelukhin
Labels: noob
Attachments: HBASE-6894.patch, HBASE-6894.patch, HBASE-6894.patch,
HBASE-6894-v2.patch, HBASE-6894-v3.1-squashed.patch,
HBASE-6894-v3-squashed.patch, HBASE-6894-v4-squashed.patch

In production we have hundreds of tables w/ whack names like 'aliaserv',
'ashish_bulk', 'age_gender_topics', etc. It be grand if you could look in
master UI and see stuff like owner, eng group responsible, miscellaneous
description, etc.
Now, HTD has support for this; each carries a dictionary. Whats a PITA
though is adding attributes to the dictionary. Here is what seems to work on
trunk (though I do not trust it is doing the right thing):
{code}
hbase create 'SOME_TABLENAME', {NAME = 'd', VERSION = 1, COMPRESSION =
'LZO'}
hbase # Here is how I added metadata
hbase disable 'SOME_TABLENAME'
hbase alter 'SOME_TABLENAME', METHOD = 'table_att', OWNER = 'SOMEON',
CONFIG = {'ENVIRONMENT' = 'BLAH BLAH', 'SIZING' = 'The size should be
between 0-10K most of the time with new URLs coming in and getting removed as
they are processed unless the pipeline has fallen behind', 'MISCELLANEOUS' =
'Holds the list of URLs waiting to be processed in the parked page detection
analyzer in ingestion pipeline.'}
...
describe...
enable...
{code}
The above doesn't work in 0.94. Complains about the CONFIG, the keyword we
are using for the HTD dictionary.
It works in 0.96 though I'd have to poke around some more to ensure it is
doing the right thing.
But this METHOD = 'table_att' stuff is really ugly can we fix it?
And I can't add table attributes on table create seemingly.
A little bit of thought and a bit of ruby could clean this all up.

[jira] [Commented] (HBASE-7092) RegionServer OOM related to onlineregions


[ 
https://issues.apache.org/jira/browse/HBASE-7092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491712#comment-13491712
 ] 

stack commented on HBASE-7092:
--

How many regions per server?   4G or RAM?  I should add a note to important 
configs. in refguide Jimmy?  In 0.96 slab is on by default.

 RegionServer OOM related to onlineregions
 -

 Key: HBASE-7092
 URL: https://issues.apache.org/jira/browse/HBASE-7092
 Project: HBase
  Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Blocker
 Fix For: 0.96.0

 Attachments: heap-dump.png


 One instance of java.util.concurrent.ConcurrentHashMap loaded by system 
 class loader occupies 3,972,154,848 (92.88%) bytes. The instance is 
 referenced by org.apache.hadoop.hbase.regionserver.HRegionServer @ 
 0x7038d3798 , loaded by sun.misc.Launcher$AppClassLoader @ 0x703994668. The 
 memory is accumulated in one instance of 
 java.util.concurrent.ConcurrentHashMap$Segment[] loaded by system class 
 loader.
 Keywords
 sun.misc.Launcher$AppClassLoader @ 0x703994668
 java.util.concurrent.ConcurrentHashMap
 java.util.concurrent.ConcurrentHashMap$Segment[]

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6793) Make hbase-examples module


[ 
https://issues.apache.org/jira/browse/HBASE-6793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491713#comment-13491713
 ] 

stack commented on HBASE-6793:
--

I'll commit this unless objection later today.

 Make hbase-examples module
 --

 Key: HBASE-6793
 URL: https://issues.apache.org/jira/browse/HBASE-6793
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.96.0
Reporter: Enis Soztutar
Assignee: Sergey Shelukhin
  Labels: noob
 Attachments: HBASE-6793.patch, HBASE-6793-v2.patch, 
 HBASE-6793-v3-thrift-0.9.0.patch, HBASE-6793-v4.1-squashed.patch, 
 HBASE-6793-v5-squashed.patch


 There are some examples under /examples/, which are not compiled as a part of 
 the build. We can move them to an hbase-examples module.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7092) RegionServer OOM related to onlineregions


[ 
https://issues.apache.org/jira/browse/HBASE-7092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491715#comment-13491715
 ] 

Jimmy Xiang commented on HBASE-7092:


I have about 2600 regions per server, and just 4G max heap. Slab is on by 
default in hbase-default.xml, not the code.

 RegionServer OOM related to onlineregions
 -

 Key: HBASE-7092
 URL: https://issues.apache.org/jira/browse/HBASE-7092
 Project: HBase
  Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Blocker
 Fix For: 0.96.0

 Attachments: heap-dump.png


 One instance of java.util.concurrent.ConcurrentHashMap loaded by system 
 class loader occupies 3,972,154,848 (92.88%) bytes. The instance is 
 referenced by org.apache.hadoop.hbase.regionserver.HRegionServer @ 
 0x7038d3798 , loaded by sun.misc.Launcher$AppClassLoader @ 0x703994668. The 
 memory is accumulated in one instance of 
 java.util.concurrent.ConcurrentHashMap$Segment[] loaded by system class 
 loader.
 Keywords
 sun.misc.Launcher$AppClassLoader @ 0x703994668
 java.util.concurrent.ConcurrentHashMap
 java.util.concurrent.ConcurrentHashMap$Segment[]

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-2645) HLog writer can do 1-2 sync operations after lease has been recovered for split process.

[
https://issues.apache.org/jira/browse/HBASE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491727#comment-13491727
]

stack commented on HBASE-2645:
--

Adding logging, I can see the 'master' return from the recover file lease call
against the namenode. We then go to open the 'wal' to read the edits out of it
splitting. Meantime, I see that the writer thread is still progressing adding
edits until it stalls in a sync call. This means the read will miss edits
(lots in this test case).

HLog writer can do 1-2 sync operations after lease has been recovered for
split process.

Attachments: 2645.txt, 2645v2.txt, 2645v3.txt,
hdfs_1.0_editswriter_recoverlease.txt,
hdfs_trunk_editswriter_recoverlease.txt,
org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit-output.txt

[jira] [Updated] (HBASE-6410) Move RegionServer Metrics to metrics2

2012-11-06 Thread Elliott Clark (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-6410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Elliott Clark updated HBASE-6410:
-

Attachment: HBASE-6410-18.patch

Rebased on trunk, and what I will be committing.

 Move RegionServer Metrics to metrics2
 -

 Key: HBASE-6410
 URL: https://issues.apache.org/jira/browse/HBASE-6410
 Project: HBase
  Issue Type: Sub-task
  Components: metrics
Affects Versions: 0.96.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Priority: Blocker
 Attachments: HBASE-6410-13.patch, HBASE-6410-15.patch, 
 HBASE-6410-16.patch, HBASE-6410-18.patch, HBASE-6410-1.patch, 
 HBASE-6410-2.patch, HBASE-6410-3.patch, HBASE-6410-4.patch, 
 HBASE-6410-5.patch, HBASE-6410-6.patch, HBASE-6410.patch


 Move RegionServer Metrics to metrics2

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-2645) HLog writer can do 1-2 sync operations after lease has been recovered for split process.

[
https://issues.apache.org/jira/browse/HBASE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

stack updated HBASE-2645:
-

Attachment:
org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit-output.txt

Log showing us missing writes.

HLog writer can do 1-2 sync operations after lease has been recovered for
split process.

Attachments: 2645.txt, 2645v2.txt, 2645v3.txt,
hdfs_1.0_editswriter_recoverlease.txt,
hdfs_trunk_editswriter_recoverlease.txt,
org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit-output.txt,
org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit-output.txt

[jira] [Commented] (HBASE-5898) Consider double-checked locking for block cache lock


[ 
https://issues.apache.org/jira/browse/HBASE-5898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491732#comment-13491732
 ] 

Lars Hofhansl commented on HBASE-5898:
--

If nobody is willing to sponsor this (by committing), maybe it should not be 
committed...?


 Consider double-checked locking for block cache lock
 

 Key: HBASE-5898
 URL: https://issues.apache.org/jira/browse/HBASE-5898
 Project: HBase
  Issue Type: Improvement
  Components: Performance
Affects Versions: 0.94.1
Reporter: Todd Lipcon
Assignee: Todd Lipcon
Priority: Critical
 Fix For: 0.94.3, 0.96.0

 Attachments: 5898-TestBlocksRead.txt, HBASE-5898-0.patch, 
 HBASE-5898-1.patch, hbase-5898.txt


 Running a workload with a high query rate against a dataset that fits in 
 cache, I saw a lot of CPU being used in IdLock.getLockEntry, being called by 
 HFileReaderV2.readBlock. Even though it was all cache hits, it was wasting a 
 lot of CPU doing lock management here. I wrote a quick patch to switch to a 
 double-checked locking and it improved throughput substantially for this 
 workload.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-7105) RS throws NPE on forcing compaction from HBase shell on a single bulk imported file.

2012-11-06 Thread Karthik Ranganathan (JIRA)

Karthik Ranganathan created HBASE-7105:
--

 Summary: RS throws NPE on forcing compaction from HBase shell on a 
single bulk imported file.
 Key: HBASE-7105
 URL: https://issues.apache.org/jira/browse/HBASE-7105
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Reporter: Karthik Ranganathan
Assignee: Karthik Ranganathan


In StoreFile, we have:
private AtomicBoolean majorCompaction = null;

In StoreFile.open(), we do:
b = metadataMap.get(MAJOR_COMPACTION_KEY);
if (b != null) {
  // init majorCompaction variable
}

Because the file was bulk imported, this is never initialized. Any subsequent 
call to isMajorCompaction() NPE's.

Fix is to initialize it to false.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6874) Implement prefetching for scanners


[ 
https://issues.apache.org/jira/browse/HBASE-6874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491740#comment-13491740
 ] 

Lars Hofhansl commented on HBASE-6874:
--

Oh, I see. I had assumed the prefetching would be implemented on the client 
only (the RPCs for scanning could be scheduled asynchronously, so the caller 
never has to wait for them); but obviously there are many ways to do this.


 Implement prefetching for scanners
 --

 Key: HBASE-6874
 URL: https://issues.apache.org/jira/browse/HBASE-6874
 Project: HBase
  Issue Type: Sub-task
Reporter: Karthik Ranganathan
Assignee: Karthik Ranganathan

 I did some quick experiments by scanning data that should be completely in 
 memory and found that adding pre-fetching increases the throughput by about 
 50% from 26MB/s to 39MB/s.
 The idea is to perform the next in a background thread, and keep the result 
 ready. When the scanner's next comes in, return the pre-computed result and 
 issue another background read.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6991) Escape \ in Bytes.toStringBinary() and its counterpart Bytes.toBytesBinary()

2012-11-06 Thread Aditya Kishore (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-6991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491751#comment-13491751
 ] 

Aditya Kishore commented on HBASE-6991:
---

[~yuzhih...@gmail.com], [~stack] Could you please review this.

https://reviews.apache.org/r/7632/

 Escape \ in Bytes.toStringBinary() and its counterpart Bytes.toBytesBinary()
 --

 Key: HBASE-6991
 URL: https://issues.apache.org/jira/browse/HBASE-6991
 Project: HBase
  Issue Type: Bug
  Components: util
Affects Versions: 0.96.0
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Fix For: 0.96.0

 Attachments: HBASE-6991_trunk.patch


 Since \ is used to escape non-printable character but not treated as 
 special character in conversion, it could lead to unexpected conversion.
 For example, please consider the following code snippet.
 {code}
 public void testConversion() {
   byte[] original = {
   '\\', 'x', 'A', 'D'
   };
   String stringFromBytes = Bytes.toStringBinary(original);
   byte[] converted = Bytes.toBytesBinary(stringFromBytes);
   System.out.println(Original:  + Arrays.toString(original));
   System.out.println(Converted:  + Arrays.toString(converted));
   System.out.println(Reversible?:  + (Bytes.compareTo(original, converted) 
 == 0));
 }
 Output:
 ---
 Original: [92, 120, 65, 68]
 Converted: [-83]
 Reversible?: false
 {code}
 The \ character needs to be treated as special and must be encoded as a 
 non-printable character (\x5C) to avoid any kind of ambiguity during 
 conversion.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6356) printStackTrace in FSUtils

2012-11-06 Thread Gustavo Anatoly (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-6356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491754#comment-13491754
 ] 

Gustavo Anatoly commented on HBASE-6356:


Hi, Nicolas.

About the message LOG.warn I think is good, because the log reader will 
understand what happened, and continue with a not valid return, but about your 
second question, would you like that check if Path#toString() is really a name 
before passed to the LOG.warn?

Thanks for your patience.

 printStackTrace in FSUtils
 --

 Key: HBASE-6356
 URL: https://issues.apache.org/jira/browse/HBASE-6356
 Project: HBase
  Issue Type: Bug
  Components: Client, master, regionserver
Affects Versions: 0.96.0
Reporter: nkeywal
Priority: Trivial
  Labels: noob

 This is bad...
 {noformat}
 public boolean accept(Path p) {
   boolean isValid = false;
   try {
 if (HConstants.HBASE_NON_USER_TABLE_DIRS.contains(p.toString())) {
   isValid = false;
 } else {
 isValid = this.fs.getFileStatus(p).isDir();
 }
   } catch (IOException e) {
 e.printStackTrace();   
   }
   return isValid;
 }
   }
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4583) Integrate RWCC with Append and Increment operations


[ 
https://issues.apache.org/jira/browse/HBASE-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491765#comment-13491765
 ] 

stack commented on HBASE-4583:
--

I'd be +1 for radical patch in 0.96.

 Integrate RWCC with Append and Increment operations
 ---

 Key: HBASE-4583
 URL: https://issues.apache.org/jira/browse/HBASE-4583
 Project: HBase
  Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.96.0

 Attachments: 4583-trunk-less-radical.txt, 
 4583-trunk-less-radical-v2.txt, 4583-trunk-less-radical-v3.txt, 
 4583-trunk-less-radical-v4.txt, 4583-trunk-less-radical-v5.txt, 
 4583-trunk-less-radical-v6.txt, 4583-trunk-radical.txt, 
 4583-trunk-radical_v2.txt, 4583-trunk-v3.txt, 4583.txt, 4583-v2.txt, 
 4583-v3.txt, 4583-v4.txt


 Currently Increment and Append operations do not work with RWCC and hence a 
 client could see the results of multiple such operation mixed in the same 
 Get/Scan.
 The semantics might be a bit more interesting here as upsert adds and removes 
 to and from the memstore.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4583) Integrate RWCC with Append and Increment operations


[ 
https://issues.apache.org/jira/browse/HBASE-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491768#comment-13491768
 ] 

stack commented on HBASE-4583:
--

I think I understand the implications -- noticeably slower (TBD) -- but more 
obviously correct and for those who come after, more obvious what the right way 
(correctness) forward is.  Looking at the patch, do we suffer version explosion 
(I don't see memstore cleanup going on).

 Integrate RWCC with Append and Increment operations
 ---

 Key: HBASE-4583
 URL: https://issues.apache.org/jira/browse/HBASE-4583
 Project: HBase
  Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.96.0

 Attachments: 4583-trunk-less-radical.txt, 
 4583-trunk-less-radical-v2.txt, 4583-trunk-less-radical-v3.txt, 
 4583-trunk-less-radical-v4.txt, 4583-trunk-less-radical-v5.txt, 
 4583-trunk-less-radical-v6.txt, 4583-trunk-radical.txt, 
 4583-trunk-radical_v2.txt, 4583-trunk-v3.txt, 4583.txt, 4583-v2.txt, 
 4583-v3.txt, 4583-v4.txt


 Currently Increment and Append operations do not work with RWCC and hence a 
 client could see the results of multiple such operation mixed in the same 
 Get/Scan.
 The semantics might be a bit more interesting here as upsert adds and removes 
 to and from the memstore.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7103) Need to fail split if SPLIT znode is deleted even before the split is completed.

[
https://issues.apache.org/jira/browse/HBASE-7103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491770#comment-13491770
]

Lars Hofhansl commented on HBASE-7103:
--

On the mailing list I had posted these as candidates:

HBASE-6854
HBASE-6329
HBASE-6088
HBASE-6070
HBASE-6713
HBASE-5986

Some of these deal with split during regionserver stops.

[~mcorgan] I assume you're still restarting servers occasionally?

Need to fail split if SPLIT znode is deleted even before the split is
completed.

Key: HBASE-7103
URL: https://issues.apache.org/jira/browse/HBASE-7103
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.94.3, 0.96.0

[jira] [Commented] (HBASE-4583) Integrate RWCC with Append and Increment operations


[ 
https://issues.apache.org/jira/browse/HBASE-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491776#comment-13491776
 ] 

Lars Hofhansl commented on HBASE-4583:
--

Yes, the radical version has no cleanup (that is what upsert does). Not 
cleaning up makes it slab friendly and also fullfills the VERSIONS contracts, 
but it will lead to more frequent region flushes (at flush time most of the KVs 
will be removed, but it will flush a tiny HFile).


 Integrate RWCC with Append and Increment operations
 ---

 Key: HBASE-4583
 URL: https://issues.apache.org/jira/browse/HBASE-4583
 Project: HBase
  Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.96.0

 Attachments: 4583-trunk-less-radical.txt, 
 4583-trunk-less-radical-v2.txt, 4583-trunk-less-radical-v3.txt, 
 4583-trunk-less-radical-v4.txt, 4583-trunk-less-radical-v5.txt, 
 4583-trunk-less-radical-v6.txt, 4583-trunk-radical.txt, 
 4583-trunk-radical_v2.txt, 4583-trunk-v3.txt, 4583.txt, 4583-v2.txt, 
 4583-v3.txt, 4583-v4.txt


 Currently Increment and Append operations do not work with RWCC and hence a 
 client could see the results of multiple such operation mixed in the same 
 Get/Scan.
 The semantics might be a bit more interesting here as upsert adds and removes 
 to and from the memstore.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6991) Escape \ in Bytes.toStringBinary() and its counterpart Bytes.toBytesBinary()


 [ 
https://issues.apache.org/jira/browse/HBASE-6991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-6991:
-

  Resolution: Fixed
Hadoop Flags: Incompatible change,Reviewed  (was: Incompatible change)
  Status: Resolved  (was: Patch Available)

Committed to trunk.  Thanks for the patch Aditya.  Nice one.  That already 
encoded stuff should be undone properly should protect us against pre 0.96 data 
looking different in 0.96.

 Escape \ in Bytes.toStringBinary() and its counterpart Bytes.toBytesBinary()
 --

 Key: HBASE-6991
 URL: https://issues.apache.org/jira/browse/HBASE-6991
 Project: HBase
  Issue Type: Bug
  Components: util
Affects Versions: 0.96.0
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Fix For: 0.96.0

 Attachments: HBASE-6991_trunk.patch


 Since \ is used to escape non-printable character but not treated as 
 special character in conversion, it could lead to unexpected conversion.
 For example, please consider the following code snippet.
 {code}
 public void testConversion() {
   byte[] original = {
   '\\', 'x', 'A', 'D'
   };
   String stringFromBytes = Bytes.toStringBinary(original);
   byte[] converted = Bytes.toBytesBinary(stringFromBytes);
   System.out.println(Original:  + Arrays.toString(original));
   System.out.println(Converted:  + Arrays.toString(converted));
   System.out.println(Reversible?:  + (Bytes.compareTo(original, converted) 
 == 0));
 }
 Output:
 ---
 Original: [92, 120, 65, 68]
 Converted: [-83]
 Reversible?: false
 {code}
 The \ character needs to be treated as special and must be encoded as a 
 non-printable character (\x5C) to avoid any kind of ambiguity during 
 conversion.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4583) Integrate RWCC with Append and Increment operations


[ 
https://issues.apache.org/jira/browse/HBASE-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491779#comment-13491779
 ] 

stack commented on HBASE-4583:
--

How would we do cleanup?  Delete from behind smallest read point?  Do this in a 
memstore compressor step?  Something that sits between current memstore and the 
snapshot we make when flushing?

 Integrate RWCC with Append and Increment operations
 ---

 Key: HBASE-4583
 URL: https://issues.apache.org/jira/browse/HBASE-4583
 Project: HBase
  Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.96.0

 Attachments: 4583-trunk-less-radical.txt, 
 4583-trunk-less-radical-v2.txt, 4583-trunk-less-radical-v3.txt, 
 4583-trunk-less-radical-v4.txt, 4583-trunk-less-radical-v5.txt, 
 4583-trunk-less-radical-v6.txt, 4583-trunk-radical.txt, 
 4583-trunk-radical_v2.txt, 4583-trunk-v3.txt, 4583.txt, 4583-v2.txt, 
 4583-v3.txt, 4583-v4.txt


 Currently Increment and Append operations do not work with RWCC and hence a 
 client could see the results of multiple such operation mixed in the same 
 Get/Scan.
 The semantics might be a bit more interesting here as upsert adds and removes 
 to and from the memstore.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5898) Consider double-checked locking for block cache lock


[ 
https://issues.apache.org/jira/browse/HBASE-5898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491781#comment-13491781
 ] 

stack commented on HBASE-5898:
--

This patch was difference between a stalling sick cluster and one that worked 
properly again.  I think it has to be committed (off-heap cache should not be 
in the way of committing this issue).

 Consider double-checked locking for block cache lock
 

 Key: HBASE-5898
 URL: https://issues.apache.org/jira/browse/HBASE-5898
 Project: HBase
  Issue Type: Improvement
  Components: Performance
Affects Versions: 0.94.1
Reporter: Todd Lipcon
Assignee: Todd Lipcon
Priority: Critical
 Fix For: 0.94.3, 0.96.0

 Attachments: 5898-TestBlocksRead.txt, HBASE-5898-0.patch, 
 HBASE-5898-1.patch, hbase-5898.txt


 Running a workload with a high query rate against a dataset that fits in 
 cache, I saw a lot of CPU being used in IdLock.getLockEntry, being called by 
 HFileReaderV2.readBlock. Even though it was all cache hits, it was wasting a 
 lot of CPU doing lock management here. I wrote a quick patch to switch to a 
 double-checked locking and it improved throughput substantially for this 
 workload.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6410) Move RegionServer Metrics to metrics2

[
https://issues.apache.org/jira/browse/HBASE-6410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491782#comment-13491782
]

Hadoop QA commented on HBASE-6410:
--

{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12552324/HBASE-6410-18.patch
against trunk revision .

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:green}+1 tests included{color}. The patch appears to include 164
new or modified tests.

{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.

{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
87 warning messages.

{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.

{color:red}-1 findbugs{color}. The patch appears to introduce 8 new
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.

{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestRegionRebalancing
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3233//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3233//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3233//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3233//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3233//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3233//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3233//console

This message is automatically generated.

Move RegionServer Metrics to metrics2
-

Key: HBASE-6410
URL: https://issues.apache.org/jira/browse/HBASE-6410
Project: HBase
Issue Type: Sub-task
Components: metrics
Affects Versions: 0.96.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Priority: Blocker
Attachments: HBASE-6410-13.patch, HBASE-6410-15.patch,
HBASE-6410-16.patch, HBASE-6410-18.patch, HBASE-6410-1.patch,
HBASE-6410-2.patch, HBASE-6410-3.patch, HBASE-6410-4.patch,
HBASE-6410-5.patch, HBASE-6410-6.patch, HBASE-6410.patch

Move RegionServer Metrics to metrics2

[jira] [Commented] (HBASE-7092) RegionServer OOM related to onlineregions


[ 
https://issues.apache.org/jira/browse/HBASE-7092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491786#comment-13491786
 ] 

stack commented on HBASE-7092:
--

I added a note to the MSLAB section that lots of regions could cause you OOME 
(MSLAB is on since 0.92, not 0.96 as I said above)

 RegionServer OOM related to onlineregions
 -

 Key: HBASE-7092
 URL: https://issues.apache.org/jira/browse/HBASE-7092
 Project: HBase
  Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Blocker
 Fix For: 0.96.0

 Attachments: heap-dump.png


 One instance of java.util.concurrent.ConcurrentHashMap loaded by system 
 class loader occupies 3,972,154,848 (92.88%) bytes. The instance is 
 referenced by org.apache.hadoop.hbase.regionserver.HRegionServer @ 
 0x7038d3798 , loaded by sun.misc.Launcher$AppClassLoader @ 0x703994668. The 
 memory is accumulated in one instance of 
 java.util.concurrent.ConcurrentHashMap$Segment[] loaded by system class 
 loader.
 Keywords
 sun.misc.Launcher$AppClassLoader @ 0x703994668
 java.util.concurrent.ConcurrentHashMap
 java.util.concurrent.ConcurrentHashMap$Segment[]

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6865) Snapshot File Cleaners


[ 
https://issues.apache.org/jira/browse/HBASE-6865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491787#comment-13491787
 ] 

Jonathan Hsieh commented on HBASE-6865:
---

review is here. https://reviews.apache.org/r/7627/

 Snapshot File Cleaners
 --

 Key: HBASE-6865
 URL: https://issues.apache.org/jira/browse/HBASE-6865
 Project: HBase
  Issue Type: Sub-task
Reporter: Jesse Yates
Assignee: Jesse Yates
 Fix For: hbase-6055


 HFile and HLog cleaners are necessary for fully implementing snapshots, but 
 can be broken out into their own piece really cleanly. The HFile cleaner is 
 necessary for both timestamp and globally consistent snapshots, but the HLog 
 cleaner is necessary for globally consistent and offline snapshots. 
 Putting everything together in one patch isn't too much overload as there is 
 a lot of overlap between the cleaners.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HBASE-6958) TestAssignmentManager sometimes fails in trunk


 [ 
https://issues.apache.org/jira/browse/HBASE-6958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang reassigned HBASE-6958:
--

Assignee: Jimmy Xiang

 TestAssignmentManager sometimes fails in trunk
 --

 Key: HBASE-6958
 URL: https://issues.apache.org/jira/browse/HBASE-6958
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Assignee: Jimmy Xiang
 Fix For: 0.96.0


 From 
 https://builds.apache.org/job/HBase-TRUNK/3432/testReport/junit/org.apache.hadoop.hbase.master/TestAssignmentManager/testBalanceOnMasterFailoverScenarioWithOpenedNode/
  :
 {code}
 Stacktrace
 java.lang.Exception: test timed out after 5000 milliseconds
   at java.lang.System.arraycopy(Native Method)
   at java.lang.ThreadGroup.remove(ThreadGroup.java:969)
   at java.lang.ThreadGroup.threadTerminated(ThreadGroup.java:942)
   at java.lang.Thread.exit(Thread.java:732)
 ...
 2012-10-06 00:46:12,521 DEBUG [MASTER_CLOSE_REGION-mockedAMExecutor-0] 
 zookeeper.ZKUtil(1141): mockedServer-0x13a33892de7000e Retrieved 81 byte(s) 
 of data from znode /hbase/unassigned/dc01abf9cd7fd0ea256af4df02811640 and set 
 watcher; region=t,,1349484359011.dc01abf9cd7fd0ea256af4df02811640., 
 state=M_ZK_REGION_OFFLINE, servername=master,1,1, createTime=1349484372509, 
 payload.length=0
 2012-10-06 00:46:12,522 ERROR [MASTER_CLOSE_REGION-mockedAMExecutor-0] 
 executor.EventHandler(205): Caught throwable while processing event 
 RS_ZK_REGION_CLOSED
 java.lang.NullPointerException
   at 
 org.apache.hadoop.hbase.master.TestAssignmentManager$MockedLoadBalancer.randomAssignment(TestAssignmentManager.java:773)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.getRegionPlan(AssignmentManager.java:1709)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.getRegionPlan(AssignmentManager.java:1666)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1435)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1155)
   at 
 org.apache.hadoop.hbase.master.TestAssignmentManager$AssignmentManagerWithExtrasForTesting.assign(TestAssignmentManager.java:1035)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1130)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1125)
   at 
 org.apache.hadoop.hbase.master.handler.ClosedRegionHandler.process(ClosedRegionHandler.java:106)
   at 
 org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:202)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
   at java.lang.Thread.run(Thread.java:722)
 2012-10-06 00:46:12,522 DEBUG [pool-1-thread-1-EventThread] 
 master.AssignmentManager(670): Handling transition=M_ZK_REGION_OFFLINE, 
 server=master,1,1, region=dc01abf9cd7fd0ea256af4df02811640, current state 
 from region state map ={t,,1349484359011.dc01abf9cd7fd0ea256af4df02811640. 
 state=OFFLINE, ts=1349484372508, server=null}
 {code}
 Looks like NPE happened on this line:
 {code}
   this.gate.set(true);
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6865) Snapshot File Cleaners


[ 
https://issues.apache.org/jira/browse/HBASE-6865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491793#comment-13491793
 ] 

Jonathan Hsieh commented on HBASE-6865:
---

High order bits.  from v2

1) There is a cache for hfile directory listings and a separate cache for hlog 
file directory listings.  
2) There are two main caches likely get plugged into the archive cleaner (but 
it doesn't seem to happen in this patch).
4) There are some a non standard use of a setter that is in the is a limitation 
of the cleaner api.  There should be a follow on issue to help that be more 
conventional.


 Snapshot File Cleaners
 --

 Key: HBASE-6865
 URL: https://issues.apache.org/jira/browse/HBASE-6865
 Project: HBase
  Issue Type: Sub-task
Reporter: Jesse Yates
Assignee: Jesse Yates
 Fix For: hbase-6055


 HFile and HLog cleaners are necessary for fully implementing snapshots, but 
 can be broken out into their own piece really cleanly. The HFile cleaner is 
 necessary for both timestamp and globally consistent snapshots, but the HLog 
 cleaner is necessary for globally consistent and offline snapshots. 
 Putting everything together in one patch isn't too much overload as there is 
 a lot of overlap between the cleaners.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7103) Need to fail split if SPLIT znode is deleted even before the split is completed.

2012-11-06 Thread Matt Corgan (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-7103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491798#comment-13491798
]

Matt Corgan commented on HBASE-7103:

So far I don't think it has to do with stopping the regionserver since we're
only doing that after this happens. I also haven't seen anything suggesting it
has to do with the META table.

Any ideas on what would cause a split to fail and retry? Is it more likely
caused by some internal regionserver problem, or the region being moved during
the split, etc?

Need to fail split if SPLIT znode is deleted even before the split is
completed.

Key: HBASE-7103
URL: https://issues.apache.org/jira/browse/HBASE-7103
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.94.3, 0.96.0

[jira] [Commented] (HBASE-7055) port HBASE-6371 tier-based compaction from 0.89-fb to trunk


[ 
https://issues.apache.org/jira/browse/HBASE-7055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491799#comment-13491799
 ] 

stack commented on HBASE-7055:
--

[~enis] and [~sershe]... so what about CompactionManager naming?  Enis wants 
CompactionPolicy which I think is better too.  Want me to do the rename on 
commit?  I'd also rename StoreUtils and StoreTools on commit (And I'll commit 
this in a different issue than this one, in an issue called refactoring 
compaction code)

 port HBASE-6371 tier-based compaction from 0.89-fb to trunk
 ---

 Key: HBASE-7055
 URL: https://issues.apache.org/jira/browse/HBASE-7055
 Project: HBase
  Issue Type: Task
  Components: Compaction
Affects Versions: 0.96.0
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
 Fix For: 0.96.0

 Attachments: HBASE-6371-squashed.patch, HBASE-6371-v2-squashed.patch, 
 HBASE-6371-v3-refactor-only-squashed.patch, 
 HBASE-6371-v4-refactor-only-squashed.patch, 
 HBASE-6371-v5-refactor-only-squashed.patch


 There's divergence in the code :(
 See HBASE-6371 for details.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6573) Distributed Three-Phase Commit framework.

[
https://issues.apache.org/jira/browse/HBASE-6573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491802#comment-13491802
]

Jonathan Hsieh commented on HBASE-6573:
---

Update on this - I've been re-organizing the code here to simplify it. A draft
can be seen here:

https://github.com/jmhsieh/hbase/tree/snapshot-ts-threadchange

I'm currently working on getting the TS snapshot functionality passing unit
tests again. Once I get that I'll, summarize the changes and rebase/port onto
the current snapshot branch.

Some key things to note.

1) This is a framework that uses zookeeper as an rpc/coordination mechanism.
Masters and region servers are only communicating via ZK.
2) This isn't a commit protocol -- if errors occur after the traditional 2pc
coordinator commit point the current behavior is to reset (remove znodes) or
and essentially abort. Traditional 2PC/3PC style algorithms would recover
from any failure after this commit point. This mechanism seems to be overkill
for log-rolling or timestamp snapshots. For the globally-consistent flavor, I
think we really want a distributed 2 phase locking protocol instead of a 2
phase commit protocol.

For the time being, my opinion is to work to get this chunk of code into the
snapshot branch, and then do patches on top of this to further simplify the
implementation.

Distributed Three-Phase Commit framework.
-

Key: HBASE-6573
URL: https://issues.apache.org/jira/browse/HBASE-6573
Project: HBase
Issue Type: Sub-task
Affects Versions: 0.96.0
Reporter: Jesse Yates
Assignee: Jesse Yates
Fix For: hbase-6055

Attachments: java_HBASE-6573-v0.patch

For HBASE-6055 (snapshots), we do two-phase commit in several places. This is
a generally useful paradigm for a distributed system.

[jira] [Updated] (HBASE-6573) Distributed Three-Phase Commit framework.


 [ 
https://issues.apache.org/jira/browse/HBASE-6573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-6573:
--

Attachment: Procedure coordination.pdf

Attached some slides with a cartoon showing how this implementation is 
different from 2pc.

 Distributed Three-Phase Commit framework.
 -

 Key: HBASE-6573
 URL: https://issues.apache.org/jira/browse/HBASE-6573
 Project: HBase
  Issue Type: Sub-task
Affects Versions: 0.96.0
Reporter: Jesse Yates
Assignee: Jesse Yates
 Fix For: hbase-6055

 Attachments: java_HBASE-6573-v0.patch, Procedure coordination.pdf


 For HBASE-6055 (snapshots), we do two-phase commit in several places. This is 
 a generally useful paradigm for a distributed system.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6958) TestAssignmentManager sometimes fails


 [ 
https://issues.apache.org/jira/browse/HBASE-6958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang updated HBASE-6958:
---

Fix Version/s: 0.94.4
  Summary: TestAssignmentManager sometimes fails  (was: 
TestAssignmentManager sometimes fails in trunk)

I don't see it fail in trunk anymore.  However, I can see it still fails in 
0.94.

 TestAssignmentManager sometimes fails
 -

 Key: HBASE-6958
 URL: https://issues.apache.org/jira/browse/HBASE-6958
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Assignee: Jimmy Xiang
 Fix For: 0.96.0, 0.94.4


 From 
 https://builds.apache.org/job/HBase-TRUNK/3432/testReport/junit/org.apache.hadoop.hbase.master/TestAssignmentManager/testBalanceOnMasterFailoverScenarioWithOpenedNode/
  :
 {code}
 Stacktrace
 java.lang.Exception: test timed out after 5000 milliseconds
   at java.lang.System.arraycopy(Native Method)
   at java.lang.ThreadGroup.remove(ThreadGroup.java:969)
   at java.lang.ThreadGroup.threadTerminated(ThreadGroup.java:942)
   at java.lang.Thread.exit(Thread.java:732)
 ...
 2012-10-06 00:46:12,521 DEBUG [MASTER_CLOSE_REGION-mockedAMExecutor-0] 
 zookeeper.ZKUtil(1141): mockedServer-0x13a33892de7000e Retrieved 81 byte(s) 
 of data from znode /hbase/unassigned/dc01abf9cd7fd0ea256af4df02811640 and set 
 watcher; region=t,,1349484359011.dc01abf9cd7fd0ea256af4df02811640., 
 state=M_ZK_REGION_OFFLINE, servername=master,1,1, createTime=1349484372509, 
 payload.length=0
 2012-10-06 00:46:12,522 ERROR [MASTER_CLOSE_REGION-mockedAMExecutor-0] 
 executor.EventHandler(205): Caught throwable while processing event 
 RS_ZK_REGION_CLOSED
 java.lang.NullPointerException
   at 
 org.apache.hadoop.hbase.master.TestAssignmentManager$MockedLoadBalancer.randomAssignment(TestAssignmentManager.java:773)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.getRegionPlan(AssignmentManager.java:1709)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.getRegionPlan(AssignmentManager.java:1666)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1435)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1155)
   at 
 org.apache.hadoop.hbase.master.TestAssignmentManager$AssignmentManagerWithExtrasForTesting.assign(TestAssignmentManager.java:1035)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1130)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1125)
   at 
 org.apache.hadoop.hbase.master.handler.ClosedRegionHandler.process(ClosedRegionHandler.java:106)
   at 
 org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:202)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
   at java.lang.Thread.run(Thread.java:722)
 2012-10-06 00:46:12,522 DEBUG [pool-1-thread-1-EventThread] 
 master.AssignmentManager(670): Handling transition=M_ZK_REGION_OFFLINE, 
 server=master,1,1, region=dc01abf9cd7fd0ea256af4df02811640, current state 
 from region state map ={t,,1349484359011.dc01abf9cd7fd0ea256af4df02811640. 
 state=OFFLINE, ts=1349484372508, server=null}
 {code}
 Looks like NPE happened on this line:
 {code}
   this.gate.set(true);
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6991) Escape \ in Bytes.toStringBinary() and its counterpart Bytes.toBytesBinary()

2012-11-06 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-6991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491808#comment-13491808
 ] 

Hudson commented on HBASE-6991:
---

Integrated in HBase-TRUNK #3514 (See 
[https://builds.apache.org/job/HBase-TRUNK/3514/])
HBASE-6991 Escape \ in Bytes.toStringBinary() and its counterpart 
Bytes.toBytesBinary() (Revision 1406297)

 Result = SUCCESS
stack : 
Files : 
* 
/hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/util/Bytes.java
* 
/hbase/trunk/hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestBytes.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlock.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlockCompatibility.java


 Escape \ in Bytes.toStringBinary() and its counterpart Bytes.toBytesBinary()
 --

 Key: HBASE-6991
 URL: https://issues.apache.org/jira/browse/HBASE-6991
 Project: HBase
  Issue Type: Bug
  Components: util
Affects Versions: 0.96.0
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Fix For: 0.96.0

 Attachments: HBASE-6991_trunk.patch


 Since \ is used to escape non-printable character but not treated as 
 special character in conversion, it could lead to unexpected conversion.
 For example, please consider the following code snippet.
 {code}
 public void testConversion() {
   byte[] original = {
   '\\', 'x', 'A', 'D'
   };
   String stringFromBytes = Bytes.toStringBinary(original);
   byte[] converted = Bytes.toBytesBinary(stringFromBytes);
   System.out.println(Original:  + Arrays.toString(original));
   System.out.println(Converted:  + Arrays.toString(converted));
   System.out.println(Reversible?:  + (Bytes.compareTo(original, converted) 
 == 0));
 }
 Output:
 ---
 Original: [92, 120, 65, 68]
 Converted: [-83]
 Reversible?: false
 {code}
 The \ character needs to be treated as special and must be encoded as a 
 non-printable character (\x5C) to avoid any kind of ambiguity during 
 conversion.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4583) Integrate RWCC with Append and Increment operations

2012-11-06 Thread Varun Sharma (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491817#comment-13491817
 ] 

Varun Sharma commented on HBASE-4583:
-

I am more +1 on the less radical patch - the more frequent flushing and tiny 
HFiles means that counters are now using a much larger fraction of the memstore 
as opposed to before. Also, more hfiles means that data will be more fragmented 
on disk until compaction happens.

 Integrate RWCC with Append and Increment operations
 ---

 Key: HBASE-4583
 URL: https://issues.apache.org/jira/browse/HBASE-4583
 Project: HBase
  Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.96.0

 Attachments: 4583-trunk-less-radical.txt, 
 4583-trunk-less-radical-v2.txt, 4583-trunk-less-radical-v3.txt, 
 4583-trunk-less-radical-v4.txt, 4583-trunk-less-radical-v5.txt, 
 4583-trunk-less-radical-v6.txt, 4583-trunk-radical.txt, 
 4583-trunk-radical_v2.txt, 4583-trunk-v3.txt, 4583.txt, 4583-v2.txt, 
 4583-v3.txt, 4583-v4.txt


 Currently Increment and Append operations do not work with RWCC and hence a 
 client could see the results of multiple such operation mixed in the same 
 Get/Scan.
 The semantics might be a bit more interesting here as upsert adds and removes 
 to and from the memstore.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6958) TestAssignmentManager sometimes fails


 [ 
https://issues.apache.org/jira/browse/HBASE-6958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang updated HBASE-6958:
---

Attachment: 6958_0.94.patch

 TestAssignmentManager sometimes fails
 -

 Key: HBASE-6958
 URL: https://issues.apache.org/jira/browse/HBASE-6958
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: Ted Yu
Assignee: Jimmy Xiang
 Fix For: 0.96.0, 0.94.4

 Attachments: 6958_0.94.patch, trunk-6958.patch


 From 
 https://builds.apache.org/job/HBase-TRUNK/3432/testReport/junit/org.apache.hadoop.hbase.master/TestAssignmentManager/testBalanceOnMasterFailoverScenarioWithOpenedNode/
  :
 {code}
 Stacktrace
 java.lang.Exception: test timed out after 5000 milliseconds
   at java.lang.System.arraycopy(Native Method)
   at java.lang.ThreadGroup.remove(ThreadGroup.java:969)
   at java.lang.ThreadGroup.threadTerminated(ThreadGroup.java:942)
   at java.lang.Thread.exit(Thread.java:732)
 ...
 2012-10-06 00:46:12,521 DEBUG [MASTER_CLOSE_REGION-mockedAMExecutor-0] 
 zookeeper.ZKUtil(1141): mockedServer-0x13a33892de7000e Retrieved 81 byte(s) 
 of data from znode /hbase/unassigned/dc01abf9cd7fd0ea256af4df02811640 and set 
 watcher; region=t,,1349484359011.dc01abf9cd7fd0ea256af4df02811640., 
 state=M_ZK_REGION_OFFLINE, servername=master,1,1, createTime=1349484372509, 
 payload.length=0
 2012-10-06 00:46:12,522 ERROR [MASTER_CLOSE_REGION-mockedAMExecutor-0] 
 executor.EventHandler(205): Caught throwable while processing event 
 RS_ZK_REGION_CLOSED
 java.lang.NullPointerException
   at 
 org.apache.hadoop.hbase.master.TestAssignmentManager$MockedLoadBalancer.randomAssignment(TestAssignmentManager.java:773)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.getRegionPlan(AssignmentManager.java:1709)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.getRegionPlan(AssignmentManager.java:1666)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1435)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1155)
   at 
 org.apache.hadoop.hbase.master.TestAssignmentManager$AssignmentManagerWithExtrasForTesting.assign(TestAssignmentManager.java:1035)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1130)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1125)
   at 
 org.apache.hadoop.hbase.master.handler.ClosedRegionHandler.process(ClosedRegionHandler.java:106)
   at 
 org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:202)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
   at java.lang.Thread.run(Thread.java:722)
 2012-10-06 00:46:12,522 DEBUG [pool-1-thread-1-EventThread] 
 master.AssignmentManager(670): Handling transition=M_ZK_REGION_OFFLINE, 
 server=master,1,1, region=dc01abf9cd7fd0ea256af4df02811640, current state 
 from region state map ={t,,1349484359011.dc01abf9cd7fd0ea256af4df02811640. 
 state=OFFLINE, ts=1349484372508, server=null}
 {code}
 Looks like NPE happened on this line:
 {code}
   this.gate.set(true);
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6958) TestAssignmentManager sometimes fails


 [ 
https://issues.apache.org/jira/browse/HBASE-6958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang updated HBASE-6958:
---

Attachment: trunk-6958.patch

 TestAssignmentManager sometimes fails
 -

 Key: HBASE-6958
 URL: https://issues.apache.org/jira/browse/HBASE-6958
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: Ted Yu
Assignee: Jimmy Xiang
 Fix For: 0.96.0, 0.94.4

 Attachments: 6958_0.94.patch, trunk-6958.patch


 From 
 https://builds.apache.org/job/HBase-TRUNK/3432/testReport/junit/org.apache.hadoop.hbase.master/TestAssignmentManager/testBalanceOnMasterFailoverScenarioWithOpenedNode/
  :
 {code}
 Stacktrace
 java.lang.Exception: test timed out after 5000 milliseconds
   at java.lang.System.arraycopy(Native Method)
   at java.lang.ThreadGroup.remove(ThreadGroup.java:969)
   at java.lang.ThreadGroup.threadTerminated(ThreadGroup.java:942)
   at java.lang.Thread.exit(Thread.java:732)
 ...
 2012-10-06 00:46:12,521 DEBUG [MASTER_CLOSE_REGION-mockedAMExecutor-0] 
 zookeeper.ZKUtil(1141): mockedServer-0x13a33892de7000e Retrieved 81 byte(s) 
 of data from znode /hbase/unassigned/dc01abf9cd7fd0ea256af4df02811640 and set 
 watcher; region=t,,1349484359011.dc01abf9cd7fd0ea256af4df02811640., 
 state=M_ZK_REGION_OFFLINE, servername=master,1,1, createTime=1349484372509, 
 payload.length=0
 2012-10-06 00:46:12,522 ERROR [MASTER_CLOSE_REGION-mockedAMExecutor-0] 
 executor.EventHandler(205): Caught throwable while processing event 
 RS_ZK_REGION_CLOSED
 java.lang.NullPointerException
   at 
 org.apache.hadoop.hbase.master.TestAssignmentManager$MockedLoadBalancer.randomAssignment(TestAssignmentManager.java:773)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.getRegionPlan(AssignmentManager.java:1709)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.getRegionPlan(AssignmentManager.java:1666)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1435)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1155)
   at 
 org.apache.hadoop.hbase.master.TestAssignmentManager$AssignmentManagerWithExtrasForTesting.assign(TestAssignmentManager.java:1035)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1130)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1125)
   at 
 org.apache.hadoop.hbase.master.handler.ClosedRegionHandler.process(ClosedRegionHandler.java:106)
   at 
 org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:202)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
   at java.lang.Thread.run(Thread.java:722)
 2012-10-06 00:46:12,522 DEBUG [pool-1-thread-1-EventThread] 
 master.AssignmentManager(670): Handling transition=M_ZK_REGION_OFFLINE, 
 server=master,1,1, region=dc01abf9cd7fd0ea256af4df02811640, current state 
 from region state map ={t,,1349484359011.dc01abf9cd7fd0ea256af4df02811640. 
 state=OFFLINE, ts=1349484372508, server=null}
 {code}
 Looks like NPE happened on this line:
 {code}
   this.gate.set(true);
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6958) TestAssignmentManager sometimes fails


 [ 
https://issues.apache.org/jira/browse/HBASE-6958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang updated HBASE-6958:
---

Component/s: test

 TestAssignmentManager sometimes fails
 -

 Key: HBASE-6958
 URL: https://issues.apache.org/jira/browse/HBASE-6958
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: Ted Yu
Assignee: Jimmy Xiang
 Fix For: 0.96.0, 0.94.4

 Attachments: 6958_0.94.patch, trunk-6958.patch


 From 
 https://builds.apache.org/job/HBase-TRUNK/3432/testReport/junit/org.apache.hadoop.hbase.master/TestAssignmentManager/testBalanceOnMasterFailoverScenarioWithOpenedNode/
  :
 {code}
 Stacktrace
 java.lang.Exception: test timed out after 5000 milliseconds
   at java.lang.System.arraycopy(Native Method)
   at java.lang.ThreadGroup.remove(ThreadGroup.java:969)
   at java.lang.ThreadGroup.threadTerminated(ThreadGroup.java:942)
   at java.lang.Thread.exit(Thread.java:732)
 ...
 2012-10-06 00:46:12,521 DEBUG [MASTER_CLOSE_REGION-mockedAMExecutor-0] 
 zookeeper.ZKUtil(1141): mockedServer-0x13a33892de7000e Retrieved 81 byte(s) 
 of data from znode /hbase/unassigned/dc01abf9cd7fd0ea256af4df02811640 and set 
 watcher; region=t,,1349484359011.dc01abf9cd7fd0ea256af4df02811640., 
 state=M_ZK_REGION_OFFLINE, servername=master,1,1, createTime=1349484372509, 
 payload.length=0
 2012-10-06 00:46:12,522 ERROR [MASTER_CLOSE_REGION-mockedAMExecutor-0] 
 executor.EventHandler(205): Caught throwable while processing event 
 RS_ZK_REGION_CLOSED
 java.lang.NullPointerException
   at 
 org.apache.hadoop.hbase.master.TestAssignmentManager$MockedLoadBalancer.randomAssignment(TestAssignmentManager.java:773)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.getRegionPlan(AssignmentManager.java:1709)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.getRegionPlan(AssignmentManager.java:1666)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1435)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1155)
   at 
 org.apache.hadoop.hbase.master.TestAssignmentManager$AssignmentManagerWithExtrasForTesting.assign(TestAssignmentManager.java:1035)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1130)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1125)
   at 
 org.apache.hadoop.hbase.master.handler.ClosedRegionHandler.process(ClosedRegionHandler.java:106)
   at 
 org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:202)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
   at java.lang.Thread.run(Thread.java:722)
 2012-10-06 00:46:12,522 DEBUG [pool-1-thread-1-EventThread] 
 master.AssignmentManager(670): Handling transition=M_ZK_REGION_OFFLINE, 
 server=master,1,1, region=dc01abf9cd7fd0ea256af4df02811640, current state 
 from region state map ={t,,1349484359011.dc01abf9cd7fd0ea256af4df02811640. 
 state=OFFLINE, ts=1349484372508, server=null}
 {code}
 Looks like NPE happened on this line:
 {code}
   this.gate.set(true);
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6958) TestAssignmentManager sometimes fails


 [ 
https://issues.apache.org/jira/browse/HBASE-6958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang updated HBASE-6958:
---

Status: Patch Available  (was: Open)

Looked into it.  The failure is because the execution order.  Some test changes 
some setting without reverting it back afterwards.

 TestAssignmentManager sometimes fails
 -

 Key: HBASE-6958
 URL: https://issues.apache.org/jira/browse/HBASE-6958
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: Ted Yu
Assignee: Jimmy Xiang
 Fix For: 0.96.0, 0.94.4

 Attachments: 6958_0.94.patch, trunk-6958.patch


 From 
 https://builds.apache.org/job/HBase-TRUNK/3432/testReport/junit/org.apache.hadoop.hbase.master/TestAssignmentManager/testBalanceOnMasterFailoverScenarioWithOpenedNode/
  :
 {code}
 Stacktrace
 java.lang.Exception: test timed out after 5000 milliseconds
   at java.lang.System.arraycopy(Native Method)
   at java.lang.ThreadGroup.remove(ThreadGroup.java:969)
   at java.lang.ThreadGroup.threadTerminated(ThreadGroup.java:942)
   at java.lang.Thread.exit(Thread.java:732)
 ...
 2012-10-06 00:46:12,521 DEBUG [MASTER_CLOSE_REGION-mockedAMExecutor-0] 
 zookeeper.ZKUtil(1141): mockedServer-0x13a33892de7000e Retrieved 81 byte(s) 
 of data from znode /hbase/unassigned/dc01abf9cd7fd0ea256af4df02811640 and set 
 watcher; region=t,,1349484359011.dc01abf9cd7fd0ea256af4df02811640., 
 state=M_ZK_REGION_OFFLINE, servername=master,1,1, createTime=1349484372509, 
 payload.length=0
 2012-10-06 00:46:12,522 ERROR [MASTER_CLOSE_REGION-mockedAMExecutor-0] 
 executor.EventHandler(205): Caught throwable while processing event 
 RS_ZK_REGION_CLOSED
 java.lang.NullPointerException
   at 
 org.apache.hadoop.hbase.master.TestAssignmentManager$MockedLoadBalancer.randomAssignment(TestAssignmentManager.java:773)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.getRegionPlan(AssignmentManager.java:1709)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.getRegionPlan(AssignmentManager.java:1666)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1435)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1155)
   at 
 org.apache.hadoop.hbase.master.TestAssignmentManager$AssignmentManagerWithExtrasForTesting.assign(TestAssignmentManager.java:1035)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1130)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1125)
   at 
 org.apache.hadoop.hbase.master.handler.ClosedRegionHandler.process(ClosedRegionHandler.java:106)
   at 
 org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:202)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
   at java.lang.Thread.run(Thread.java:722)
 2012-10-06 00:46:12,522 DEBUG [pool-1-thread-1-EventThread] 
 master.AssignmentManager(670): Handling transition=M_ZK_REGION_OFFLINE, 
 server=master,1,1, region=dc01abf9cd7fd0ea256af4df02811640, current state 
 from region state map ={t,,1349484359011.dc01abf9cd7fd0ea256af4df02811640. 
 state=OFFLINE, ts=1349484372508, server=null}
 {code}
 Looks like NPE happened on this line:
 {code}
   this.gate.set(true);
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6793) Make hbase-examples module


 [ 
https://issues.apache.org/jira/browse/HBASE-6793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-6793:
-

   Resolution: Fixed
Fix Version/s: 0.96.0
   Status: Resolved  (was: Patch Available)

THanks for the nice clean up Sergey.  Committed to trunk.

 Make hbase-examples module
 --

 Key: HBASE-6793
 URL: https://issues.apache.org/jira/browse/HBASE-6793
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.96.0
Reporter: Enis Soztutar
Assignee: Sergey Shelukhin
  Labels: noob
 Fix For: 0.96.0

 Attachments: HBASE-6793.patch, HBASE-6793-v2.patch, 
 HBASE-6793-v3-thrift-0.9.0.patch, HBASE-6793-v4.1-squashed.patch, 
 HBASE-6793-v5-squashed.patch


 There are some examples under /examples/, which are not compiled as a part of 
 the build. We can move them to an hbase-examples module.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6865) Snapshot File Cleaners

2012-11-06 Thread Ted Yu (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-6865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491841#comment-13491841
 ] 

Ted Yu commented on HBASE-6865:
---

@Jon:
Can you clarify point #4 above (w.r.t. where the setter is) ?

 Snapshot File Cleaners
 --

 Key: HBASE-6865
 URL: https://issues.apache.org/jira/browse/HBASE-6865
 Project: HBase
  Issue Type: Sub-task
Reporter: Jesse Yates
Assignee: Jesse Yates
 Fix For: hbase-6055


 HFile and HLog cleaners are necessary for fully implementing snapshots, but 
 can be broken out into their own piece really cleanly. The HFile cleaner is 
 necessary for both timestamp and globally consistent snapshots, but the HLog 
 cleaner is necessary for globally consistent and offline snapshots. 
 Putting everything together in one patch isn't too much overload as there is 
 a lot of overlap between the cleaners.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7097) Log message in SecureServer.class uses wrong class name

2012-11-06 Thread Andrew Purtell (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-7097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-7097:
--

Attachment: HBASE-7097-addendum.patch

Proposed addendum attached. Any more of the DEBUG level messages should go to 
TRACE?

 Log message in SecureServer.class uses wrong class name
 ---

 Key: HBASE-7097
 URL: https://issues.apache.org/jira/browse/HBASE-7097
 Project: HBase
  Issue Type: Improvement
  Components: security
Affects Versions: 0.92.2, 0.94.2
Reporter: Y. SREENIVASULU REDDY
Priority: Minor
 Fix For: 0.92.3, 0.94.3

 Attachments: HBASE-7097_94.patch, HBASE-7097-addendum.patch


 log messages are printing wrongly in 
 org.apache.hadoop.hbase.ipc.SecureServer.class 
 {code}
 public static final Log LOG = 
 LogFactory.getLog(org.apache.hadoop.ipc.SecureServer); 
   private static final Log AUDITLOG = 
 LogFactory.getLog(SecurityLogger.org.apache.hadoop.ipc.SecureServer); 
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6338) Cache Method in RPC handler

2012-11-06 Thread Devaraj Das (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-6338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491851#comment-13491851
 ] 

Devaraj Das commented on HBASE-6338:


On trunk (0.96), the ProtobufRpcEngine already does keep a similar mapping of 
methodname to Method objects.. 

 Cache Method in RPC handler
 ---

 Key: HBASE-6338
 URL: https://issues.apache.org/jira/browse/HBASE-6338
 Project: HBase
  Issue Type: Improvement
Reporter: binlijin
 Attachments: HBASE-6338-90-2.patch, HBASE-6338-90.patch, 
 HBASE-6338-92-2.patch, HBASE-6338-92.patch, HBASE-6338-94-2.patch, 
 HBASE-6338-94.patch, HBASE-6338-trunk-2.patch, HBASE-6338-trunk.patch


 Every call in rpc handler a Method will be created, if we cache the method 
 will improve a little.
 I test with 0.90, Average Class.getMethod(String name, Class... 
 parameterTypes) cost 4780 ns , if we cache it cost 2620 ns.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4583) Integrate RWCC with Append and Increment operations

[
https://issues.apache.org/jira/browse/HBASE-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491854#comment-13491854
]

Lars Hofhansl commented on HBASE-4583:
--

[~saint@gmail.com] That's what the less radical patch does, it removed old
versions of the KVs when they are no longer visible to concurrent scanners.
You cannot clean up the in memory KVs (without a lot of refactoring and
repacking into new slabs) while still using mslab.
There was talk about in memory compactions that could something like this... If
we consider this an issue then that would be the proper route.

Let me try to summarize:
# The less radical patch is correct w.r.t. to MVCC. It does not correct the
Increment behavior when it comes to historical scans. For practical purposes
there is only a single version of the Incremented column, which is changed -
regardless of how the CF is configured.
# The radical removes upsert. Increments are just treated like Puts, all
special code is removed.
Upon flush all excess versions are removed before they are flushed to disk
(HBASE-4241). The flushed files will be small, compaction will be fast. No
attempt is made to clean up KVs before that, so it works with mslab, but the
memstore will fill up more quickly. This patch causes a 10-15% performance
degradation for pure Increments.

As I stated before, in my mind only the radical version is true to HBase's
design and upsert was a hack, which should be removed.
However, I'm fine committing the less radical version, which retains (mostly)
the current performance and fixes the behavior w.r.t. MVCC.

Integrate RWCC with Append and Increment operations
---

Key: HBASE-4583
URL: https://issues.apache.org/jira/browse/HBASE-4583
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Fix For: 0.96.0

Attachments: 4583-trunk-less-radical.txt,
4583-trunk-less-radical-v2.txt, 4583-trunk-less-radical-v3.txt,
4583-trunk-less-radical-v4.txt, 4583-trunk-less-radical-v5.txt,
4583-trunk-less-radical-v6.txt, 4583-trunk-radical.txt,
4583-trunk-radical_v2.txt, 4583-trunk-v3.txt, 4583.txt, 4583-v2.txt,
4583-v3.txt, 4583-v4.txt

Currently Increment and Append operations do not work with RWCC and hence a
client could see the results of multiple such operation mixed in the same
Get/Scan.
The semantics might be a bit more interesting here as upsert adds and removes
to and from the memstore.

[jira] [Commented] (HBASE-6353) Snapshots shell

[
https://issues.apache.org/jira/browse/HBASE-6353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491858#comment-13491858
]

Jonathan Hsieh commented on HBASE-6353:
---

Jesse, where do we stand on this? Do we need more or is this good to go?

Snapshots shell
---

Key: HBASE-6353
URL: https://issues.apache.org/jira/browse/HBASE-6353
Project: HBase
Issue Type: Sub-task
Components: shell
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Attachments: HBASE-6353-v0.patch, HBASE-6353-v1.patch,
snapshot-offline-test-demo.shell

h6. hbase shell with snapshot commands
* snapshot snapshot name table name
** Take a snapshot of the specified name with the specified name
* restore_snapshot snapshot name
** Restore specified snapshot on the original table
* clone_snapshot snapshot name table name
** Create a new table based on the snapshot content
* list_snapshots [filter]
** Show the list of snapshots taken
* delete_snapshot snapshot name
** Remove a specified snapshot
* rename_snapshot snapshot name new snapshot name
** Rename a snapshot to a new name
h6. Restore Table
Given a snapshot name restore override the original table with the snapshot
content.
Before restoring a new snapshot of the table is taken, just to avoid bad
situations.
This allows a full and quick rollback to a previous snapshot.
h6. Clone from Snapshot
Given a snapshot name a new table is created with the content of the
specified snapshot.
This operation allows:
* To have an old version of the table in parallel with the current one.
** Look at snapshot side-by-side with the current before making the
decision whether to roll back or not
* To Add/Remove data from this table without affecting the original one or
the snapshot.
* To Restore only individual items (only some small range of data was lost
from current)
** MR job that scan the cloned table and update the data in the original
one. (Partial restore of the data)
h6. Open points
* Add snapshot type option on take snapshot command (global, timestamp)?

[jira] [Updated] (HBASE-6353) Snapshots shell

[
https://issues.apache.org/jira/browse/HBASE-6353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jonathan Hsieh updated HBASE-6353:
--

Hadoop Flags: Reviewed
Status: Patch Available (was: Open)

Snapshots shell
---

[jira] [Commented] (HBASE-6353) Snapshots shell

[
https://issues.apache.org/jira/browse/HBASE-6353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491862#comment-13491862
]

Hadoop QA commented on HBASE-6353:
--

{color:red}-1 overall{color}. Here are the results of testing the latest
attachment

http://issues.apache.org/jira/secure/attachment/12551976/snapshot-offline-test-demo.shell
against trunk revision .

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.

{color:red}-1 patch{color}. The patch command could not apply the patch.

Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3235//console

This message is automatically generated.

Snapshots shell
---

[jira] [Commented] (HBASE-6865) Snapshot File Cleaners

[
https://issues.apache.org/jira/browse/HBASE-6865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491864#comment-13491864
]

Jonathan Hsieh commented on HBASE-6865:
---

(Oops, there was a point 3 but it got folded into one of the others).

[~te...@apache.org] Take a look at this link and some of the comments around
it and its analog.
https://reviews.apache.org/r/7627/diff/2/?file=183690#file183690line64

Setters conventionally don't io, doesn't start threads, and can be called
anytime in the lifecycle of an object. In the code in this patch, the conf
being is more like part of a constructor or a thread start method.
Specifically, it instantiates a cache object that has a thread. Hm.. I should
have asked why it wasn't in the constructor -- I'll add that to the review.

Snapshot File Cleaners
--

Key: HBASE-6865
URL: https://issues.apache.org/jira/browse/HBASE-6865
Project: HBase
Issue Type: Sub-task
Reporter: Jesse Yates
Assignee: Jesse Yates
Fix For: hbase-6055

HFile and HLog cleaners are necessary for fully implementing snapshots, but
can be broken out into their own piece really cleanly. The HFile cleaner is
necessary for both timestamp and globally consistent snapshots, but the HLog
cleaner is necessary for globally consistent and offline snapshots.
Putting everything together in one patch isn't too much overload as there is
a lot of overlap between the cleaners.

[jira] [Commented] (HBASE-7097) Log message in SecureServer.class uses wrong class name


[ 
https://issues.apache.org/jira/browse/HBASE-7097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491865#comment-13491865
 ] 

Lars Hofhansl commented on HBASE-7097:
--

I think this looks reasonable. 
Should we leave the SecurityEnabled message?

I have no good sense about how frequently
{code}
if (LOG.isDebugEnabled())
  LOG.debug(Created SASL server with mechanism = 
  + authMethod.getMechanismName());
{code}
or 
{code}
 if (LOG.isDebugEnabled())
LOG.debug(Kerberos principal name is  + fullName);
{code}
Would called, but I think it should be OK to leave those at DEBUG.

 Log message in SecureServer.class uses wrong class name
 ---

 Key: HBASE-7097
 URL: https://issues.apache.org/jira/browse/HBASE-7097
 Project: HBase
  Issue Type: Improvement
  Components: security
Affects Versions: 0.92.2, 0.94.2
Reporter: Y. SREENIVASULU REDDY
Priority: Minor
 Fix For: 0.92.3, 0.94.3

 Attachments: HBASE-7097_94.patch, HBASE-7097-addendum.patch


 log messages are printing wrongly in 
 org.apache.hadoop.hbase.ipc.SecureServer.class 
 {code}
 public static final Log LOG = 
 LogFactory.getLog(org.apache.hadoop.ipc.SecureServer); 
   private static final Log AUDITLOG = 
 LogFactory.getLog(SecurityLogger.org.apache.hadoop.ipc.SecureServer); 
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6777) Snapshot Restore interface


 [ 
https://issues.apache.org/jira/browse/HBASE-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-6777:
--

Status: Patch Available  (was: Open)

 Snapshot Restore interface
 --

 Key: HBASE-6777
 URL: https://issues.apache.org/jira/browse/HBASE-6777
 Project: HBase
  Issue Type: Sub-task
  Components: Client, master, snapshots
Affects Versions: 0.96.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: hbase-6055


 Add interfaces for restoring a snapshot

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6802) Export Snapshot


 [ 
https://issues.apache.org/jira/browse/HBASE-6802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-6802:
--

Status: Patch Available  (was: Open)

 Export Snapshot
 ---

 Key: HBASE-6802
 URL: https://issues.apache.org/jira/browse/HBASE-6802
 Project: HBase
  Issue Type: Sub-task
  Components: snapshots
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: hbase-6055, 0.96.0


 Export a snapshot to another cluster.
  - Copy the .snapshot/name folder with all the references
  - Copy the hfiles/hlogs needed by the snapshot
 Once the other cluster has the files and the snapshot information it can 
 restore the snapshot.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6863) Offline snapshots


 [ 
https://issues.apache.org/jira/browse/HBASE-6863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-6863:
--

Status: Patch Available  (was: Open)

 Offline snapshots
 -

 Key: HBASE-6863
 URL: https://issues.apache.org/jira/browse/HBASE-6863
 Project: HBase
  Issue Type: Sub-task
Reporter: Jesse Yates
Assignee: Jesse Yates
 Fix For: hbase-6055


 Create a snapshot of a table while the table is offline. This also should 
 handle a lot of the common utils/scaffolding for taking snapshots 
 (HBASE-6055) with minimal overhead as the code itself is pretty simple.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6865) Snapshot File Cleaners


 [ 
https://issues.apache.org/jira/browse/HBASE-6865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-6865:
--

Status: Patch Available  (was: Open)

 Snapshot File Cleaners
 --

 Key: HBASE-6865
 URL: https://issues.apache.org/jira/browse/HBASE-6865
 Project: HBase
  Issue Type: Sub-task
Reporter: Jesse Yates
Assignee: Jesse Yates
 Fix For: hbase-6055


 HFile and HLog cleaners are necessary for fully implementing snapshots, but 
 can be broken out into their own piece really cleanly. The HFile cleaner is 
 necessary for both timestamp and globally consistent snapshots, but the HLog 
 cleaner is necessary for globally consistent and offline snapshots. 
 Putting everything together in one patch isn't too much overload as there is 
 a lot of overlap between the cleaners.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6353) Snapshots shell

[
https://issues.apache.org/jira/browse/HBASE-6353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491869#comment-13491869
]

Hadoop QA commented on HBASE-6353:
--

{color:red}-1 overall{color}. Here are the results of testing the latest
attachment

http://issues.apache.org/jira/secure/attachment/12551976/snapshot-offline-test-demo.shell
against trunk revision .

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:red}-1 patch{color}. The patch command could not apply the patch.

Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3236//console

This message is automatically generated.

Snapshots shell
---

[jira] [Commented] (HBASE-4583) Integrate RWCC with Append and Increment operations

2012-11-06 Thread Jonathan Gray (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491871#comment-13491871
 ] 

Jonathan Gray commented on HBASE-4583:
--

My vote (if only for one implementation) would be for the less radical patch 
that removes in-memory versions that are not visible rather than doing this 
cleanup on flush which has a number of performance implications.  I can see 
some reasons for wanting to keep versions around (providing support to an 
Omid-like transaction engine requires retaining old versions for at least some 
time), but it would be cool to have an option to prevent the deletion of the 
old versions rather than require that these exist in cases I won't ever use 
them.  In all my increment performance tests, of which there have been many, 
the upsert/removal of old versions is one of the biggest gains, especially if 
you have particularly hot columns.

I'm not sure which design you are referring to when you talk about being true 
to HBase's design ;) Or maybe you're referring to the general principles of 
HBase (append-only), but the increment operation itself was not part of any 
original design or implementation of HBase and has been a hack in one way or 
another from the very first implementation.  For the reason that the 
implementation has been targeted at performance over purity.  I've always seen 
it as an atomic operation that would have any notion of versioning as opaque to 
the user of the atomic increment.  Again, I can see use cases for it, but I'd 
lean towards having it as an option rather than requirement.

Thanks for doing this work, good stuff.  +1

 Integrate RWCC with Append and Increment operations
 ---

 Key: HBASE-4583
 URL: https://issues.apache.org/jira/browse/HBASE-4583
 Project: HBase
  Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.96.0

 Attachments: 4583-trunk-less-radical.txt, 
 4583-trunk-less-radical-v2.txt, 4583-trunk-less-radical-v3.txt, 
 4583-trunk-less-radical-v4.txt, 4583-trunk-less-radical-v5.txt, 
 4583-trunk-less-radical-v6.txt, 4583-trunk-radical.txt, 
 4583-trunk-radical_v2.txt, 4583-trunk-v3.txt, 4583.txt, 4583-v2.txt, 
 4583-v3.txt, 4583-v4.txt


 Currently Increment and Append operations do not work with RWCC and hence a 
 client could see the results of multiple such operation mixed in the same 
 Get/Scan.
 The semantics might be a bit more interesting here as upsert adds and removes 
 to and from the memstore.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7047) [snapshots] Refactor error handling to use javax.management

[
https://issues.apache.org/jira/browse/HBASE-7047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491875#comment-13491875
]

Jonathan Hsieh commented on HBASE-7047:
---

I agree. Let's get the nits taken care of on the offline snapshots, commit,
and then file some follow on jiras for the remaining issues.

[snapshots] Refactor error handling to use javax.management
---

Key: HBASE-7047
URL: https://issues.apache.org/jira/browse/HBASE-7047
Project: HBase
Issue Type: Sub-task
Components: Client, master, regionserver, snapshots, Zookeeper
Affects Versions: hbase-6055
Reporter: Jesse Yates
Assignee: Jesse Yates
Fix For: hbase-6055

Attachments: hbase-7047-v0-adv.patch, hbase-7047-v0.patch

The current error handling framework introduced in HBASE-6571 adds a lot of
complexity for what is essentially a solved problem. Specifically,
cross-thread notifications have been generalized for the JMX tooling in the
javax.management classes.
Similar to what we developed, they have a NotifciationBroadcaster,
NotificationListener, etc. though these are interfaces rather than general
classes. These javax classes can be used almost 1-to-1 as replacements for
things like the ExceptionOrchestrator and ExceptionListener. This also gives
us the opportunity to easily add primitive notifications for standard HBase
things like (1) timeouts, (2) aborts, and (3) server stops since the
framework already considers things like typed notifications.

[jira] [Commented] (HBASE-7055) port HBASE-6371 tier-based compaction from 0.89-fb to trunk

2012-11-06 Thread Sergey Shelukhin (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-7055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491880#comment-13491880
 ] 

Sergey Shelukhin commented on HBASE-7055:
-

already renamed in the recent patch

 port HBASE-6371 tier-based compaction from 0.89-fb to trunk
 ---

 Key: HBASE-7055
 URL: https://issues.apache.org/jira/browse/HBASE-7055
 Project: HBase
  Issue Type: Task
  Components: Compaction
Affects Versions: 0.96.0
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
 Fix For: 0.96.0

 Attachments: HBASE-6371-squashed.patch, HBASE-6371-v2-squashed.patch, 
 HBASE-6371-v3-refactor-only-squashed.patch, 
 HBASE-6371-v4-refactor-only-squashed.patch, 
 HBASE-6371-v5-refactor-only-squashed.patch


 There's divergence in the code :(
 See HBASE-6371 for details.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-7106) [89-fb] Fix the NPE in unit tests for JDK7

Liyin Tang created HBASE-7106:
-

 Summary: [89-fb] Fix the NPE in unit tests for JDK7
 Key: HBASE-7106
 URL: https://issues.apache.org/jira/browse/HBASE-7106
 Project: HBase
  Issue Type: Improvement
Reporter: Liyin Tang
Priority: Trivial


In JDK7, it will throw out NPE if put a NULL into a TreeSet. So the easy fix is 
to skip putting the NULL qualifier into the family map for the GET and SCAN 
objects, and everything else shall be backward compatible.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4583) Integrate RWCC with Append and Increment operations


[ 
https://issues.apache.org/jira/browse/HBASE-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491884#comment-13491884
 ] 

Lars Hofhansl commented on HBASE-4583:
--

As I said I'm fine with the less radical patch :)
Increments are special in they are only useful when updating something existing.

 Integrate RWCC with Append and Increment operations
 ---

 Key: HBASE-4583
 URL: https://issues.apache.org/jira/browse/HBASE-4583
 Project: HBase
  Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.96.0

 Attachments: 4583-trunk-less-radical.txt, 
 4583-trunk-less-radical-v2.txt, 4583-trunk-less-radical-v3.txt, 
 4583-trunk-less-radical-v4.txt, 4583-trunk-less-radical-v5.txt, 
 4583-trunk-less-radical-v6.txt, 4583-trunk-radical.txt, 
 4583-trunk-radical_v2.txt, 4583-trunk-v3.txt, 4583.txt, 4583-v2.txt, 
 4583-v3.txt, 4583-v4.txt


 Currently Increment and Append operations do not work with RWCC and hence a 
 client could see the results of multiple such operation mixed in the same 
 Get/Scan.
 The semantics might be a bit more interesting here as upsert adds and removes 
 to and from the memstore.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-7107) Snapshot References Utils (FileSystem Visitor)

Matteo Bertozzi created HBASE-7107:
--

 Summary: Snapshot References Utils (FileSystem Visitor)
 Key: HBASE-7107
 URL: https://issues.apache.org/jira/browse/HBASE-7107
 Project: HBase
  Issue Type: Sub-task
  Components: snapshots
Affects Versions: 0.96.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: 0.96.0


Utils to traverse the table and snapshot directory.
Used by Restore and Export and should be used by cleaner, and other that want 
to look inside the snapshot folder.
It provides an abstraction to the snapshot metadata format, and allows to get 
information about files, logs and recovered.edits snapshotted.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6958) TestAssignmentManager sometimes fails

2012-11-06 Thread Ted Yu (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-6958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491892#comment-13491892
 ] 

Ted Yu commented on HBASE-6958:
---

+1 on patch for 0.94

Looped TestAssignmentManager 8 times locally with the patch and they passed.

 TestAssignmentManager sometimes fails
 -

 Key: HBASE-6958
 URL: https://issues.apache.org/jira/browse/HBASE-6958
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: Ted Yu
Assignee: Jimmy Xiang
 Fix For: 0.96.0, 0.94.4

 Attachments: 6958_0.94.patch, trunk-6958.patch


 From 
 https://builds.apache.org/job/HBase-TRUNK/3432/testReport/junit/org.apache.hadoop.hbase.master/TestAssignmentManager/testBalanceOnMasterFailoverScenarioWithOpenedNode/
  :
 {code}
 Stacktrace
 java.lang.Exception: test timed out after 5000 milliseconds
   at java.lang.System.arraycopy(Native Method)
   at java.lang.ThreadGroup.remove(ThreadGroup.java:969)
   at java.lang.ThreadGroup.threadTerminated(ThreadGroup.java:942)
   at java.lang.Thread.exit(Thread.java:732)
 ...
 2012-10-06 00:46:12,521 DEBUG [MASTER_CLOSE_REGION-mockedAMExecutor-0] 
 zookeeper.ZKUtil(1141): mockedServer-0x13a33892de7000e Retrieved 81 byte(s) 
 of data from znode /hbase/unassigned/dc01abf9cd7fd0ea256af4df02811640 and set 
 watcher; region=t,,1349484359011.dc01abf9cd7fd0ea256af4df02811640., 
 state=M_ZK_REGION_OFFLINE, servername=master,1,1, createTime=1349484372509, 
 payload.length=0
 2012-10-06 00:46:12,522 ERROR [MASTER_CLOSE_REGION-mockedAMExecutor-0] 
 executor.EventHandler(205): Caught throwable while processing event 
 RS_ZK_REGION_CLOSED
 java.lang.NullPointerException
   at 
 org.apache.hadoop.hbase.master.TestAssignmentManager$MockedLoadBalancer.randomAssignment(TestAssignmentManager.java:773)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.getRegionPlan(AssignmentManager.java:1709)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.getRegionPlan(AssignmentManager.java:1666)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1435)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1155)
   at 
 org.apache.hadoop.hbase.master.TestAssignmentManager$AssignmentManagerWithExtrasForTesting.assign(TestAssignmentManager.java:1035)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1130)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1125)
   at 
 org.apache.hadoop.hbase.master.handler.ClosedRegionHandler.process(ClosedRegionHandler.java:106)
   at 
 org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:202)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
   at java.lang.Thread.run(Thread.java:722)
 2012-10-06 00:46:12,522 DEBUG [pool-1-thread-1-EventThread] 
 master.AssignmentManager(670): Handling transition=M_ZK_REGION_OFFLINE, 
 server=master,1,1, region=dc01abf9cd7fd0ea256af4df02811640, current state 
 from region state map ={t,,1349484359011.dc01abf9cd7fd0ea256af4df02811640. 
 state=OFFLINE, ts=1349484372508, server=null}
 {code}
 Looks like NPE happened on this line:
 {code}
   this.gate.set(true);
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7107) Snapshot References Utils (FileSystem Visitor)


 [ 
https://issues.apache.org/jira/browse/HBASE-7107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-7107:
---

Status: Patch Available  (was: Open)

 Snapshot References Utils (FileSystem Visitor)
 --

 Key: HBASE-7107
 URL: https://issues.apache.org/jira/browse/HBASE-7107
 Project: HBase
  Issue Type: Sub-task
  Components: snapshots
Affects Versions: 0.96.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: 0.96.0


 Utils to traverse the table and snapshot directory.
 Used by Restore and Export and should be used by cleaner, and other that want 
 to look inside the snapshot folder.
 It provides an abstraction to the snapshot metadata format, and allows to 
 get information about files, logs and recovered.edits snapshotted.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7107) Snapshot References Utils (FileSystem Visitor)


[ 
https://issues.apache.org/jira/browse/HBASE-7107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491894#comment-13491894
 ] 

Matteo Bertozzi commented on HBASE-7107:


Review is here: https://reviews.apache.org/r/7788/

 Snapshot References Utils (FileSystem Visitor)
 --

 Key: HBASE-7107
 URL: https://issues.apache.org/jira/browse/HBASE-7107
 Project: HBase
  Issue Type: Sub-task
  Components: snapshots
Affects Versions: 0.96.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: hbase-6055


 Utils to traverse the table and snapshot directory.
 Used by Restore and Export and should be used by cleaner, and other that want 
 to look inside the snapshot folder.
 It provides an abstraction to the snapshot metadata format, and allows to 
 get information about files, logs and recovered.edits snapshotted.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7107) Snapshot References Utils (FileSystem Visitor)


 [ 
https://issues.apache.org/jira/browse/HBASE-7107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-7107:
---

Fix Version/s: (was: 0.96.0)
   hbase-6055

 Snapshot References Utils (FileSystem Visitor)
 --

 Key: HBASE-7107
 URL: https://issues.apache.org/jira/browse/HBASE-7107
 Project: HBase
  Issue Type: Sub-task
  Components: snapshots
Affects Versions: 0.96.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: hbase-6055


 Utils to traverse the table and snapshot directory.
 Used by Restore and Export and should be used by cleaner, and other that want 
 to look inside the snapshot folder.
 It provides an abstraction to the snapshot metadata format, and allows to 
 get information about files, logs and recovered.edits snapshotted.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7106) [89-fb] Fix the NPE in unit tests for JDK7

[
https://issues.apache.org/jira/browse/HBASE-7106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Liyin Tang updated HBASE-7106:
--

Description:
In JDK7, it will throw out NPE if put a NULL into a TreeSet. And in the unit
tests, user can add a NULL as qualifier into the family map for GET or SCAN.
So we shall do the followings:

1) Make sure the semantics of NULL column qualifier is equal to that of the
EMPYT_BYTE_ARRAY column qualifier.

2) An easy fix is to use the EMPYT_BYTE_ARRAY qualifier to replace NULL
qualifier in the family map for the GET or SCAN objects, and everything else
shall be backward compatible.

3) Add a jdk option in the pom.xml (Assuming user install the fb packaged jdk)
eg: mvn test -Dtest=TestFromClientSide -Pjdk7

was:In JDK7, it will throw out NPE if put a NULL into a TreeSet. So the easy
fix is to skip putting the NULL qualifier into the family map for the GET and
SCAN objects, and everything else shall be backward compatible.

[89-fb] Fix the NPE in unit tests for JDK7
--

Key: HBASE-7106
URL: https://issues.apache.org/jira/browse/HBASE-7106
Project: HBase
Issue Type: Improvement
Reporter: Liyin Tang
Priority: Trivial

In JDK7, it will throw out NPE if put a NULL into a TreeSet. And in the unit
tests, user can add a NULL as qualifier into the family map for GET or SCAN.
So we shall do the followings:
1) Make sure the semantics of NULL column qualifier is equal to that of the
EMPYT_BYTE_ARRAY column qualifier.
2) An easy fix is to use the EMPYT_BYTE_ARRAY qualifier to replace NULL
qualifier in the family map for the GET or SCAN objects, and everything else
shall be backward compatible.
3) Add a jdk option in the pom.xml (Assuming user install the fb packaged jdk)
eg: mvn test -Dtest=TestFromClientSide -Pjdk7

[jira] [Updated] (HBASE-7106) [89-fb] Fix the NPE in unit tests for JDK7

[
https://issues.apache.org/jira/browse/HBASE-7106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Liyin Tang updated HBASE-7106:
--

Description:
In JDK7, it will throw out NPE if put a NULL into a TreeSet. And in the unit
tests, user can add a NULL as qualifier into the family map for GET or SCAN.
So we shall do the followings:

1) Make sure the semantics of NULL column qualifier is equal to that of the
EMPYT_BYTE_ARRAY column qualifier.

2) An easy fix is to use the EMPYT_BYTE_ARRAY qualifier to replace NULL
qualifier in the family map for the GET or SCAN objects, and everything else
shall be backward compatible.

3) Add a jdk option in the pom.xml (Assuming user installed the fb packaged jdk)
eg: mvn test -Dtest=TestFromClientSide -Pjdk7

was:
In JDK7, it will throw out NPE if put a NULL into a TreeSet. And in the unit
tests, user can add a NULL as qualifier into the family map for GET or SCAN.
So we shall do the followings:

1) Make sure the semantics of NULL column qualifier is equal to that of the
EMPYT_BYTE_ARRAY column qualifier.

2) An easy fix is to use the EMPYT_BYTE_ARRAY qualifier to replace NULL
qualifier in the family map for the GET or SCAN objects, and everything else
shall be backward compatible.

3) Add a jdk option in the pom.xml (Assuming user install the fb packaged jdk)
eg: mvn test -Dtest=TestFromClientSide -Pjdk7

[89-fb] Fix the NPE in unit tests for JDK7
--

Key: HBASE-7106
URL: https://issues.apache.org/jira/browse/HBASE-7106
Project: HBase
Issue Type: Improvement
Reporter: Liyin Tang
Priority: Trivial

In JDK7, it will throw out NPE if put a NULL into a TreeSet. And in the unit
tests, user can add a NULL as qualifier into the family map for GET or SCAN.
So we shall do the followings:
1) Make sure the semantics of NULL column qualifier is equal to that of the
EMPYT_BYTE_ARRAY column qualifier.
2) An easy fix is to use the EMPYT_BYTE_ARRAY qualifier to replace NULL
qualifier in the family map for the GET or SCAN objects, and everything else
shall be backward compatible.
3) Add a jdk option in the pom.xml (Assuming user installed the fb packaged
jdk)
eg: mvn test -Dtest=TestFromClientSide -Pjdk7

[jira] [Commented] (HBASE-6958) TestAssignmentManager sometimes fails


[ 
https://issues.apache.org/jira/browse/HBASE-6958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491910#comment-13491910
 ] 

Hadoop QA commented on HBASE-6958:
--

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12552340/trunk-6958.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 3 new 
or modified tests.

{color:green}+1 hadoop2.0{color}.  The patch compiles against the hadoop 
2.0 profile.

{color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 
85 warning messages.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:red}-1 findbugs{color}.  The patch appears to introduce 11 new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3234//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3234//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3234//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3234//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3234//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3234//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3234//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3234//console

This message is automatically generated.

 TestAssignmentManager sometimes fails
 -

 Key: HBASE-6958
 URL: https://issues.apache.org/jira/browse/HBASE-6958
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: Ted Yu
Assignee: Jimmy Xiang
 Fix For: 0.96.0, 0.94.4

 Attachments: 6958_0.94.patch, trunk-6958.patch


 From 
 https://builds.apache.org/job/HBase-TRUNK/3432/testReport/junit/org.apache.hadoop.hbase.master/TestAssignmentManager/testBalanceOnMasterFailoverScenarioWithOpenedNode/
  :
 {code}
 Stacktrace
 java.lang.Exception: test timed out after 5000 milliseconds
   at java.lang.System.arraycopy(Native Method)
   at java.lang.ThreadGroup.remove(ThreadGroup.java:969)
   at java.lang.ThreadGroup.threadTerminated(ThreadGroup.java:942)
   at java.lang.Thread.exit(Thread.java:732)
 ...
 2012-10-06 00:46:12,521 DEBUG [MASTER_CLOSE_REGION-mockedAMExecutor-0] 
 zookeeper.ZKUtil(1141): mockedServer-0x13a33892de7000e Retrieved 81 byte(s) 
 of data from znode /hbase/unassigned/dc01abf9cd7fd0ea256af4df02811640 and set 
 watcher; region=t,,1349484359011.dc01abf9cd7fd0ea256af4df02811640., 
 state=M_ZK_REGION_OFFLINE, servername=master,1,1, createTime=1349484372509, 
 payload.length=0
 2012-10-06 00:46:12,522 ERROR [MASTER_CLOSE_REGION-mockedAMExecutor-0] 
 executor.EventHandler(205): Caught throwable while processing event 
 RS_ZK_REGION_CLOSED
 java.lang.NullPointerException
   at 
 org.apache.hadoop.hbase.master.TestAssignmentManager$MockedLoadBalancer.randomAssignment(TestAssignmentManager.java:773)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.getRegionPlan(AssignmentManager.java:1709)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.getRegionPlan(AssignmentManager.java:1666)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1435)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1155)
   at 
 org.apache.hadoop.hbase.master.TestAssignmentManager$AssignmentManagerWithExtrasForTesting.assign(TestAssignmentManager.java:1035)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1130)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1125)
   at 
 org.apache.hadoop.hbase.master.handler.ClosedRegionHandler.process(ClosedRegionHandler.java:106)
   at 
 org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:202)
   at

[jira] [Updated] (HBASE-7009) Port HBaseCluster interface/tests to 0.94

2012-11-06 Thread Sergey Shelukhin (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-7009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sergey Shelukhin updated HBASE-7009:


Attachment: HBASE-7009-v3-squashed.patch

added proper class finder that seems to work with jars, refactor code out of a 
test class into separate classes

 Port HBaseCluster interface/tests to 0.94
 -

 Key: HBASE-7009
 URL: https://issues.apache.org/jira/browse/HBASE-7009
 Project: HBase
  Issue Type: Sub-task
  Components: test
Affects Versions: 0.94.3
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
 Fix For: 0.94.4

 Attachments: HBASE-7009-p1.patch, HBASE-7009.patch, 
 HBASE-7009-v2-squashed.patch, HBASE-7009-v3-squashed.patch


 Need to port. I am porting V5 patch from the original JIRA; I have a 
 partially ported (V3) patch from Enis with protocol buffers being reverted to 
 HRegionInterface/HMasterInterface

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-7108) Don't allow recovered.edits as legal family name

Matteo Bertozzi created HBASE-7108:
--

 Summary: Don't allow recovered.edits as legal family name
 Key: HBASE-7108
 URL: https://issues.apache.org/jira/browse/HBASE-7108
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.94.2, 0.92.2, 0.96.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: 0.96.0


Region directories can contain a folder called recovered.edits, log splitting 
related.
But there's nothing that prevent a user to create a family with that name...

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7108) Don't allow recovered.edits as legal family name


 [ 
https://issues.apache.org/jira/browse/HBASE-7108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-7108:
---

Status: Patch Available  (was: Open)

 Don't allow recovered.edits as legal family name
 --

 Key: HBASE-7108
 URL: https://issues.apache.org/jira/browse/HBASE-7108
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.94.2, 0.92.2, 0.96.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: 0.96.0

 Attachments: HBASE-7108-v0.patch


 Region directories can contain a folder called recovered.edits, log 
 splitting related.
 But there's nothing that prevent a user to create a family with that name...

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7108) Don't allow recovered.edits as legal family name


 [ 
https://issues.apache.org/jira/browse/HBASE-7108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-7108:
---

Attachment: HBASE-7108-v0.patch

added patch to mark recovered.edits as Illegal family name

 Don't allow recovered.edits as legal family name
 --

 Key: HBASE-7108
 URL: https://issues.apache.org/jira/browse/HBASE-7108
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.92.2, 0.94.2, 0.96.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: 0.96.0

 Attachments: HBASE-7108-v0.patch


 Region directories can contain a folder called recovered.edits, log 
 splitting related.
 But there's nothing that prevent a user to create a family with that name...

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7108) Don't allow recovered.edits as legal family name

2012-11-06 Thread Todd Lipcon (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-7108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491924#comment-13491924
 ] 

Todd Lipcon commented on HBASE-7108:


Instead of disallowing it, could we change recovered.edits to be something 
starting with a '.'? I think we already have some requirement that CFs not 
start with '.', right? (if not, is there any other prefix which we've already 
disallowed for users?)

 Don't allow recovered.edits as legal family name
 --

 Key: HBASE-7108
 URL: https://issues.apache.org/jira/browse/HBASE-7108
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.92.2, 0.94.2, 0.96.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: 0.96.0

 Attachments: HBASE-7108-v0.patch


 Region directories can contain a folder called recovered.edits, log 
 splitting related.
 But there's nothing that prevent a user to create a family with that name...

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6793) Make hbase-examples module

2012-11-06 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-6793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491925#comment-13491925
 ] 

Hudson commented on HBASE-6793:
---

Integrated in HBase-TRUNK #3516 (See 
[https://builds.apache.org/job/HBase-TRUNK/3516/])
HBASE-6793 Make hbase-examples module (Revision 1406339)

 Result = FAILURE
stack : 
Files : 
* /hbase/trunk/examples
* /hbase/trunk/hbase-examples
* /hbase/trunk/hbase-examples/README.txt
* /hbase/trunk/hbase-examples/pom.xml
* /hbase/trunk/hbase-examples/src
* /hbase/trunk/hbase-examples/src/main
* /hbase/trunk/hbase-examples/src/main/cpp
* /hbase/trunk/hbase-examples/src/main/cpp/DemoClient.cpp
* /hbase/trunk/hbase-examples/src/main/cpp/Makefile
* /hbase/trunk/hbase-examples/src/main/cpp/gen-cpp
* /hbase/trunk/hbase-examples/src/main/cpp/gen-cpp/Hbase.cpp
* /hbase/trunk/hbase-examples/src/main/cpp/gen-cpp/Hbase.h
* /hbase/trunk/hbase-examples/src/main/cpp/gen-cpp/Hbase_constants.cpp
* /hbase/trunk/hbase-examples/src/main/cpp/gen-cpp/Hbase_constants.h
* /hbase/trunk/hbase-examples/src/main/cpp/gen-cpp/Hbase_server.skeleton.cpp
* /hbase/trunk/hbase-examples/src/main/cpp/gen-cpp/Hbase_types.cpp
* /hbase/trunk/hbase-examples/src/main/cpp/gen-cpp/Hbase_types.h
* /hbase/trunk/hbase-examples/src/main/java
* /hbase/trunk/hbase-examples/src/main/java/org
* /hbase/trunk/hbase-examples/src/main/java/org/apache
* /hbase/trunk/hbase-examples/src/main/java/org/apache/hadoop
* /hbase/trunk/hbase-examples/src/main/java/org/apache/hadoop/hbase
* /hbase/trunk/hbase-examples/src/main/java/org/apache/hadoop/hbase/mapreduce
* 
/hbase/trunk/hbase-examples/src/main/java/org/apache/hadoop/hbase/mapreduce/IndexBuilder.java
* 
/hbase/trunk/hbase-examples/src/main/java/org/apache/hadoop/hbase/mapreduce/SampleUploader.java
* /hbase/trunk/hbase-examples/src/main/java/org/apache/hadoop/hbase/thrift
* 
/hbase/trunk/hbase-examples/src/main/java/org/apache/hadoop/hbase/thrift/DemoClient.java
* 
/hbase/trunk/hbase-examples/src/main/java/org/apache/hadoop/hbase/thrift/generated
* 
/hbase/trunk/hbase-examples/src/main/java/org/apache/hadoop/hbase/thrift/generated/AlreadyExists.java
* 
/hbase/trunk/hbase-examples/src/main/java/org/apache/hadoop/hbase/thrift/generated/BatchMutation.java
* 
/hbase/trunk/hbase-examples/src/main/java/org/apache/hadoop/hbase/thrift/generated/ColumnDescriptor.java
* 
/hbase/trunk/hbase-examples/src/main/java/org/apache/hadoop/hbase/thrift/generated/Hbase.java
* 
/hbase/trunk/hbase-examples/src/main/java/org/apache/hadoop/hbase/thrift/generated/IOError.java
* 
/hbase/trunk/hbase-examples/src/main/java/org/apache/hadoop/hbase/thrift/generated/IllegalArgument.java
* 
/hbase/trunk/hbase-examples/src/main/java/org/apache/hadoop/hbase/thrift/generated/Mutation.java
* 
/hbase/trunk/hbase-examples/src/main/java/org/apache/hadoop/hbase/thrift/generated/TCell.java
* 
/hbase/trunk/hbase-examples/src/main/java/org/apache/hadoop/hbase/thrift/generated/TIncrement.java
* 
/hbase/trunk/hbase-examples/src/main/java/org/apache/hadoop/hbase/thrift/generated/TRegionInfo.java
* 
/hbase/trunk/hbase-examples/src/main/java/org/apache/hadoop/hbase/thrift/generated/TRowResult.java
* 
/hbase/trunk/hbase-examples/src/main/java/org/apache/hadoop/hbase/thrift/generated/TScan.java
* /hbase/trunk/hbase-examples/src/main/perl
* /hbase/trunk/hbase-examples/src/main/perl/DemoClient.pl
* /hbase/trunk/hbase-examples/src/main/perl/gen-perl
* /hbase/trunk/hbase-examples/src/main/perl/gen-perl/Hbase
* /hbase/trunk/hbase-examples/src/main/perl/gen-perl/Hbase/Constants.pm
* /hbase/trunk/hbase-examples/src/main/perl/gen-perl/Hbase/Hbase.pm
* /hbase/trunk/hbase-examples/src/main/perl/gen-perl/Hbase/Types.pm
* /hbase/trunk/hbase-examples/src/main/php
* /hbase/trunk/hbase-examples/src/main/php/DemoClient.php
* /hbase/trunk/hbase-examples/src/main/php/gen-php
* /hbase/trunk/hbase-examples/src/main/php/gen-php/Hbase
* /hbase/trunk/hbase-examples/src/main/php/gen-php/Hbase/Hbase.php
* /hbase/trunk/hbase-examples/src/main/php/gen-php/Hbase/Types.php
* /hbase/trunk/hbase-examples/src/main/python
* /hbase/trunk/hbase-examples/src/main/python/DemoClient.py
* /hbase/trunk/hbase-examples/src/main/python/gen-py
* /hbase/trunk/hbase-examples/src/main/python/gen-py/hbase
* /hbase/trunk/hbase-examples/src/main/python/gen-py/hbase/Hbase-remote
* /hbase/trunk/hbase-examples/src/main/python/gen-py/hbase/Hbase.py
* /hbase/trunk/hbase-examples/src/main/python/gen-py/hbase/__init__.py
* /hbase/trunk/hbase-examples/src/main/python/gen-py/hbase/constants.py
* /hbase/trunk/hbase-examples/src/main/python/gen-py/hbase/ttypes.py
* /hbase/trunk/hbase-examples/src/main/ruby
* /hbase/trunk/hbase-examples/src/main/ruby/DemoClient.rb
* /hbase/trunk/hbase-examples/src/main/ruby/gen-rb
* /hbase/trunk/hbase-examples/src/main/ruby/gen-rb/hbase.rb
* /hbase/trunk/hbase-examples/src/main/ruby/gen-rb/hbase_constants.rb
*

[jira] [Commented] (HBASE-7108) Don't allow recovered.edits as legal family name


[ 
https://issues.apache.org/jira/browse/HBASE-7108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491932#comment-13491932
 ] 

Matteo Bertozzi commented on HBASE-7108:


yeah good catch, we don't allow the '.' as starting character in the family 
name.
So, can I just rename recovered.edits in .recovered.edits?
do we preserve fs layout compatibility in some way?




 Don't allow recovered.edits as legal family name
 --

 Key: HBASE-7108
 URL: https://issues.apache.org/jira/browse/HBASE-7108
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.92.2, 0.94.2, 0.96.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: 0.96.0

 Attachments: HBASE-7108-v0.patch


 Region directories can contain a folder called recovered.edits, log 
 splitting related.
 But there's nothing that prevent a user to create a family with that name...

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6410) Move RegionServer Metrics to metrics2

2012-11-06 Thread Elliott Clark (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-6410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Elliott Clark updated HBASE-6410:
-

   Resolution: Fixed
Fix Version/s: 0.96.0
 Release Note: Region server metrics are now exported through Hadoop's new 
metrics 2 sub system.  Naming and location of MBeans have changed to be more 
consistent and informative.
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

Committed revision 1406396

Thanks all for the reviews.

 Move RegionServer Metrics to metrics2
 -

 Key: HBASE-6410
 URL: https://issues.apache.org/jira/browse/HBASE-6410
 Project: HBase
  Issue Type: Sub-task
  Components: metrics
Affects Versions: 0.96.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Priority: Blocker
 Fix For: 0.96.0

 Attachments: HBASE-6410-13.patch, HBASE-6410-15.patch, 
 HBASE-6410-16.patch, HBASE-6410-18.patch, HBASE-6410-1.patch, 
 HBASE-6410-2.patch, HBASE-6410-3.patch, HBASE-6410-4.patch, 
 HBASE-6410-5.patch, HBASE-6410-6.patch, HBASE-6410.patch


 Move RegionServer Metrics to metrics2

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6958) TestAssignmentManager sometimes fails