[jira] [Commented] (HBASE-5747) Forward port hbase-5708 [89-fb] Make MiniMapRedCluster directory a subdirectory of target/test

2012-04-12 Thread stack (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252228#comment-13252228
 ] 

stack commented on HBASE-5747:
--

Not sure why tests are not completing.  Running on a mac I see problem in this 
test:

{code}
Running org.apache.hadoop.hbase.zookeeper.TestZKLeaderManager
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 4.415 sec  
FAILURE!

Results :

Failed tests:   
testLeaderSelection(org.apache.hadoop.hbase.zookeeper.TestZKLeaderManager): New 
leader should exist
{code}

 Forward port hbase-5708 [89-fb] Make MiniMapRedCluster directory a 
 subdirectory of target/test
 

 Key: HBASE-5747
 URL: https://issues.apache.org/jira/browse/HBASE-5747
 Project: HBase
  Issue Type: Task
Reporter: stack
Assignee: stack
Priority: Blocker
 Attachments: 5474.txt, 5474v2.txt, 5474v3 (1).txt, 5474v3.txt


 Forward port as much as we can of Mikhail's hard-won test cleanups over on 
 0.89 branch  Will improve our being able to run unit tests in //.  He also 
 found a few bugs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5770) Add a clock skew warning threshold

2012-04-12 Thread Hadoop QA (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252235#comment-13252235
 ] 

Hadoop QA commented on HBASE-5770:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12522360/HBASE_5770_v1.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 3 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

+1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
 

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1494//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1494//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1494//console

This message is automatically generated.

 Add a clock skew warning threshold
 --

 Key: HBASE-5770
 URL: https://issues.apache.org/jira/browse/HBASE-5770
 Project: HBase
  Issue Type: Improvement
  Components: master, regionserver
Reporter: Ian Varley
Assignee: Ian Varley
Priority: Minor
 Attachments: HBASE_5770_v1.patch

   Original Estimate: 1h
  Remaining Estimate: 1h

 There's currently an exception thrown by the master when a region server 
 attempts to start up with clock skew greater than some configured amount 
 (defaulting to 30 seconds). However, it'd be nice to get some warnings logged 
 at a value that isn't severe enough to warrant killing the RS, but still 
 represents significant skew that could affect correctness. Will attach a 
 simple patch to add this as a setting.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5737) Minor Improvements related to balancer.

2012-04-12 Thread ramkrishna.s.vasudevan (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252280#comment-13252280
 ] 

ramkrishna.s.vasudevan commented on HBASE-5737:
---

@Stack
I prefer passing as constructor but that needs changes in many places.
Actually i had 2 ways of doing this.  But the TestAssignmentManager was failing 
if we remove the balancer in the AssignmentManager.

Though the changes are many passing in constructor should be fine.

 Minor Improvements related to balancer.
 ---

 Key: HBASE-5737
 URL: https://issues.apache.org/jira/browse/HBASE-5737
 Project: HBase
  Issue Type: Improvement
  Components: master
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Minor
 Attachments: HBASE-5737.patch, HBASE-5737_1.patch, HBASE-5737_2.patch


 Currently in Am.getAssignmentByTable()  we use a result map which is currenly 
 a hashmap.  It could be better if we have a treeMap.  Even in 
 MetaReader.fullScan we have the treeMap only so that we have the naming order 
 maintained. I felt this change could be very useful in cases where we are 
 extending the DefaultLoadBalancer.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Created] (HBASE-5771) unable to start regionservers on slave machines in hbase

2012-04-12 Thread amsal (Created) (JIRA)
unable to start regionservers on slave machines in hbase


 Key: HBASE-5771
 URL: https://issues.apache.org/jira/browse/HBASE-5771
 Project: HBase
  Issue Type: Task
Reporter: amsal
Priority: Critical


have a problem starting regionservers on slave pc,s. when i enlist only master 
pc in conf/regionservers every thing works fine but when i add two more slaves 
to it the hbase does not start . if i delete all hbase folders in the tmp 
folder from all pc,s and then start regionserver (with 3 regionservers 
enlisted)the hbase gets started but when i try to create a table it again 
fails(gets stuck) pls anyone help i am using hadoop 0.20.0 which is working 
fine and hbase 0.92.0 i have 3 pc's in cluster one master and two slaves all 
running windows xp

also tell that is DNS (both forward and backward lookup working)necessary for 
hbase in my case is there any way to replicate hbase table to all region 
servers i.e. i want to have a copy of table at each pc and want to access them 
locally(when i execute map task they should use their local copy of hbase 
table) plz help..!! thanx in advance

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Created] (HBASE-5772) Unable to open the few links in http://hbase.apache.org/

2012-04-12 Thread Kiran BC (Created) (JIRA)
Unable to open the few links in http://hbase.apache.org/


 Key: HBASE-5772
 URL: https://issues.apache.org/jira/browse/HBASE-5772
 Project: HBase
  Issue Type: Bug
  Components: documentation
Affects Versions: 0.94.0
Reporter: Kiran BC


Few links in http://hbase.apache.org/ is not working. 
For example, Ref Guide (multi-page) will actually link to 
http://hbase.apache.org/book/book.html and if I try to open this, Page not 
found error is coming.
If I add /book in the url, like http://hbase.apache.org/book/book/book.html, it 
is taking me to the Apache HBase Reference Guide 



I think the folder structure has been changed.



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Created] (HBASE-5773) HtablePool constructor not reading config files in certain cases

2012-04-12 Thread Ioan Eugen Stan (Created) (JIRA)
HtablePool constructor not reading config files in certain cases


 Key: HBASE-5773
 URL: https://issues.apache.org/jira/browse/HBASE-5773
 Project: HBase
  Issue Type: Bug
  Components: client
Affects Versions: 0.92.1, 0.90.6, 0.94.1
Reporter: Ioan Eugen Stan
Priority: Minor


Creating a HtablePool can issue two behaviour depanding on the constructor 
called. 

Case 1: loads the configs from hbase-site
  public HTablePool() {
this(HBaseConfiguration.create(), Integer.MAX_VALUE);
  }

Calling this with null values for Configuration: 
public HTablePool(final Configuration config, final int maxSize) {
this(config, maxSize, null, null);
  }

will issue:

 public HTablePool(final Configuration config, final int maxSize,
  final HTableInterfaceFactory tableFactory, PoolType poolType) {
// Make a new configuration instance so I can safely cleanup when
// done with the pool.
this.config = config == null ? new Configuration() : config;

which does not read the hbase-site config files as HBaseConfiguration.create() 
does. 

I've tracked this problem to all versions of hbase. 


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5773) HtablePool constructor not reading config files in certain cases

2012-04-12 Thread Ioan Eugen Stan (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ioan Eugen Stan updated HBASE-5773:
---

Fix Version/s: 0.94.1
   0.92.2
   0.90.7
   Status: Patch Available  (was: Open)

 HtablePool constructor not reading config files in certain cases
 

 Key: HBASE-5773
 URL: https://issues.apache.org/jira/browse/HBASE-5773
 Project: HBase
  Issue Type: Bug
  Components: client
Affects Versions: 0.92.1, 0.90.6, 0.94.1
Reporter: Ioan Eugen Stan
Priority: Minor
 Fix For: 0.90.7, 0.92.2, 0.94.1

 Attachments: different-config-behaviour.patch


 Creating a HtablePool can issue two behaviour depanding on the constructor 
 called. 
 Case 1: loads the configs from hbase-site
   public HTablePool() {
 this(HBaseConfiguration.create(), Integer.MAX_VALUE);
   }
 Calling this with null values for Configuration: 
 public HTablePool(final Configuration config, final int maxSize) {
 this(config, maxSize, null, null);
   }
 will issue:
  public HTablePool(final Configuration config, final int maxSize,
   final HTableInterfaceFactory tableFactory, PoolType poolType) {
 // Make a new configuration instance so I can safely cleanup when
 // done with the pool.
 this.config = config == null ? new Configuration() : config;
 which does not read the hbase-site config files as 
 HBaseConfiguration.create() does. 
 I've tracked this problem to all versions of hbase. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5773) HtablePool constructor not reading config files in certain cases

2012-04-12 Thread Ioan Eugen Stan (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ioan Eugen Stan updated HBASE-5773:
---

Attachment: different-config-behaviour.patch

 HtablePool constructor not reading config files in certain cases
 

 Key: HBASE-5773
 URL: https://issues.apache.org/jira/browse/HBASE-5773
 Project: HBase
  Issue Type: Bug
  Components: client
Affects Versions: 0.90.6, 0.92.1, 0.94.1
Reporter: Ioan Eugen Stan
Priority: Minor
 Fix For: 0.90.7, 0.92.2, 0.94.1

 Attachments: different-config-behaviour.patch


 Creating a HtablePool can issue two behaviour depanding on the constructor 
 called. 
 Case 1: loads the configs from hbase-site
   public HTablePool() {
 this(HBaseConfiguration.create(), Integer.MAX_VALUE);
   }
 Calling this with null values for Configuration: 
 public HTablePool(final Configuration config, final int maxSize) {
 this(config, maxSize, null, null);
   }
 will issue:
  public HTablePool(final Configuration config, final int maxSize,
   final HTableInterfaceFactory tableFactory, PoolType poolType) {
 // Make a new configuration instance so I can safely cleanup when
 // done with the pool.
 this.config = config == null ? new Configuration() : config;
 which does not read the hbase-site config files as 
 HBaseConfiguration.create() does. 
 I've tracked this problem to all versions of hbase. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5677) The master never does balance because duplicate openhandled the one region

2012-04-12 Thread xufeng (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252358#comment-13252358
 ] 

xufeng commented on HBASE-5677:
---

Test by trunk version is ok.
master do nothing if it has not initialized.

 The master never does balance because duplicate openhandled the one region
 --

 Key: HBASE-5677
 URL: https://issues.apache.org/jira/browse/HBASE-5677
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.6
 Environment: 0.90
Reporter: xufeng
Assignee: xufeng
 Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0

 Attachments: 5677-proposal.txt, HBASE-5677-90-v1.patch, 
 surefire-report_no_patched_v1.html, surefire-report_patched_v1.html


 If region be assigned When the master is doing initialization(before do 
 processFailover),the region will be duplicate openhandled.
 because the unassigned node in zookeeper will be handled again in 
 AssignmentManager#processFailover()
 it cause the region in RIT,thus the master never does balance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5677) The master never does balance because duplicate openhandled the one region

2012-04-12 Thread xufeng (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252356#comment-13252356
 ] 

xufeng commented on HBASE-5677:
---

@Ted @stack @Lars
I test it use trunk version.
then I got this in shell and my test case:
${noformat}
12/04/12 19:38:35 INFO client.HBaseAdmin: Started enable of Table02
org.apache.hadoop.hbase.PleaseHoldException: 
org.apache.hadoop.hbase.PleaseHoldException: Master is initializing
${noformat}

PleaseHoldException be added in HBASE-5454,the patch of this issue be 
integrated to trunk and 0.94 version.

 The master never does balance because duplicate openhandled the one region
 --

 Key: HBASE-5677
 URL: https://issues.apache.org/jira/browse/HBASE-5677
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.6
 Environment: 0.90
Reporter: xufeng
Assignee: xufeng
 Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0

 Attachments: 5677-proposal.txt, HBASE-5677-90-v1.patch, 
 surefire-report_no_patched_v1.html, surefire-report_patched_v1.html


 If region be assigned When the master is doing initialization(before do 
 processFailover),the region will be duplicate openhandled.
 because the unassigned node in zookeeper will be handled again in 
 AssignmentManager#processFailover()
 it cause the region in RIT,thus the master never does balance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5773) HtablePool constructor not reading config files in certain cases

2012-04-12 Thread Hadoop QA (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252386#comment-13252386
 ] 

Hadoop QA commented on HBASE-5773:
--

-1 overall.  Here are the results of testing the latest attachment 
  
http://issues.apache.org/jira/secure/attachment/12522403/different-config-behaviour.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

-1 tests included.  The patch doesn't appear to include any new or modified 
tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

+1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

+1 core tests.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1495//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1495//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1495//console

This message is automatically generated.

 HtablePool constructor not reading config files in certain cases
 

 Key: HBASE-5773
 URL: https://issues.apache.org/jira/browse/HBASE-5773
 Project: HBase
  Issue Type: Bug
  Components: client
Affects Versions: 0.90.6, 0.92.1, 0.94.1
Reporter: Ioan Eugen Stan
Priority: Minor
 Fix For: 0.90.7, 0.92.2, 0.94.1

 Attachments: different-config-behaviour.patch


 Creating a HtablePool can issue two behaviour depanding on the constructor 
 called. 
 Case 1: loads the configs from hbase-site
   public HTablePool() {
 this(HBaseConfiguration.create(), Integer.MAX_VALUE);
   }
 Calling this with null values for Configuration: 
 public HTablePool(final Configuration config, final int maxSize) {
 this(config, maxSize, null, null);
   }
 will issue:
  public HTablePool(final Configuration config, final int maxSize,
   final HTableInterfaceFactory tableFactory, PoolType poolType) {
 // Make a new configuration instance so I can safely cleanup when
 // done with the pool.
 this.config = config == null ? new Configuration() : config;
 which does not read the hbase-site config files as 
 HBaseConfiguration.create() does. 
 I've tracked this problem to all versions of hbase. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5754) data lost with gora continuous ingest test (goraci)

2012-04-12 Thread Eric Newton (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252454#comment-13252454
 ] 

Eric Newton commented on HBASE-5754:


Yes, that looks like a clean run.

My run of 1B completed correctly yesterday.

However, I just did a run of 100M.  I suspect split/balancing is causing the 
data loss.  So, during the 100M run, I pushed the split button on the master 
web page, every 5 seconds until I had several hundred tablets.  Then I left it 
alone.

{noformat}
12/04/12 14:17:01 INFO mapred.JobClient:   goraci.Verify$Counts
12/04/12 14:17:01 INFO mapred.JobClient: UNDEFINED=37099
12/04/12 14:17:01 INFO mapred.JobClient: REFERENCED=89961385
12/04/12 14:17:01 INFO mapred.JobClient: UNREFERENCED=10001516
{noformat}

Perhaps if you do this you can replicate the problem.

 data lost with gora continuous ingest test (goraci)
 ---

 Key: HBASE-5754
 URL: https://issues.apache.org/jira/browse/HBASE-5754
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.1
 Environment: 10 node test cluster
Reporter: Eric Newton
Assignee: stack

 Keith Turner re-wrote the accumulo continuous ingest test using gora, which 
 has both hbase and accumulo back-ends.
 I put a billion entries into HBase, and ran the Verify map/reduce job.  The 
 verification failed because about 21K entries were missing.  The goraci 
 [README|https://github.com/keith-turner/goraci] explains the test, and how it 
 detects missing data.
 I re-ran the test with 100 million entries, and it verified successfully.  
 Both of the times I tested using a billion entries, the verification failed.
 If I run the verification step twice, the results are consistent, so the 
 problem is
 probably not on the verify step.
 Here's the versions of the various packages:
 ||package||version||
 |hadoop|0.20.205.0|
 |hbase|0.92.1|
 |gora|http://svn.apache.org/repos/asf/gora/trunk r1311277|
 |goraci|https://github.com/ericnewton/goraci  tagged 2012-04-08|
 The change I made to goraci was to configure it for hbase and to allow it to 
 build properly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Resolved] (HBASE-5771) unable to start regionservers on slave machines in hbase

2012-04-12 Thread stack (Resolved) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-5771.
--

Resolution: Invalid

Closing as invalid.  This is an incomplete question better suited for mailing 
list

 unable to start regionservers on slave machines in hbase
 

 Key: HBASE-5771
 URL: https://issues.apache.org/jira/browse/HBASE-5771
 Project: HBase
  Issue Type: Task
Reporter: amsal
Priority: Critical

 have a problem starting regionservers on slave pc,s. when i enlist only 
 master pc in conf/regionservers every thing works fine but when i add two 
 more slaves to it the hbase does not start . if i delete all hbase 
 folders in the tmp folder from all pc,s and then start regionserver (with 3 
 regionservers enlisted)the hbase gets started but when i try to create a 
 table it again fails(gets stuck) pls anyone help i am using hadoop 0.20.0 
 which is working fine and hbase 0.92.0 i have 3 pc's in cluster one master 
 and two slaves all running windows xp
 also tell that is DNS (both forward and backward lookup working)necessary for 
 hbase in my case is there any way to replicate hbase table to all region 
 servers i.e. i want to have a copy of table at each pc and want to access 
 them locally(when i execute map task they should use their local copy of 
 hbase table) plz help..!! thanx in advance

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5754) data lost with gora continuous ingest test (goraci)

2012-04-12 Thread Keith Turner (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252491#comment-13252491
 ] 

Keith Turner commented on HBASE-5754:
-

The counts for the 1B run seem odd to me , but maybe thats just an artifact of 
how many map task you ran for the generator and how much data each task 
generated.  If a map task does not does not generate a multiple of 25,000,000 
then it will leave some unreferenced.  It generates a circular linked list 
every 25M.   

{noformat}
12/04/12 03:54:11 INFO mapred.JobClient: REFERENCED=564459547
12/04/12 03:54:11 INFO mapred.JobClient: UNREFERENCED=104000
{noformat}

If you were to run 10 map task each generating 100M, then this should generate 
1B with all nodes referenced.  Minimizing the number of unreferenced is ideal, 
because the test can not detect the loss of unreferenced nodes.  I should 
probably add this info to the readme.


 data lost with gora continuous ingest test (goraci)
 ---

 Key: HBASE-5754
 URL: https://issues.apache.org/jira/browse/HBASE-5754
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.1
 Environment: 10 node test cluster
Reporter: Eric Newton
Assignee: stack

 Keith Turner re-wrote the accumulo continuous ingest test using gora, which 
 has both hbase and accumulo back-ends.
 I put a billion entries into HBase, and ran the Verify map/reduce job.  The 
 verification failed because about 21K entries were missing.  The goraci 
 [README|https://github.com/keith-turner/goraci] explains the test, and how it 
 detects missing data.
 I re-ran the test with 100 million entries, and it verified successfully.  
 Both of the times I tested using a billion entries, the verification failed.
 If I run the verification step twice, the results are consistent, so the 
 problem is
 probably not on the verify step.
 Here's the versions of the various packages:
 ||package||version||
 |hadoop|0.20.205.0|
 |hbase|0.92.1|
 |gora|http://svn.apache.org/repos/asf/gora/trunk r1311277|
 |goraci|https://github.com/ericnewton/goraci  tagged 2012-04-08|
 The change I made to goraci was to configure it for hbase and to allow it to 
 build properly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-3505) hbck should be able to fix case where region is missing from META but on FS

2012-04-12 Thread Anoop Sam John (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-3505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252492#comment-13252492
 ] 

Anoop Sam John commented on HBASE-3505:
---

Uber HBCK is handling this case now right? This issue can be closed?

 hbck should be able to fix case where region is missing from META but on FS
 ---

 Key: HBASE-3505
 URL: https://issues.apache.org/jira/browse/HBASE-3505
 Project: HBase
  Issue Type: Improvement
  Components: hbck
Reporter: Todd Lipcon
Assignee: Jonathan Hsieh
 Attachments: hbase-3505.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5754) data lost with gora continuous ingest test (goraci)

2012-04-12 Thread Keith Turner (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252502#comment-13252502
 ] 

Keith Turner commented on HBASE-5754:
-

When Eric ran 10 generators each adding 100M against HBase and no data was 
lost, he saw 1B referenced, 0 unref, and 0 undef.

 data lost with gora continuous ingest test (goraci)
 ---

 Key: HBASE-5754
 URL: https://issues.apache.org/jira/browse/HBASE-5754
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.1
 Environment: 10 node test cluster
Reporter: Eric Newton
Assignee: stack

 Keith Turner re-wrote the accumulo continuous ingest test using gora, which 
 has both hbase and accumulo back-ends.
 I put a billion entries into HBase, and ran the Verify map/reduce job.  The 
 verification failed because about 21K entries were missing.  The goraci 
 [README|https://github.com/keith-turner/goraci] explains the test, and how it 
 detects missing data.
 I re-ran the test with 100 million entries, and it verified successfully.  
 Both of the times I tested using a billion entries, the verification failed.
 If I run the verification step twice, the results are consistent, so the 
 problem is
 probably not on the verify step.
 Here's the versions of the various packages:
 ||package||version||
 |hadoop|0.20.205.0|
 |hbase|0.92.1|
 |gora|http://svn.apache.org/repos/asf/gora/trunk r1311277|
 |goraci|https://github.com/ericnewton/goraci  tagged 2012-04-08|
 The change I made to goraci was to configure it for hbase and to allow it to 
 build properly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5754) data lost with gora continuous ingest test (goraci)

2012-04-12 Thread stack (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252504#comment-13252504
 ] 

stack commented on HBASE-5754:
--

Let me do the same.  I did not match generator map tasks to verify reducers.  
Then let me recreate the split issue Eric describes above.  Thanks lads.

 data lost with gora continuous ingest test (goraci)
 ---

 Key: HBASE-5754
 URL: https://issues.apache.org/jira/browse/HBASE-5754
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.1
 Environment: 10 node test cluster
Reporter: Eric Newton
Assignee: stack

 Keith Turner re-wrote the accumulo continuous ingest test using gora, which 
 has both hbase and accumulo back-ends.
 I put a billion entries into HBase, and ran the Verify map/reduce job.  The 
 verification failed because about 21K entries were missing.  The goraci 
 [README|https://github.com/keith-turner/goraci] explains the test, and how it 
 detects missing data.
 I re-ran the test with 100 million entries, and it verified successfully.  
 Both of the times I tested using a billion entries, the verification failed.
 If I run the verification step twice, the results are consistent, so the 
 problem is
 probably not on the verify step.
 Here's the versions of the various packages:
 ||package||version||
 |hadoop|0.20.205.0|
 |hbase|0.92.1|
 |gora|http://svn.apache.org/repos/asf/gora/trunk r1311277|
 |goraci|https://github.com/ericnewton/goraci  tagged 2012-04-08|
 The change I made to goraci was to configure it for hbase and to allow it to 
 build properly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5767) Add the hbase shell table_att for any attribute

2012-04-12 Thread ShiXing (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252507#comment-13252507
 ] 

ShiXing commented on HBASE-5767:


@Zhihong : I found that the  HBASE-5335 has solved this issue by 
HConstants.CONFIG.

And can I add this to 0.92 Version? The HBASE-5335 is just on 0.96 trunk.

 Add the hbase shell table_att for any attribute
 ---

 Key: HBASE-5767
 URL: https://issues.apache.org/jira/browse/HBASE-5767
 Project: HBase
  Issue Type: Improvement
  Components: shell
Reporter: Xing Shi
Priority: Minor
 Attachments: HBASE-5767-V2.patch, HBASE-5767.patch


 Now the HTableDescriptor supports setValue(String key, String value) method, 
 but the hbase shell not support it.
 May be like this:
 {quota}
 hbase(main):003:0 alter 'test', METHOD='table_att', 'key1'='value1'
 Updating all regions with the new schema...
 1/1 regions updated.
 Done.
 0 row(s) in 1.0820 seconds
 hbase(main):005:0 describe 'test'
 DESCRIPTION   
 ENABLED  
  {NAME = 'test', key1 = 'value1', FAMILIES = [{NAME = 'f1', BLOOMFILTER 
 = 'NONE', RE true 
  PLICATION_SCOPE = '0', VERSIONS = '3', COMPRESSION = 'NONE', MIN_VERSIONS 
 = '0', TTL  
   = '2147483647', BLOCKSIZE = '65536', IN_MEMORY = 'false', BLOCKCACHE = 
 'true'}]} 
 1 row(s) in 0.0300 seconds
 hbase(main):007:0 alter 'test', METHOD='table_att_unset', NAME='key1'
 Updating all regions with the new schema...
 1/1 regions updated.
 Done.
 0 row(s) in 1.0860 seconds
 hbase(main):008:0 describe 'test'
 DESCRIPTION   
 ENABLED  
  {NAME = 'test', FAMILIES = [{NAME = 'f1', BLOOMFILTER = 'NONE', 
 REPLICATION_SCOPE = false
   '0', VERSIONS = '3', COMPRESSION = 'NONE', MIN_VERSIONS = '0', TTL = 
 '2147483647',   
  BLOCKSIZE = '65536', IN_MEMORY = 'false', BLOCKCACHE = 'true'}]}  
  
 1 row(s) in 0.0280 seconds
 {quota}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Resolved] (HBASE-5736) ThriftServerRunner.HbaseHandler.mutateRow() does not use ByteBuffer correctly

2012-04-12 Thread Lars Hofhansl (Resolved) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl resolved HBASE-5736.
--

Resolution: Fixed

Committed, marking as fixed.

 ThriftServerRunner.HbaseHandler.mutateRow() does not use ByteBuffer correctly
 -

 Key: HBASE-5736
 URL: https://issues.apache.org/jira/browse/HBASE-5736
 Project: HBase
  Issue Type: Bug
Reporter: Scott Chen
Assignee: Scott Chen
 Fix For: 0.94.0, 0.96.0

 Attachments: 5736-94.txt, HBASE-5736.D2649.1.patch, 
 HBASE-5736.D2649.2.patch, HBASE-5736.D2649.3.patch


 We have fixed similar bug in
 https://issues.apache.org/jira/browse/HBASE-5507
 It uses ByteBuffer.array() to read the ByteBuffer.
 This will ignore the offset return the whole underlying byte array.
 The bug can be triggered by using framed Transport thrift servers.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5717) Scanner metrics are only reported if you get to the end of a scanner

2012-04-12 Thread Lars Hofhansl (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-5717:
-

Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed to 0.94 and 0.96.
Thanks for the patch Ian.
Thanks for the review Ted.

 Scanner metrics are only reported if you get to the end of a scanner
 

 Key: HBASE-5717
 URL: https://issues.apache.org/jira/browse/HBASE-5717
 Project: HBase
  Issue Type: Bug
  Components: client, metrics
Reporter: Ian Varley
Assignee: Ian Varley
Priority: Minor
 Fix For: 0.94.0, 0.96.0

 Attachments: 5717-v4.patch, ClientScanner_HBASE_5717-v2.patch, 
 ClientScanner_HBASE_5717-v3.patch, ClientScanner_HBASE_5717.patch

   Original Estimate: 4h
  Remaining Estimate: 4h

 When you turn on Scanner Metrics, the metrics are currently only made 
 available if you run over all records available in the scanner. If you stop 
 iterating before the end, the values are never flushed into the metrics 
 object (in the Scan attribute).
 Will supply a patch with fix and test.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5741) ImportTsv does not check for table existence

2012-04-12 Thread Lars Hofhansl (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-5741:
-

Fix Version/s: (was: 0.94.0)
   0.94.1

Moving to 0.94.1, since we're still discussing (again, sorry about the late 
chime in)

 ImportTsv does not check for table existence 
 -

 Key: HBASE-5741
 URL: https://issues.apache.org/jira/browse/HBASE-5741
 Project: HBase
  Issue Type: Bug
  Components: mapreduce
Affects Versions: 0.90.4
Reporter: Clint Heath
Assignee: Himanshu Vashishtha
 Fix For: 0.96.0, 0.94.1

 Attachments: 5741-94.txt, 5741-v3.txt, HBase-5741-v2.patch, 
 HBase-5741.patch


 The usage statement for the importtsv command to hbase claims this:
 Note: if you do not use this option, then the target table must already 
 exist in HBase (in reference to the importtsv.bulk.output command-line 
 option)
 The truth is, the table must exist no matter what, importtsv cannot and will 
 not create it for you.
 This is the case because the createSubmittableJob method of ImportTsv does 
 not even attempt to check if the table exists already, much less create it:
 (From org.apache.hadoop.hbase.mapreduce.ImportTsv.java)
 305 HTable table = new HTable(conf, tableName);
 The HTable method signature in use there assumes the table exists and runs a 
 meta scan on it:
 (From org.apache.hadoop.hbase.client.HTable.java)
 142 * Creates an object to access a HBase table.
 ...
 151 public HTable(Configuration conf, final String tableName)
 What we should do inside of createSubmittableJob is something similar to what 
 the completebulkloads command would do:
 (Taken from org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.java)
 690 boolean tableExists = this.doesTableExist(tableName);
 691 if (!tableExists) this.createTable(tableName,dirPath);
 Currently the docs are misleading, the table in fact must exist prior to 
 running importtsv. We should check if it exists rather than assume it's 
 already there and throw the below exception:
 12/03/14 17:15:42 WARN client.HConnectionManager$HConnectionImplementation: 
 Encountered problems when prefetch META table: 
 org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for 
 table: myTable2, row=myTable2,,99
   at 
 org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:150)
 ...

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5604) M/R tool to replay WAL files

2012-04-12 Thread Lars Hofhansl (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252530#comment-13252530
 ] 

Lars Hofhansl commented on HBASE-5604:
--

Since this is only new code I propose this for 0.94.0 as well.

 M/R tool to replay WAL files
 

 Key: HBASE-5604
 URL: https://issues.apache.org/jira/browse/HBASE-5604
 Project: HBase
  Issue Type: New Feature
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Attachments: 5604-v4.txt, 5604-v6.txt, 5604-v7.txt, 5604-v8.txt, 
 5604-v9.txt, HLog-5604-v3.txt


 Just an idea I had. Might be useful for restore of a backup using the HLogs.
 This could an M/R (with a mapper per HLog file).
 The tool would get a timerange and a (set of) table(s). We'd pick the right 
 HLogs based on time before the M/R job is started and then have a mapper per 
 HLog file.
 The mapper would then go through the HLog, filter all WALEdits that didn't 
 fit into the time range or are not any of the tables and then uses 
 HFileOutputFormat to generate HFiles.
 Would need to indicate the splits we want, probably from a live table.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Created] (HBASE-5774) Add documentation for WALPlayer to HBase reference guide.

2012-04-12 Thread Lars Hofhansl (Created) (JIRA)
Add documentation for WALPlayer to HBase reference guide.
-

 Key: HBASE-5774
 URL: https://issues.apache.org/jira/browse/HBASE-5774
 Project: HBase
  Issue Type: Sub-task
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5677) The master never does balance because duplicate openhandled the one region

2012-04-12 Thread Lars Hofhansl (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252533#comment-13252533
 ] 

Lars Hofhansl commented on HBASE-5677:
--

@Stack: This seems like a good addition to 0.94.
Let's say if we can track down the test failures today we'll put this in 0.94.0 
otherwise 0.94.1

 The master never does balance because duplicate openhandled the one region
 --

 Key: HBASE-5677
 URL: https://issues.apache.org/jira/browse/HBASE-5677
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.6
 Environment: 0.90
Reporter: xufeng
Assignee: xufeng
 Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0

 Attachments: 5677-proposal.txt, HBASE-5677-90-v1.patch, 
 surefire-report_no_patched_v1.html, surefire-report_patched_v1.html


 If region be assigned When the master is doing initialization(before do 
 processFailover),the region will be duplicate openhandled.
 because the unassigned node in zookeeper will be handled again in 
 AssignmentManager#processFailover()
 it cause the region in RIT,thus the master never does balance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5604) M/R tool to replay WAL files

2012-04-12 Thread Ted Yu (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252537#comment-13252537
 ] 

Ted Yu commented on HBASE-5604:
---

If you run the patch through 'arc lint', you would see (just excerpt):
{code}
   Warning  (TXT3) Line Too Long
This line is 106 characters long, but the convention is 100 characters.

 274 System.err.println(  -D + BULK_OUTPUT_CONF_KEY + 
=/path/for/output);
 275 System.err.println(  (Exactly one table must be specified 
for this option, and not mapping can be specified!));
 276 System.err.println(Other options:);
  277 System.err.println(  -D + HLogInputFormat.START_TIME_KEY 
+ =ms (only apply edit after this time));
 278 System.err.println(  -D + HLogInputFormat.END_TIME_KEY + 
=ms (only apply edit before this time));
 279 System.err.println(For performance also consider the 
following options:\n
 280 +   -Dmapred.map.tasks.speculative.execution=false\n

   Warning  (TXT3) Line Too Long
This line is 105 characters long, but the convention is 100 characters.

 275 System.err.println(  (Exactly one table must be specified 
for this option, and not mapping can be specified!));
 276 System.err.println(Other options:);
 277 System.err.println(  -D + HLogInputFormat.START_TIME_KEY 
+ =ms (only apply edit after this time));
  278 System.err.println(  -D + HLogInputFormat.END_TIME_KEY + 
=ms (only apply edit before this time));
 279 System.err.println(For performance also consider the 
following options:\n
 280 +   -Dmapred.map.tasks.speculative.execution=false\n
 281 +   
-Dmapred.reduce.tasks.speculative.execution=false);
{code}
Below is a long line:
{code}
+throw new IOException(option +  must be specified either in the form 
2001-02-20T16:35:06.99 or as a number of milliseconds);
{code}
Minor comment: the 'a' in 'as a number' should be omitted.

 M/R tool to replay WAL files
 

 Key: HBASE-5604
 URL: https://issues.apache.org/jira/browse/HBASE-5604
 Project: HBase
  Issue Type: New Feature
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Attachments: 5604-v4.txt, 5604-v6.txt, 5604-v7.txt, 5604-v8.txt, 
 5604-v9.txt, HLog-5604-v3.txt


 Just an idea I had. Might be useful for restore of a backup using the HLogs.
 This could an M/R (with a mapper per HLog file).
 The tool would get a timerange and a (set of) table(s). We'd pick the right 
 HLogs based on time before the M/R job is started and then have a mapper per 
 HLog file.
 The mapper would then go through the HLog, filter all WALEdits that didn't 
 fit into the time range or are not any of the tables and then uses 
 HFileOutputFormat to generate HFiles.
 Would need to indicate the splits we want, probably from a live table.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5773) HtablePool constructor not reading config files in certain cases

2012-04-12 Thread Ted Yu (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252551#comment-13252551
 ] 

Ted Yu commented on HBASE-5773:
---

Patch makes sense.

 HtablePool constructor not reading config files in certain cases
 

 Key: HBASE-5773
 URL: https://issues.apache.org/jira/browse/HBASE-5773
 Project: HBase
  Issue Type: Bug
  Components: client
Affects Versions: 0.90.6, 0.92.1, 0.94.1
Reporter: Ioan Eugen Stan
Priority: Minor
 Fix For: 0.90.7, 0.92.2, 0.94.1

 Attachments: different-config-behaviour.patch


 Creating a HtablePool can issue two behaviour depanding on the constructor 
 called. 
 Case 1: loads the configs from hbase-site
   public HTablePool() {
 this(HBaseConfiguration.create(), Integer.MAX_VALUE);
   }
 Calling this with null values for Configuration: 
 public HTablePool(final Configuration config, final int maxSize) {
 this(config, maxSize, null, null);
   }
 will issue:
  public HTablePool(final Configuration config, final int maxSize,
   final HTableInterfaceFactory tableFactory, PoolType poolType) {
 // Make a new configuration instance so I can safely cleanup when
 // done with the pool.
 this.config = config == null ? new Configuration() : config;
 which does not read the hbase-site config files as 
 HBaseConfiguration.create() does. 
 I've tracked this problem to all versions of hbase. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Resolved] (HBASE-4311) HBaseTestingUtility.createMultiRegions generates table that fails in hbck.

2012-04-12 Thread Jonathan Hsieh (Resolved) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh resolved HBASE-4311.
---

Resolution: Won't Fix

Used other mechanisms to create tables for hbck / HBASE-5128.  Might be better 
to create new issue to remove or deprecate these methods (if this hasn't 
happened yet).

 HBaseTestingUtility.createMultiRegions generates table that fails in hbck.
 --

 Key: HBASE-4311
 URL: https://issues.apache.org/jira/browse/HBASE-4311
 Project: HBase
  Issue Type: Bug
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh

 When refactoring TestHBaseFsckRepair to add more hbck test cases, I noticed 
 that HBaseTestingUtility.createMultiRegions uses an existing table with empty 
 region, adds more regions, and then attempts to remove the region.  The 
 region remains in meta and is causes hbck to report at inconsistency. Ideally 
 these test table generation utility functions should generate clean tables.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5774) Add documentation for WALPlayer to HBase reference guide.

2012-04-12 Thread Lars Hofhansl (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-5774:
-

Attachment: 5774.txt

How's this.

 Add documentation for WALPlayer to HBase reference guide.
 -

 Key: HBASE-5774
 URL: https://issues.apache.org/jira/browse/HBASE-5774
 Project: HBase
  Issue Type: Sub-task
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Attachments: 5774.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5773) HtablePool constructor not reading config files in certain cases

2012-04-12 Thread stack (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252586#comment-13252586
 ] 

stack commented on HBASE-5773:
--

It doesn't apply to 0.90 branch.

 HtablePool constructor not reading config files in certain cases
 

 Key: HBASE-5773
 URL: https://issues.apache.org/jira/browse/HBASE-5773
 Project: HBase
  Issue Type: Bug
  Components: client
Affects Versions: 0.90.6, 0.92.1, 0.94.1
Reporter: Ioan Eugen Stan
Priority: Minor
 Fix For: 0.92.2, 0.94.0

 Attachments: different-config-behaviour.patch


 Creating a HtablePool can issue two behaviour depanding on the constructor 
 called. 
 Case 1: loads the configs from hbase-site
   public HTablePool() {
 this(HBaseConfiguration.create(), Integer.MAX_VALUE);
   }
 Calling this with null values for Configuration: 
 public HTablePool(final Configuration config, final int maxSize) {
 this(config, maxSize, null, null);
   }
 will issue:
  public HTablePool(final Configuration config, final int maxSize,
   final HTableInterfaceFactory tableFactory, PoolType poolType) {
 // Make a new configuration instance so I can safely cleanup when
 // done with the pool.
 this.config = config == null ? new Configuration() : config;
 which does not read the hbase-site config files as 
 HBaseConfiguration.create() does. 
 I've tracked this problem to all versions of hbase. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5773) HtablePool constructor not reading config files in certain cases

2012-04-12 Thread stack (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-5773:
-

   Resolution: Fixed
Fix Version/s: (was: 0.94.1)
   (was: 0.90.7)
   0.94.0
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

Committed to trunk, 0.92, and 0.94 branches.  Thanks for the patch Ioan.

 HtablePool constructor not reading config files in certain cases
 

 Key: HBASE-5773
 URL: https://issues.apache.org/jira/browse/HBASE-5773
 Project: HBase
  Issue Type: Bug
  Components: client
Affects Versions: 0.90.6, 0.92.1, 0.94.1
Reporter: Ioan Eugen Stan
Priority: Minor
 Fix For: 0.92.2, 0.94.0

 Attachments: different-config-behaviour.patch


 Creating a HtablePool can issue two behaviour depanding on the constructor 
 called. 
 Case 1: loads the configs from hbase-site
   public HTablePool() {
 this(HBaseConfiguration.create(), Integer.MAX_VALUE);
   }
 Calling this with null values for Configuration: 
 public HTablePool(final Configuration config, final int maxSize) {
 this(config, maxSize, null, null);
   }
 will issue:
  public HTablePool(final Configuration config, final int maxSize,
   final HTableInterfaceFactory tableFactory, PoolType poolType) {
 // Make a new configuration instance so I can safely cleanup when
 // done with the pool.
 this.config = config == null ? new Configuration() : config;
 which does not read the hbase-site config files as 
 HBaseConfiguration.create() does. 
 I've tracked this problem to all versions of hbase. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Assigned] (HBASE-5774) Add documentation for WALPlayer to HBase reference guide.

2012-04-12 Thread stack (Assigned) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack reassigned HBASE-5774:


Assignee: Doug Meil  (was: Lars Hofhansl)

Assigning Doug to take a looksee (assign back to Lars if you all good w/ it 
Doug)

 Add documentation for WALPlayer to HBase reference guide.
 -

 Key: HBASE-5774
 URL: https://issues.apache.org/jira/browse/HBASE-5774
 Project: HBase
  Issue Type: Sub-task
Reporter: Lars Hofhansl
Assignee: Doug Meil
 Attachments: 5774.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-2223) Handle 10min+ network partitions between clusters

2012-04-12 Thread Jean-Daniel Cryans (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-2223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252602#comment-13252602
 ] 

Jean-Daniel Cryans commented on HBASE-2223:
---

@Himanshu,

It's been in the code base since 0.90.0 but there were a lot of other changes 
on top of it.

 Handle 10min+ network partitions between clusters
 -

 Key: HBASE-2223
 URL: https://issues.apache.org/jira/browse/HBASE-2223
 Project: HBase
  Issue Type: Sub-task
Reporter: Jean-Daniel Cryans
Assignee: Jean-Daniel Cryans
 Fix For: 0.90.0

 Attachments: HBASE-2223.patch


 We need a nice way of handling long network partitions without impacting a 
 master cluster (which pushes the data). Currently it will just retry over and 
 over again.
 I think we could:
  - Stop replication to a slave cluster if it didn't respond for more than 10 
 minutes
  - Keep track of the duration of the partition
  - When the slave cluster comes back, initiate a MR job like HBASE-2221 
 Maybe we want less than 10 minutes, maybe we want this to be all automatic or 
 just the first 2 parts. Discuss.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5620) Convert the client protocol of HRegionInterface to PB

2012-04-12 Thread jirapos...@reviews.apache.org (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252613#comment-13252613
 ] 

jirapos...@reviews.apache.org commented on HBASE-5620:
--



bq.  On 2012-04-12 00:03:44, Michael Stack wrote:
bq.   src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java, 
line 187
bq.   https://reviews.apache.org/r/4629/diff/2/?file=100889#file100889line187
bq.  
bq.   Do these have unit tests?  Looks like it would be easy enough to add 
and could find some nasty bugs early.

Not for now, I can add it in a separate patch if it is ok with you.


bq.  On 2012-04-12 00:03:44, Michael Stack wrote:
bq.   src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java, line 
28
bq.   https://reviews.apache.org/r/4629/diff/2/?file=100885#file100885line28
bq.  
bq.   This is the right comment?

Fixed.


bq.  On 2012-04-12 00:03:44, Michael Stack wrote:
bq.   src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java, line 57
bq.   https://reviews.apache.org/r/4629/diff/2/?file=100882#file100882line57
bq.  
bq.   Fix this text

Fixed.


bq.  On 2012-04-12 00:03:44, Michael Stack wrote:
bq.   src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java, line 
190
bq.   https://reviews.apache.org/r/4629/diff/2/?file=100876#file100876line190
bq.  
bq.   Yeah, we were going to try and get the data up from rpc layer

Yes, after changed pb Result to use key value bytes, we should be able to get 
the data now.


bq.  On 2012-04-12 00:03:44, Michael Stack wrote:
bq.   src/main/java/org/apache/hadoop/hbase/client/HTable.java, line 672
bq.   https://reviews.apache.org/r/4629/diff/2/?file=100875#file100875line672
bq.  
bq.   Is there an EOFE story here too?   You are dealing w/ that elsewhere?

EOFE is also an IOE.  It should be covered as before.


- Jimmy


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4629/#review6863
---


On 2012-04-07 21:30:32, Jimmy Xiang wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/4629/
bq.  ---
bq.  
bq.  (Updated 2012-04-07 21:30:32)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  This is the client protocol part of region interface. The admin protocol 
part will be done in a different jira.
bq.  
bq.  The HRegionInterface is still there since the admin part is not done yet.  
The other reason is that in case some people still wants the old interface
bq.  
bq.  Filters, comparators and coprocessor parameters are still Writable.  They 
will be addressed in different jiras.
bq.  
bq.  The existing client interface is not changed so that we don't break 
existing clients.
bq.  
bq.  
bq.  This addresses bug HBASE-5620.
bq.  https://issues.apache.org/jira/browse/HBASE-5620
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9 
bq.src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa 
bq.src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017 
bq.src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086 
bq.src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 
0b783ce 
bq.src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f 
bq.src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 
9903df3 
bq.src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad 
bq.src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab 
bq.src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java 
cbfa489 
bq.src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393 
bq.src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717 
bq.src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0 
bq.src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457 
bq.
src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 
19ae18c 
bq.src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java 
PRE-CREATION 
bq.src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java 
PRE-CREATION 
bq.src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de 
bq.src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java 
PRE-CREATION 
bq.src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java 
PRE-CREATION 
bq.
src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java 
PRE-CREATION 
bq.

[jira] [Updated] (HBASE-5770) Add a clock skew warning threshold

2012-04-12 Thread stack (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-5770:
-

   Resolution: Fixed
Fix Version/s: 0.94.0
   Status: Resolved  (was: Patch Available)

Committed to 0.94 and to trunk.  Hang (seems to be around test thrift cmd 
line...) seems unrelated.  Thanks for the patch Ian.

 Add a clock skew warning threshold
 --

 Key: HBASE-5770
 URL: https://issues.apache.org/jira/browse/HBASE-5770
 Project: HBase
  Issue Type: Improvement
  Components: master, regionserver
Reporter: Ian Varley
Assignee: Ian Varley
Priority: Minor
 Fix For: 0.94.0

 Attachments: HBASE_5770_v1.patch

   Original Estimate: 1h
  Remaining Estimate: 1h

 There's currently an exception thrown by the master when a region server 
 attempts to start up with clock skew greater than some configured amount 
 (defaulting to 30 seconds). However, it'd be nice to get some warnings logged 
 at a value that isn't severe enough to warrant killing the RS, but still 
 represents significant skew that could affect correctness. Will attach a 
 simple patch to add this as a setting.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5620) Convert the client protocol of HRegionInterface to PB

2012-04-12 Thread jirapos...@reviews.apache.org (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252617#comment-13252617
 ] 

jirapos...@reviews.apache.org commented on HBASE-5620:
--



bq.  On 2012-04-12 04:50:15, Michael Stack wrote:
bq.   What you want to do Jimmy?  On next iteration, if all tests pass, should 
we check it in?  I think its missing unit tests for the new protobuf utility.  
We should add those.  I'd be fine adding them in a separate patch if that would 
suit you.  This thing is already massive.  We could commit to a branch too but 
that probably doesn't get us far; you have your own setup where you are to 
which you can commit, right?

Could you please check it in if all tests pass?  It can save me some 
rebasing/merging efforts. :)

I can add unit tests for the new protobuf utility in a separate patch.  I do 
have my own setup to commit my changes.  I am also working on HBASE-5621.
I can resolve other issues in HBASE-5621, or separate patches.


bq.  On 2012-04-12 04:50:15, Michael Stack wrote:
bq.   src/main/protobuf/Client.proto, line 56
bq.   https://reviews.apache.org/r/4629/diff/2/?file=100900#file100900line56
bq.  
bq.   Call this keyValueBytes then so method  names give more of a clue as 
to what you'll be getting; save on the surprises?

Sure, will fix.


bq.  On 2012-04-12 04:50:15, Michael Stack wrote:
bq.   src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java, 
line 143
bq.   https://reviews.apache.org/r/4629/diff/2/?file=100898#file100898line143
bq.  
bq.   Should this class implement Closeable, Stoppable and Abortable?

RegionServerServices defined Stoppable, Abortable. Now RegionServer implements 
RegionServerServices.
Not sure about Closeable.


bq.  On 2012-04-12 04:50:15, Michael Stack wrote:
bq.   src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java, 
line 98
bq.   https://reviews.apache.org/r/4629/diff/2/?file=100898#file100898line98
bq.  
bq.   Shouldn't this implement Server and RegionServerServices?
bq.   
bq.   I would love to take the time to break this class up into smaller 
pieces but thats not for this JIRA.  Can do that after this goes in.

RegionServer now implements RegionServerServices which extends Server.

It will be great to break this class up.


bq.  On 2012-04-12 04:50:15, Michael Stack wrote:
bq.   src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java, 
line 94
bq.   https://reviews.apache.org/r/4629/diff/2/?file=100898#file100898line94
bq.  
bq.   Fix comment.
bq.   
bq.   Should you explain your intent for this class too?  That 
HRegionServer will wither away?  I think it woud be useful since it was not 
plain to me at first but after you explained it, I was on your side.

Sure, added comments.  Thanks.


- Jimmy


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4629/#review6868
---


On 2012-04-07 21:30:32, Jimmy Xiang wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/4629/
bq.  ---
bq.  
bq.  (Updated 2012-04-07 21:30:32)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  This is the client protocol part of region interface. The admin protocol 
part will be done in a different jira.
bq.  
bq.  The HRegionInterface is still there since the admin part is not done yet.  
The other reason is that in case some people still wants the old interface
bq.  
bq.  Filters, comparators and coprocessor parameters are still Writable.  They 
will be addressed in different jiras.
bq.  
bq.  The existing client interface is not changed so that we don't break 
existing clients.
bq.  
bq.  
bq.  This addresses bug HBASE-5620.
bq.  https://issues.apache.org/jira/browse/HBASE-5620
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9 
bq.src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa 
bq.src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017 
bq.src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086 
bq.src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 
0b783ce 
bq.src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f 
bq.src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 
9903df3 
bq.src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad 
bq.src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab 
bq.src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java 
cbfa489 
bq.

[jira] [Commented] (HBASE-5620) Convert the client protocol of HRegionInterface to PB

2012-04-12 Thread jirapos...@reviews.apache.org (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252619#comment-13252619
 ] 

jirapos...@reviews.apache.org commented on HBASE-5620:
--



bq.  On 2012-04-12 00:03:44, Michael Stack wrote:
bq.   src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java, 
line 187
bq.   https://reviews.apache.org/r/4629/diff/2/?file=100889#file100889line187
bq.  
bq.   Do these have unit tests?  Looks like it would be easy enough to add 
and could find some nasty bugs early.
bq.  
bq.  Jimmy Xiang wrote:
bq.  Not for now, I can add it in a separate patch if it is ok with you.

Yes.  Make it a blocker though.  I see unit tests finding bugs in this stuff.


- Michael


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4629/#review6863
---


On 2012-04-07 21:30:32, Jimmy Xiang wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/4629/
bq.  ---
bq.  
bq.  (Updated 2012-04-07 21:30:32)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  This is the client protocol part of region interface. The admin protocol 
part will be done in a different jira.
bq.  
bq.  The HRegionInterface is still there since the admin part is not done yet.  
The other reason is that in case some people still wants the old interface
bq.  
bq.  Filters, comparators and coprocessor parameters are still Writable.  They 
will be addressed in different jiras.
bq.  
bq.  The existing client interface is not changed so that we don't break 
existing clients.
bq.  
bq.  
bq.  This addresses bug HBASE-5620.
bq.  https://issues.apache.org/jira/browse/HBASE-5620
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9 
bq.src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa 
bq.src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017 
bq.src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086 
bq.src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 
0b783ce 
bq.src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f 
bq.src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 
9903df3 
bq.src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad 
bq.src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab 
bq.src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java 
cbfa489 
bq.src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393 
bq.src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717 
bq.src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0 
bq.src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457 
bq.
src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 
19ae18c 
bq.src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java 
PRE-CREATION 
bq.src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java 
PRE-CREATION 
bq.src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de 
bq.src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java 
PRE-CREATION 
bq.src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java 
PRE-CREATION 
bq.
src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java 
PRE-CREATION 
bq.
src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java 
PRE-CREATION 
bq.
src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 
4026da0 
bq.
src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionAdminProtos.java 
2169310 
bq.
src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java
 b36a9c0 
bq.src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 
8a61f7d 
bq.
src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 
703e73d 
bq.src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f 
bq.src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java 
PRE-CREATION 
bq.src/main/protobuf/Admin.proto PRE-CREATION 
bq.src/main/protobuf/Client.proto PRE-CREATION 
bq.src/main/protobuf/RegionAdmin.proto c64d68b 
bq.src/main/protobuf/RegionClient.proto 358382b 
bq.src/main/protobuf/hbase.proto da78788 
bq.src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java 
c7284dc 
bq.

[jira] [Commented] (HBASE-5620) Convert the client protocol of HRegionInterface to PB

2012-04-12 Thread jirapos...@reviews.apache.org (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252622#comment-13252622
 ] 

jirapos...@reviews.apache.org commented on HBASE-5620:
--



bq.  On 2012-04-12 04:50:15, Michael Stack wrote:
bq.   What you want to do Jimmy?  On next iteration, if all tests pass, should 
we check it in?  I think its missing unit tests for the new protobuf utility.  
We should add those.  I'd be fine adding them in a separate patch if that would 
suit you.  This thing is already massive.  We could commit to a branch too but 
that probably doesn't get us far; you have your own setup where you are to 
which you can commit, right?
bq.  
bq.  Jimmy Xiang wrote:
bq.  Could you please check it in if all tests pass?  It can save me some 
rebasing/merging efforts. :)
bq.  
bq.  I can add unit tests for the new protobuf utility in a separate patch. 
 I do have my own setup to commit my changes.  I am also working on HBASE-5621.
bq.  I can resolve other issues in HBASE-5621, or separate patches.
bq.

I can.  You going to do another version to address above feedback?


bq.  On 2012-04-12 04:50:15, Michael Stack wrote:
bq.   src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java, 
line 143
bq.   https://reviews.apache.org/r/4629/diff/2/?file=100898#file100898line143
bq.  
bq.   Should this class implement Closeable, Stoppable and Abortable?
bq.  
bq.  Jimmy Xiang wrote:
bq.  RegionServerServices defined Stoppable, Abortable. Now RegionServer 
implements RegionServerServices.
bq.  Not sure about Closeable.

Good


- Michael


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4629/#review6868
---


On 2012-04-07 21:30:32, Jimmy Xiang wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/4629/
bq.  ---
bq.  
bq.  (Updated 2012-04-07 21:30:32)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  This is the client protocol part of region interface. The admin protocol 
part will be done in a different jira.
bq.  
bq.  The HRegionInterface is still there since the admin part is not done yet.  
The other reason is that in case some people still wants the old interface
bq.  
bq.  Filters, comparators and coprocessor parameters are still Writable.  They 
will be addressed in different jiras.
bq.  
bq.  The existing client interface is not changed so that we don't break 
existing clients.
bq.  
bq.  
bq.  This addresses bug HBASE-5620.
bq.  https://issues.apache.org/jira/browse/HBASE-5620
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9 
bq.src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa 
bq.src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017 
bq.src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086 
bq.src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 
0b783ce 
bq.src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f 
bq.src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 
9903df3 
bq.src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad 
bq.src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab 
bq.src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java 
cbfa489 
bq.src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393 
bq.src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717 
bq.src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0 
bq.src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457 
bq.
src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 
19ae18c 
bq.src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java 
PRE-CREATION 
bq.src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java 
PRE-CREATION 
bq.src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de 
bq.src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java 
PRE-CREATION 
bq.src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java 
PRE-CREATION 
bq.
src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java 
PRE-CREATION 
bq.
src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java 
PRE-CREATION 
bq.
src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 
4026da0 
bq.

[jira] [Commented] (HBASE-5620) Convert the client protocol of HRegionInterface to PB

2012-04-12 Thread jirapos...@reviews.apache.org (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252623#comment-13252623
 ] 

jirapos...@reviews.apache.org commented on HBASE-5620:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4629/
---

(Updated 2012-04-12 17:25:51.153610)


Review request for hbase.


Changes
---

Addressed Stack's comments.


Summary
---

This is the client protocol part of region interface. The admin protocol part 
will be done in a different jira.

The HRegionInterface is still there since the admin part is not done yet.  The 
other reason is that in case some people still wants the old interface

Filters, comparators and coprocessor parameters are still Writable.  They will 
be addressed in different jiras.

The existing client interface is not changed so that we don't break existing 
clients.


This addresses bug HBASE-5620.
https://issues.apache.org/jira/browse/HBASE-5620


Diffs (updated)
-

  src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9 
  src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 3167f23 
  src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017 
  src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086 
  src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 0b783ce 
  src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f 
  src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3 
  src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad 
  src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab 
  src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489 
  src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393 
  src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717 
  src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0 
  src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 0573c68 
  src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 
b71ae66 
  src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java 
PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java 
PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de 
  src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java 
PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java 
PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java 
PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java 
PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 
4026da0 
  
src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionAdminProtos.java 
2169310 
  
src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java
 b36a9c0 
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 9b34e61 
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 
703e73d 
  src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f 
  src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java 
PRE-CREATION 
  src/main/protobuf/Admin.proto PRE-CREATION 
  src/main/protobuf/Client.proto PRE-CREATION 
  src/main/protobuf/RegionAdmin.proto c64d68b 
  src/main/protobuf/RegionClient.proto 358382b 
  src/main/protobuf/hbase.proto da78788 
  src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc 
  
src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java
 0042468 
  src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java 
e34d8bc 
  src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3 
  src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java 41616c8 
  src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 
6ed4ba2 
  src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java b4dcb83 
  src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5 
  src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java 
cac2989 
  
src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java
 a1bf73b 

Diff: https://reviews.apache.org/r/4629/diff


Testing
---

mvn -PrunAllTests clean test is green, except some flaky tests which need to 
run again.

Also tested it on a real cluster with ycsb and bigtop.


Thanks,

Jimmy



 Convert the client protocol of HRegionInterface to PB
 -

 Key: HBASE-5620
  

[jira] [Updated] (HBASE-5620) Convert the client protocol of HRegionInterface to PB

2012-04-12 Thread Jimmy Xiang (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang updated HBASE-5620:
---

Hadoop Flags: Reviewed
  Status: Patch Available  (was: Open)

 Convert the client protocol of HRegionInterface to PB
 -

 Key: HBASE-5620
 URL: https://issues.apache.org/jira/browse/HBASE-5620
 Project: HBase
  Issue Type: Sub-task
  Components: ipc, master, migration, regionserver
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Fix For: 0.96.0

 Attachments: hbase-5620_v3.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5774) Add documentation for WALPlayer to HBase reference guide.

2012-04-12 Thread Doug Meil (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252624#comment-13252624
 ] 

Doug Meil commented on HBASE-5774:
--

I'll take a look

 Add documentation for WALPlayer to HBase reference guide.
 -

 Key: HBASE-5774
 URL: https://issues.apache.org/jira/browse/HBASE-5774
 Project: HBase
  Issue Type: Sub-task
Reporter: Lars Hofhansl
Assignee: Doug Meil
 Attachments: 5774.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5604) M/R tool to replay WAL files

2012-04-12 Thread Lars Hofhansl (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-5604:
-

Attachment: 5604-v10.txt

all arc lint problems fixed.

 M/R tool to replay WAL files
 

 Key: HBASE-5604
 URL: https://issues.apache.org/jira/browse/HBASE-5604
 Project: HBase
  Issue Type: New Feature
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Attachments: 5604-v10.txt, 5604-v4.txt, 5604-v6.txt, 5604-v7.txt, 
 5604-v8.txt, 5604-v9.txt, HLog-5604-v3.txt


 Just an idea I had. Might be useful for restore of a backup using the HLogs.
 This could an M/R (with a mapper per HLog file).
 The tool would get a timerange and a (set of) table(s). We'd pick the right 
 HLogs based on time before the M/R job is started and then have a mapper per 
 HLog file.
 The mapper would then go through the HLog, filter all WALEdits that didn't 
 fit into the time range or are not any of the tables and then uses 
 HFileOutputFormat to generate HFiles.
 Would need to indicate the splits we want, probably from a live table.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5773) HtablePool constructor not reading config files in certain cases

2012-04-12 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252650#comment-13252650
 ] 

Hudson commented on HBASE-5773:
---

Integrated in HBase-0.94 #105 (See 
[https://builds.apache.org/job/HBase-0.94/105/])
HBASE-5773 HtablePool constructor not reading config files in certain cases 
(Revision 1325381)

 Result = SUCCESS
stack : 
Files : 
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/HTablePool.java


 HtablePool constructor not reading config files in certain cases
 

 Key: HBASE-5773
 URL: https://issues.apache.org/jira/browse/HBASE-5773
 Project: HBase
  Issue Type: Bug
  Components: client
Affects Versions: 0.90.6, 0.92.1, 0.94.1
Reporter: Ioan Eugen Stan
Priority: Minor
 Fix For: 0.92.2, 0.94.0

 Attachments: different-config-behaviour.patch


 Creating a HtablePool can issue two behaviour depanding on the constructor 
 called. 
 Case 1: loads the configs from hbase-site
   public HTablePool() {
 this(HBaseConfiguration.create(), Integer.MAX_VALUE);
   }
 Calling this with null values for Configuration: 
 public HTablePool(final Configuration config, final int maxSize) {
 this(config, maxSize, null, null);
   }
 will issue:
  public HTablePool(final Configuration config, final int maxSize,
   final HTableInterfaceFactory tableFactory, PoolType poolType) {
 // Make a new configuration instance so I can safely cleanup when
 // done with the pool.
 this.config = config == null ? new Configuration() : config;
 which does not read the hbase-site config files as 
 HBaseConfiguration.create() does. 
 I've tracked this problem to all versions of hbase. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Resolved] (HBASE-3443) ICV optimization to look in memstore first and then store files (HBASE-3082) does not work when deletes are in the mix

2012-04-12 Thread Lars Hofhansl (Resolved) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl resolved HBASE-3443.
--

  Resolution: Fixed
Hadoop Flags: Reviewed

Committed to trunk

 ICV optimization to look in memstore first and then store files (HBASE-3082) 
 does not work when deletes are in the mix
 --

 Key: HBASE-3443
 URL: https://issues.apache.org/jira/browse/HBASE-3443
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.90.0, 0.90.1, 0.90.2, 0.90.3, 0.90.4, 0.90.5, 0.90.6, 
 0.92.0, 0.92.1
Reporter: Kannan Muthukkaruppan
Assignee: Lars Hofhansl
Priority: Critical
  Labels: corruption
 Attachments: 3443.txt


 For incrementColumnValue() HBASE-3082 adds an optimization to check memstores 
 first, and only if not present in the memstore then check the store files. In 
 the presence of deletes, the above optimization is not reliable.
 If the column is marked as deleted in the memstore, one should not look 
 further into the store files. But currently, the code does so.
 Sample test code outline:
 {code}
 admin.createTable(desc)
 table = HTable.new(conf, tableName)
 table.incrementColumnValue(Bytes.toBytes(row), cf1name, 
 Bytes.toBytes(column), 5);
 admin.flush(tableName)
 sleep(2)
 del = Delete.new(Bytes.toBytes(row))
 table.delete(del)
 table.incrementColumnValue(Bytes.toBytes(row), cf1name, 
 Bytes.toBytes(column), 5);
 get = Get.new(Bytes.toBytes(row))
 keyValues = table.get(get).raw()
 keyValues.each do |keyValue|
   puts Expect 5; Got Value=#{Bytes.toLong(keyValue.getValue())};
 end
 {code}
 The above prints:
 {code}
 Expect 5; Got Value=10
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5719) Enhance hbck to sideline overlapped mega regions

2012-04-12 Thread Jonathan Hsieh (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-5719:
--

   Resolution: Fixed
Fix Version/s: 0.94.0
   0.92.2
   0.90.7
   Status: Resolved  (was: Patch Available)

Thanks for the patch and the backports Jimmy!

Commited to 0.96/0.94/0.92/0.90.  

 Enhance hbck to sideline overlapped mega regions
 

 Key: HBASE-5719
 URL: https://issues.apache.org/jira/browse/HBASE-5719
 Project: HBase
  Issue Type: New Feature
  Components: hbck
Affects Versions: 0.94.0, 0.96.0
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0

 Attachments: 5719.patch, 5719_0.90.patch, 5719_0.92.patch, 
 5719_0.94.patch, hbase-5719.patch, hbase-5719_0.90.patch, 
 hbase-5719_0.92.patch, hbase-5719_0.94.patch, hbase-5719_v3-new.patch, 
 hbase-5719_v3.patch


 If there are too many regions in one overlapped group (by default, more than 
 10), hbck currently doesn't merge them since it takes time.
 In this case, we can sideline some regions in the group and break the 
 overlapping to fix the inconsistency.  Later on, sidelined regions can be 
 bulk loaded manually.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5620) Convert the client protocol of HRegionInterface to PB

2012-04-12 Thread Hadoop QA (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252687#comment-13252687
 ] 

Hadoop QA commented on HBASE-5620:
--

+1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12522452/hbase-5620_v3.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 30 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

+1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

+1 core tests.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1496//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1496//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1496//console

This message is automatically generated.

 Convert the client protocol of HRegionInterface to PB
 -

 Key: HBASE-5620
 URL: https://issues.apache.org/jira/browse/HBASE-5620
 Project: HBase
  Issue Type: Sub-task
  Components: ipc, master, migration, regionserver
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Fix For: 0.96.0

 Attachments: hbase-5620_v3.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-3443) ICV optimization to look in memstore first and then store files (HBASE-3082) does not work when deletes are in the mix

2012-04-12 Thread Lars Hofhansl (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-3443:
-

Fix Version/s: 0.96.0

 ICV optimization to look in memstore first and then store files (HBASE-3082) 
 does not work when deletes are in the mix
 --

 Key: HBASE-3443
 URL: https://issues.apache.org/jira/browse/HBASE-3443
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.90.0, 0.90.1, 0.90.2, 0.90.3, 0.90.4, 0.90.5, 0.90.6, 
 0.92.0, 0.92.1
Reporter: Kannan Muthukkaruppan
Assignee: Lars Hofhansl
Priority: Critical
  Labels: corruption
 Fix For: 0.96.0

 Attachments: 3443.txt


 For incrementColumnValue() HBASE-3082 adds an optimization to check memstores 
 first, and only if not present in the memstore then check the store files. In 
 the presence of deletes, the above optimization is not reliable.
 If the column is marked as deleted in the memstore, one should not look 
 further into the store files. But currently, the code does so.
 Sample test code outline:
 {code}
 admin.createTable(desc)
 table = HTable.new(conf, tableName)
 table.incrementColumnValue(Bytes.toBytes(row), cf1name, 
 Bytes.toBytes(column), 5);
 admin.flush(tableName)
 sleep(2)
 del = Delete.new(Bytes.toBytes(row))
 table.delete(del)
 table.incrementColumnValue(Bytes.toBytes(row), cf1name, 
 Bytes.toBytes(column), 5);
 get = Get.new(Bytes.toBytes(row))
 keyValues = table.get(get).raw()
 keyValues.each do |keyValue|
   puts Expect 5; Got Value=#{Bytes.toLong(keyValue.getValue())};
 end
 {code}
 The above prints:
 {code}
 Expect 5; Got Value=10
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5604) M/R tool to replay WAL files

2012-04-12 Thread Hadoop QA (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252698#comment-13252698
 ] 

Hadoop QA commented on HBASE-5604:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12522454/5604-v10.txt
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 6 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 3 new Findbugs (version 1.3.9) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

+1 core tests.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1497//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1497//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1497//console

This message is automatically generated.

 M/R tool to replay WAL files
 

 Key: HBASE-5604
 URL: https://issues.apache.org/jira/browse/HBASE-5604
 Project: HBase
  Issue Type: New Feature
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Attachments: 5604-v10.txt, 5604-v4.txt, 5604-v6.txt, 5604-v7.txt, 
 5604-v8.txt, 5604-v9.txt, HLog-5604-v3.txt


 Just an idea I had. Might be useful for restore of a backup using the HLogs.
 This could an M/R (with a mapper per HLog file).
 The tool would get a timerange and a (set of) table(s). We'd pick the right 
 HLogs based on time before the M/R job is started and then have a mapper per 
 HLog file.
 The mapper would then go through the HLog, filter all WALEdits that didn't 
 fit into the time range or are not any of the tables and then uses 
 HFileOutputFormat to generate HFiles.
 Would need to indicate the splits we want, probably from a live table.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-3443) ICV optimization to look in memstore first and then store files (HBASE-3082) does not work when deletes are in the mix

2012-04-12 Thread stack (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252709#comment-13252709
 ] 

stack commented on HBASE-3443:
--

0.94?

 ICV optimization to look in memstore first and then store files (HBASE-3082) 
 does not work when deletes are in the mix
 --

 Key: HBASE-3443
 URL: https://issues.apache.org/jira/browse/HBASE-3443
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.90.0, 0.90.1, 0.90.2, 0.90.3, 0.90.4, 0.90.5, 0.90.6, 
 0.92.0, 0.92.1
Reporter: Kannan Muthukkaruppan
Assignee: Lars Hofhansl
Priority: Critical
  Labels: corruption
 Fix For: 0.96.0

 Attachments: 3443.txt


 For incrementColumnValue() HBASE-3082 adds an optimization to check memstores 
 first, and only if not present in the memstore then check the store files. In 
 the presence of deletes, the above optimization is not reliable.
 If the column is marked as deleted in the memstore, one should not look 
 further into the store files. But currently, the code does so.
 Sample test code outline:
 {code}
 admin.createTable(desc)
 table = HTable.new(conf, tableName)
 table.incrementColumnValue(Bytes.toBytes(row), cf1name, 
 Bytes.toBytes(column), 5);
 admin.flush(tableName)
 sleep(2)
 del = Delete.new(Bytes.toBytes(row))
 table.delete(del)
 table.incrementColumnValue(Bytes.toBytes(row), cf1name, 
 Bytes.toBytes(column), 5);
 get = Get.new(Bytes.toBytes(row))
 keyValues = table.get(get).raw()
 keyValues.each do |keyValue|
   puts Expect 5; Got Value=#{Bytes.toLong(keyValue.getValue())};
 end
 {code}
 The above prints:
 {code}
 Expect 5; Got Value=10
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-3443) ICV optimization to look in memstore first and then store files (HBASE-3082) does not work when deletes are in the mix

2012-04-12 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252711#comment-13252711
 ] 

Hudson commented on HBASE-3443:
---

Integrated in HBase-TRUNK #2746 (See 
[https://builds.apache.org/job/HBase-TRUNK/2746/])
HBASE-3443 ICV optimization to look in memstore first and then store files 
(HBASE-3082) does not work when deletes are in the mix (Revision 1325406)

 Result = SUCCESS
larsh : 
Files : 
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java


 ICV optimization to look in memstore first and then store files (HBASE-3082) 
 does not work when deletes are in the mix
 --

 Key: HBASE-3443
 URL: https://issues.apache.org/jira/browse/HBASE-3443
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.90.0, 0.90.1, 0.90.2, 0.90.3, 0.90.4, 0.90.5, 0.90.6, 
 0.92.0, 0.92.1
Reporter: Kannan Muthukkaruppan
Assignee: Lars Hofhansl
Priority: Critical
  Labels: corruption
 Fix For: 0.96.0

 Attachments: 3443.txt


 For incrementColumnValue() HBASE-3082 adds an optimization to check memstores 
 first, and only if not present in the memstore then check the store files. In 
 the presence of deletes, the above optimization is not reliable.
 If the column is marked as deleted in the memstore, one should not look 
 further into the store files. But currently, the code does so.
 Sample test code outline:
 {code}
 admin.createTable(desc)
 table = HTable.new(conf, tableName)
 table.incrementColumnValue(Bytes.toBytes(row), cf1name, 
 Bytes.toBytes(column), 5);
 admin.flush(tableName)
 sleep(2)
 del = Delete.new(Bytes.toBytes(row))
 table.delete(del)
 table.incrementColumnValue(Bytes.toBytes(row), cf1name, 
 Bytes.toBytes(column), 5);
 get = Get.new(Bytes.toBytes(row))
 keyValues = table.get(get).raw()
 keyValues.each do |keyValue|
   puts Expect 5; Got Value=#{Bytes.toLong(keyValue.getValue())};
 end
 {code}
 The above prints:
 {code}
 Expect 5; Got Value=10
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5719) Enhance hbck to sideline overlapped mega regions

2012-04-12 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252713#comment-13252713
 ] 

Hudson commented on HBASE-5719:
---

Integrated in HBase-TRUNK #2746 (See 
[https://builds.apache.org/job/HBase-TRUNK/2746/])
HBASE-5719 Enhance hbck to sideline overlapped mega regions (Jimmy Xiang) 
(Revision 1325404)

 Result = SUCCESS
jmhsieh : 
Files : 
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/TestRegionSplitCalculator.java


 Enhance hbck to sideline overlapped mega regions
 

 Key: HBASE-5719
 URL: https://issues.apache.org/jira/browse/HBASE-5719
 Project: HBase
  Issue Type: New Feature
  Components: hbck
Affects Versions: 0.94.0, 0.96.0
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0

 Attachments: 5719.patch, 5719_0.90.patch, 5719_0.92.patch, 
 5719_0.94.patch, hbase-5719.patch, hbase-5719_0.90.patch, 
 hbase-5719_0.92.patch, hbase-5719_0.94.patch, hbase-5719_v3-new.patch, 
 hbase-5719_v3.patch


 If there are too many regions in one overlapped group (by default, more than 
 10), hbck currently doesn't merge them since it takes time.
 In this case, we can sideline some regions in the group and break the 
 overlapping to fix the inconsistency.  Later on, sidelined regions can be 
 bulk loaded manually.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-3082) For ICV gets, first look in MemStore before reading StoreFiles

2012-04-12 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-3082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252712#comment-13252712
 ] 

Hudson commented on HBASE-3082:
---

Integrated in HBase-TRUNK #2746 (See 
[https://builds.apache.org/job/HBase-TRUNK/2746/])
HBASE-3443 ICV optimization to look in memstore first and then store files 
(HBASE-3082) does not work when deletes are in the mix (Revision 1325406)

 Result = SUCCESS
larsh : 
Files : 
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java


 For ICV gets, first look in MemStore before reading StoreFiles
 --

 Key: HBASE-3082
 URL: https://issues.apache.org/jira/browse/HBASE-3082
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Reporter: Jonathan Gray
Assignee: Prakash Khemani
 Fix For: 0.90.0

 Attachments: HBASE-3082-FINAL.patch


 For incrementColumnValue operations, it is possible to check MemStore for the 
 column being incremented without sacrificing correctness.  If the column is 
 not found in MemStore, we would then have to do a normal Get that 
 opens/checks all StoreFiles for the given Store.
 In practice, this makes increment operations significantly faster for 
 recently/frequently incremented columns.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5770) Add a clock skew warning threshold

2012-04-12 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252717#comment-13252717
 ] 

Hudson commented on HBASE-5770:
---

Integrated in HBase-0.94 #106 (See 
[https://builds.apache.org/job/HBase-0.94/106/])
HBASE-5770 Add a clock skew warning threshold (Revision 1325388)

 Result = SUCCESS
stack : 
Files : 
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestClockSkewDetection.java


 Add a clock skew warning threshold
 --

 Key: HBASE-5770
 URL: https://issues.apache.org/jira/browse/HBASE-5770
 Project: HBase
  Issue Type: Improvement
  Components: master, regionserver
Reporter: Ian Varley
Assignee: Ian Varley
Priority: Minor
 Fix For: 0.94.0

 Attachments: HBASE_5770_v1.patch

   Original Estimate: 1h
  Remaining Estimate: 1h

 There's currently an exception thrown by the master when a region server 
 attempts to start up with clock skew greater than some configured amount 
 (defaulting to 30 seconds). However, it'd be nice to get some warnings logged 
 at a value that isn't severe enough to warrant killing the RS, but still 
 represents significant skew that could affect correctness. Will attach a 
 simple patch to add this as a setting.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5719) Enhance hbck to sideline overlapped mega regions

2012-04-12 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252718#comment-13252718
 ] 

Hudson commented on HBASE-5719:
---

Integrated in HBase-0.94 #106 (See 
[https://builds.apache.org/job/HBase-0.94/106/])
HBASE-5719 Enhance hbck to sideline overlapped mega regions (Jimmy Xiang) 
(Revision 1325403)

 Result = SUCCESS
jmhsieh : 
Files : 
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/TestRegionSplitCalculator.java


 Enhance hbck to sideline overlapped mega regions
 

 Key: HBASE-5719
 URL: https://issues.apache.org/jira/browse/HBASE-5719
 Project: HBase
  Issue Type: New Feature
  Components: hbck
Affects Versions: 0.94.0, 0.96.0
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0

 Attachments: 5719.patch, 5719_0.90.patch, 5719_0.92.patch, 
 5719_0.94.patch, hbase-5719.patch, hbase-5719_0.90.patch, 
 hbase-5719_0.92.patch, hbase-5719_0.94.patch, hbase-5719_v3-new.patch, 
 hbase-5719_v3.patch


 If there are too many regions in one overlapped group (by default, more than 
 10), hbck currently doesn't merge them since it takes time.
 In this case, we can sideline some regions in the group and break the 
 overlapping to fix the inconsistency.  Later on, sidelined regions can be 
 bulk loaded manually.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-3443) ICV optimization to look in memstore first and then store files (HBASE-3082) does not work when deletes are in the mix

2012-04-12 Thread Lars Hofhansl (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252726#comment-13252726
 ] 

Lars Hofhansl commented on HBASE-3443:
--

@Stack: Wasn't sure about this one. Didn't feel right to add this to a 
performance release :)

You think this should go into 0.94? Might be good to have this correctness fix 
early.

 ICV optimization to look in memstore first and then store files (HBASE-3082) 
 does not work when deletes are in the mix
 --

 Key: HBASE-3443
 URL: https://issues.apache.org/jira/browse/HBASE-3443
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.90.0, 0.90.1, 0.90.2, 0.90.3, 0.90.4, 0.90.5, 0.90.6, 
 0.92.0, 0.92.1
Reporter: Kannan Muthukkaruppan
Assignee: Lars Hofhansl
Priority: Critical
  Labels: corruption
 Fix For: 0.96.0

 Attachments: 3443.txt


 For incrementColumnValue() HBASE-3082 adds an optimization to check memstores 
 first, and only if not present in the memstore then check the store files. In 
 the presence of deletes, the above optimization is not reliable.
 If the column is marked as deleted in the memstore, one should not look 
 further into the store files. But currently, the code does so.
 Sample test code outline:
 {code}
 admin.createTable(desc)
 table = HTable.new(conf, tableName)
 table.incrementColumnValue(Bytes.toBytes(row), cf1name, 
 Bytes.toBytes(column), 5);
 admin.flush(tableName)
 sleep(2)
 del = Delete.new(Bytes.toBytes(row))
 table.delete(del)
 table.incrementColumnValue(Bytes.toBytes(row), cf1name, 
 Bytes.toBytes(column), 5);
 get = Get.new(Bytes.toBytes(row))
 keyValues = table.get(get).raw()
 keyValues.each do |keyValue|
   puts Expect 5; Got Value=#{Bytes.toLong(keyValue.getValue())};
 end
 {code}
 The above prints:
 {code}
 Expect 5; Got Value=10
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-3443) ICV optimization to look in memstore first and then store files (HBASE-3082) does not work when deletes are in the mix

2012-04-12 Thread Jean-Daniel Cryans (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252729#comment-13252729
 ] 

Jean-Daniel Cryans commented on HBASE-3443:
---

Correctness should always come first IMO.

 ICV optimization to look in memstore first and then store files (HBASE-3082) 
 does not work when deletes are in the mix
 --

 Key: HBASE-3443
 URL: https://issues.apache.org/jira/browse/HBASE-3443
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.90.0, 0.90.1, 0.90.2, 0.90.3, 0.90.4, 0.90.5, 0.90.6, 
 0.92.0, 0.92.1
Reporter: Kannan Muthukkaruppan
Assignee: Lars Hofhansl
Priority: Critical
  Labels: corruption
 Fix For: 0.96.0

 Attachments: 3443.txt


 For incrementColumnValue() HBASE-3082 adds an optimization to check memstores 
 first, and only if not present in the memstore then check the store files. In 
 the presence of deletes, the above optimization is not reliable.
 If the column is marked as deleted in the memstore, one should not look 
 further into the store files. But currently, the code does so.
 Sample test code outline:
 {code}
 admin.createTable(desc)
 table = HTable.new(conf, tableName)
 table.incrementColumnValue(Bytes.toBytes(row), cf1name, 
 Bytes.toBytes(column), 5);
 admin.flush(tableName)
 sleep(2)
 del = Delete.new(Bytes.toBytes(row))
 table.delete(del)
 table.incrementColumnValue(Bytes.toBytes(row), cf1name, 
 Bytes.toBytes(column), 5);
 get = Get.new(Bytes.toBytes(row))
 keyValues = table.get(get).raw()
 keyValues.each do |keyValue|
   puts Expect 5; Got Value=#{Bytes.toLong(keyValue.getValue())};
 end
 {code}
 The above prints:
 {code}
 Expect 5; Got Value=10
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5774) Add documentation for WALPlayer to HBase reference guide.

2012-04-12 Thread Doug Meil (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252732#comment-13252732
 ] 

Doug Meil commented on HBASE-5774:
--

+1

 Add documentation for WALPlayer to HBase reference guide.
 -

 Key: HBASE-5774
 URL: https://issues.apache.org/jira/browse/HBASE-5774
 Project: HBase
  Issue Type: Sub-task
Reporter: Lars Hofhansl
Assignee: Doug Meil
 Attachments: 5774.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-3443) ICV optimization to look in memstore first and then store files (HBASE-3082) does not work when deletes are in the mix

2012-04-12 Thread Jean-Daniel Cryans (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252730#comment-13252730
 ] 

Jean-Daniel Cryans commented on HBASE-3443:
---

The release notes should mention this tho.

 ICV optimization to look in memstore first and then store files (HBASE-3082) 
 does not work when deletes are in the mix
 --

 Key: HBASE-3443
 URL: https://issues.apache.org/jira/browse/HBASE-3443
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.90.0, 0.90.1, 0.90.2, 0.90.3, 0.90.4, 0.90.5, 0.90.6, 
 0.92.0, 0.92.1
Reporter: Kannan Muthukkaruppan
Assignee: Lars Hofhansl
Priority: Critical
  Labels: corruption
 Fix For: 0.96.0

 Attachments: 3443.txt


 For incrementColumnValue() HBASE-3082 adds an optimization to check memstores 
 first, and only if not present in the memstore then check the store files. In 
 the presence of deletes, the above optimization is not reliable.
 If the column is marked as deleted in the memstore, one should not look 
 further into the store files. But currently, the code does so.
 Sample test code outline:
 {code}
 admin.createTable(desc)
 table = HTable.new(conf, tableName)
 table.incrementColumnValue(Bytes.toBytes(row), cf1name, 
 Bytes.toBytes(column), 5);
 admin.flush(tableName)
 sleep(2)
 del = Delete.new(Bytes.toBytes(row))
 table.delete(del)
 table.incrementColumnValue(Bytes.toBytes(row), cf1name, 
 Bytes.toBytes(column), 5);
 get = Get.new(Bytes.toBytes(row))
 keyValues = table.get(get).raw()
 keyValues.each do |keyValue|
   puts Expect 5; Got Value=#{Bytes.toLong(keyValue.getValue())};
 end
 {code}
 The above prints:
 {code}
 Expect 5; Got Value=10
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Created] (HBASE-5775) ZKUtil doesn't handle deleteRecurisively cleanly

2012-04-12 Thread Jesse Yates (Created) (JIRA)
ZKUtil doesn't handle deleteRecurisively cleanly


 Key: HBASE-5775
 URL: https://issues.apache.org/jira/browse/HBASE-5775
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: Jesse Yates
Assignee: Jesse Yates
 Fix For: 0.94.0, 0.96.0


ZKUtil.deleteNodeRecursively()'s contract says that it handles deletion of the 
node and all its children. However, nothing is mentioned as to what happens if 
the node you are attempting to delete doesn't actually exist. Turns out, it 
throws a null pointer exception. I
'm proposing that we change the code s.t. it handles the case where the parent 
is already gone and exits cleanly, rather than failing horribly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5775) ZKUtil doesn't handle deleteRecurisively cleanly

2012-04-12 Thread Jesse Yates (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252728#comment-13252728
 ] 

Jesse Yates commented on HBASE-5775:


patch coming momentarily.

 ZKUtil doesn't handle deleteRecurisively cleanly
 

 Key: HBASE-5775
 URL: https://issues.apache.org/jira/browse/HBASE-5775
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: Jesse Yates
Assignee: Jesse Yates
 Fix For: 0.94.0, 0.96.0


 ZKUtil.deleteNodeRecursively()'s contract says that it handles deletion of 
 the node and all its children. However, nothing is mentioned as to what 
 happens if the node you are attempting to delete doesn't actually exist. 
 Turns out, it throws a null pointer exception. I
 'm proposing that we change the code s.t. it handles the case where the 
 parent is already gone and exits cleanly, rather than failing horribly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5774) Add documentation for WALPlayer to HBase reference guide.

2012-04-12 Thread Doug Meil (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-5774:
-

Assignee: Lars Hofhansl  (was: Doug Meil)

 Add documentation for WALPlayer to HBase reference guide.
 -

 Key: HBASE-5774
 URL: https://issues.apache.org/jira/browse/HBASE-5774
 Project: HBase
  Issue Type: Sub-task
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Attachments: 5774.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5775) ZKUtil doesn't handle deleteRecurisively cleanly

2012-04-12 Thread Jesse Yates (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jesse Yates updated HBASE-5775:
---

Attachment: java_HBASE-5775.patch

Attaching patch. Simple one line fix and updating associated test.

 ZKUtil doesn't handle deleteRecurisively cleanly
 

 Key: HBASE-5775
 URL: https://issues.apache.org/jira/browse/HBASE-5775
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: Jesse Yates
Assignee: Jesse Yates
 Fix For: 0.94.0, 0.96.0

 Attachments: java_HBASE-5775.patch


 ZKUtil.deleteNodeRecursively()'s contract says that it handles deletion of 
 the node and all its children. However, nothing is mentioned as to what 
 happens if the node you are attempting to delete doesn't actually exist. 
 Turns out, it throws a null pointer exception. I
 'm proposing that we change the code s.t. it handles the case where the 
 parent is already gone and exits cleanly, rather than failing horribly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-3443) ICV optimization to look in memstore first and then store files (HBASE-3082) does not work when deletes are in the mix

2012-04-12 Thread Lars Hofhansl (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-3443:
-

Release Note: 
This is a correctness fix and will incur a 10-20% performance penalty for ICV 
and Increment operations. Other operations are not affected.


 ICV optimization to look in memstore first and then store files (HBASE-3082) 
 does not work when deletes are in the mix
 --

 Key: HBASE-3443
 URL: https://issues.apache.org/jira/browse/HBASE-3443
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.90.0, 0.90.1, 0.90.2, 0.90.3, 0.90.4, 0.90.5, 0.90.6, 
 0.92.0, 0.92.1
Reporter: Kannan Muthukkaruppan
Assignee: Lars Hofhansl
Priority: Critical
  Labels: corruption
 Fix For: 0.96.0

 Attachments: 3443.txt


 For incrementColumnValue() HBASE-3082 adds an optimization to check memstores 
 first, and only if not present in the memstore then check the store files. In 
 the presence of deletes, the above optimization is not reliable.
 If the column is marked as deleted in the memstore, one should not look 
 further into the store files. But currently, the code does so.
 Sample test code outline:
 {code}
 admin.createTable(desc)
 table = HTable.new(conf, tableName)
 table.incrementColumnValue(Bytes.toBytes(row), cf1name, 
 Bytes.toBytes(column), 5);
 admin.flush(tableName)
 sleep(2)
 del = Delete.new(Bytes.toBytes(row))
 table.delete(del)
 table.incrementColumnValue(Bytes.toBytes(row), cf1name, 
 Bytes.toBytes(column), 5);
 get = Get.new(Bytes.toBytes(row))
 keyValues = table.get(get).raw()
 keyValues.each do |keyValue|
   puts Expect 5; Got Value=#{Bytes.toLong(keyValue.getValue())};
 end
 {code}
 The above prints:
 {code}
 Expect 5; Got Value=10
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5775) ZKUtil doesn't handle deleteRecurisively cleanly

2012-04-12 Thread stack (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-5775:
-

Status: Patch Available  (was: Open)

+1 on patch

 ZKUtil doesn't handle deleteRecurisively cleanly
 

 Key: HBASE-5775
 URL: https://issues.apache.org/jira/browse/HBASE-5775
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: Jesse Yates
Assignee: Jesse Yates
 Fix For: 0.94.0, 0.96.0

 Attachments: java_HBASE-5775.patch


 ZKUtil.deleteNodeRecursively()'s contract says that it handles deletion of 
 the node and all its children. However, nothing is mentioned as to what 
 happens if the node you are attempting to delete doesn't actually exist. 
 Turns out, it throws a null pointer exception. I
 'm proposing that we change the code s.t. it handles the case where the 
 parent is already gone and exits cleanly, rather than failing horribly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-3443) ICV optimization to look in memstore first and then store files (HBASE-3082) does not work when deletes are in the mix

2012-04-12 Thread stack (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252744#comment-13252744
 ] 

stack commented on HBASE-3443:
--

Oh, and if you don't fix it, you'll have to explain why you didn't to BenoƮt.

 ICV optimization to look in memstore first and then store files (HBASE-3082) 
 does not work when deletes are in the mix
 --

 Key: HBASE-3443
 URL: https://issues.apache.org/jira/browse/HBASE-3443
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.90.0, 0.90.1, 0.90.2, 0.90.3, 0.90.4, 0.90.5, 0.90.6, 
 0.92.0, 0.92.1
Reporter: Kannan Muthukkaruppan
Assignee: Lars Hofhansl
Priority: Critical
  Labels: corruption
 Fix For: 0.96.0

 Attachments: 3443.txt


 For incrementColumnValue() HBASE-3082 adds an optimization to check memstores 
 first, and only if not present in the memstore then check the store files. In 
 the presence of deletes, the above optimization is not reliable.
 If the column is marked as deleted in the memstore, one should not look 
 further into the store files. But currently, the code does so.
 Sample test code outline:
 {code}
 admin.createTable(desc)
 table = HTable.new(conf, tableName)
 table.incrementColumnValue(Bytes.toBytes(row), cf1name, 
 Bytes.toBytes(column), 5);
 admin.flush(tableName)
 sleep(2)
 del = Delete.new(Bytes.toBytes(row))
 table.delete(del)
 table.incrementColumnValue(Bytes.toBytes(row), cf1name, 
 Bytes.toBytes(column), 5);
 get = Get.new(Bytes.toBytes(row))
 keyValues = table.get(get).raw()
 keyValues.each do |keyValue|
   puts Expect 5; Got Value=#{Bytes.toLong(keyValue.getValue())};
 end
 {code}
 The above prints:
 {code}
 Expect 5; Got Value=10
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-3443) ICV optimization to look in memstore first and then store files (HBASE-3082) does not work when deletes are in the mix

2012-04-12 Thread Lars Hofhansl (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-3443:
-

Fix Version/s: 0.94.0

a'right, committed to 0.94 aswell, and added release notes.

 ICV optimization to look in memstore first and then store files (HBASE-3082) 
 does not work when deletes are in the mix
 --

 Key: HBASE-3443
 URL: https://issues.apache.org/jira/browse/HBASE-3443
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.90.0, 0.90.1, 0.90.2, 0.90.3, 0.90.4, 0.90.5, 0.90.6, 
 0.92.0, 0.92.1
Reporter: Kannan Muthukkaruppan
Assignee: Lars Hofhansl
Priority: Critical
  Labels: corruption
 Fix For: 0.94.0, 0.96.0

 Attachments: 3443.txt


 For incrementColumnValue() HBASE-3082 adds an optimization to check memstores 
 first, and only if not present in the memstore then check the store files. In 
 the presence of deletes, the above optimization is not reliable.
 If the column is marked as deleted in the memstore, one should not look 
 further into the store files. But currently, the code does so.
 Sample test code outline:
 {code}
 admin.createTable(desc)
 table = HTable.new(conf, tableName)
 table.incrementColumnValue(Bytes.toBytes(row), cf1name, 
 Bytes.toBytes(column), 5);
 admin.flush(tableName)
 sleep(2)
 del = Delete.new(Bytes.toBytes(row))
 table.delete(del)
 table.incrementColumnValue(Bytes.toBytes(row), cf1name, 
 Bytes.toBytes(column), 5);
 get = Get.new(Bytes.toBytes(row))
 keyValues = table.get(get).raw()
 keyValues.each do |keyValue|
   puts Expect 5; Got Value=#{Bytes.toLong(keyValue.getValue())};
 end
 {code}
 The above prints:
 {code}
 Expect 5; Got Value=10
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5774) Add documentation for WALPlayer to HBase reference guide.

2012-04-12 Thread Lars Hofhansl (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252755#comment-13252755
 ] 

Lars Hofhansl commented on HBASE-5774:
--

Thanks Doug. I think I'll just roll this up into parent.

 Add documentation for WALPlayer to HBase reference guide.
 -

 Key: HBASE-5774
 URL: https://issues.apache.org/jira/browse/HBASE-5774
 Project: HBase
  Issue Type: Sub-task
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Attachments: 5774.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5604) M/R tool to replay WAL files

2012-04-12 Thread Lars Hofhansl (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-5604:
-

Attachment: 5604-v11.txt

Patch that includes the documentation from HBASE-5774.

 M/R tool to replay WAL files
 

 Key: HBASE-5604
 URL: https://issues.apache.org/jira/browse/HBASE-5604
 Project: HBase
  Issue Type: New Feature
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Attachments: 5604-v10.txt, 5604-v11.txt, 5604-v4.txt, 5604-v6.txt, 
 5604-v7.txt, 5604-v8.txt, 5604-v9.txt, HLog-5604-v3.txt


 Just an idea I had. Might be useful for restore of a backup using the HLogs.
 This could an M/R (with a mapper per HLog file).
 The tool would get a timerange and a (set of) table(s). We'd pick the right 
 HLogs based on time before the M/R job is started and then have a mapper per 
 HLog file.
 The mapper would then go through the HLog, filter all WALEdits that didn't 
 fit into the time range or are not any of the tables and then uses 
 HFileOutputFormat to generate HFiles.
 Would need to indicate the splits we want, probably from a live table.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Resolved] (HBASE-5774) Add documentation for WALPlayer to HBase reference guide.

2012-04-12 Thread Lars Hofhansl (Resolved) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl resolved HBASE-5774.
--

Resolution: Duplicate

 Add documentation for WALPlayer to HBase reference guide.
 -

 Key: HBASE-5774
 URL: https://issues.apache.org/jira/browse/HBASE-5774
 Project: HBase
  Issue Type: Sub-task
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Attachments: 5774.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5773) HtablePool constructor not reading config files in certain cases

2012-04-12 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252767#comment-13252767
 ] 

Hudson commented on HBASE-5773:
---

Integrated in HBase-0.92 #368 (See 
[https://builds.apache.org/job/HBase-0.92/368/])
HBASE-5773 HtablePool constructor not reading config files in certain cases 
(Revision 1325383)

 Result = FAILURE
stack : 
Files : 
* /hbase/branches/0.92/CHANGES.txt
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/client/HTablePool.java


 HtablePool constructor not reading config files in certain cases
 

 Key: HBASE-5773
 URL: https://issues.apache.org/jira/browse/HBASE-5773
 Project: HBase
  Issue Type: Bug
  Components: client
Affects Versions: 0.90.6, 0.92.1, 0.94.1
Reporter: Ioan Eugen Stan
Priority: Minor
 Fix For: 0.92.2, 0.94.0

 Attachments: different-config-behaviour.patch


 Creating a HtablePool can issue two behaviour depanding on the constructor 
 called. 
 Case 1: loads the configs from hbase-site
   public HTablePool() {
 this(HBaseConfiguration.create(), Integer.MAX_VALUE);
   }
 Calling this with null values for Configuration: 
 public HTablePool(final Configuration config, final int maxSize) {
 this(config, maxSize, null, null);
   }
 will issue:
  public HTablePool(final Configuration config, final int maxSize,
   final HTableInterfaceFactory tableFactory, PoolType poolType) {
 // Make a new configuration instance so I can safely cleanup when
 // done with the pool.
 this.config = config == null ? new Configuration() : config;
 which does not read the hbase-site config files as 
 HBaseConfiguration.create() does. 
 I've tracked this problem to all versions of hbase. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5300) Research website symolic links for renaming of HBase book to HBase Reference Guide

2012-04-12 Thread Doug Meil (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252777#comment-13252777
 ] 

Doug Meil commented on HBASE-5300:
--

I'm not sure we need to do this.  The document name is book.html on the 
website, but the document title has been changed to HBase Reference Guide.

I'm wondering if changing all the filenames is overkill given that everybody 
calls the online book the Reference Guide now.

 Research website symolic links for renaming of HBase book to HBase 
 Reference Guide
 --

 Key: HBASE-5300
 URL: https://issues.apache.org/jira/browse/HBASE-5300
 Project: HBase
  Issue Type: Task
Reporter: Doug Meil
Assignee: stack

 Creating this task per a conversation I had with stack last week.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5604) M/R tool to replay WAL files

2012-04-12 Thread Hadoop QA (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252812#comment-13252812
 ] 

Hadoop QA commented on HBASE-5604:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12522473/5604-v11.txt
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 6 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 3 new Findbugs (version 1.3.9) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
   org.apache.hadoop.hbase.replication.TestReplication
  org.apache.hadoop.hbase.client.TestFromClientSide
  org.apache.hadoop.hbase.io.hfile.TestForceCacheImportantBlocks

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1498//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1498//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1498//console

This message is automatically generated.

 M/R tool to replay WAL files
 

 Key: HBASE-5604
 URL: https://issues.apache.org/jira/browse/HBASE-5604
 Project: HBase
  Issue Type: New Feature
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Attachments: 5604-v10.txt, 5604-v11.txt, 5604-v4.txt, 5604-v6.txt, 
 5604-v7.txt, 5604-v8.txt, 5604-v9.txt, HLog-5604-v3.txt


 Just an idea I had. Might be useful for restore of a backup using the HLogs.
 This could an M/R (with a mapper per HLog file).
 The tool would get a timerange and a (set of) table(s). We'd pick the right 
 HLogs based on time before the M/R job is started and then have a mapper per 
 HLog file.
 The mapper would then go through the HLog, filter all WALEdits that didn't 
 fit into the time range or are not any of the tables and then uses 
 HFileOutputFormat to generate HFiles.
 Would need to indicate the splits we want, probably from a live table.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-3443) ICV optimization to look in memstore first and then store files (HBASE-3082) does not work when deletes are in the mix

2012-04-12 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252813#comment-13252813
 ] 

Hudson commented on HBASE-3443:
---

Integrated in HBase-0.94 #107 (See 
[https://builds.apache.org/job/HBase-0.94/107/])
HBASE-3443 ICV optimization to look in memstore first and then store files 
(HBASE-3082) does not work when deletes are in the mix (Revision 1325453)

 Result = SUCCESS
larsh : 
Files : 
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java


 ICV optimization to look in memstore first and then store files (HBASE-3082) 
 does not work when deletes are in the mix
 --

 Key: HBASE-3443
 URL: https://issues.apache.org/jira/browse/HBASE-3443
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.90.0, 0.90.1, 0.90.2, 0.90.3, 0.90.4, 0.90.5, 0.90.6, 
 0.92.0, 0.92.1
Reporter: Kannan Muthukkaruppan
Assignee: Lars Hofhansl
Priority: Critical
  Labels: corruption
 Fix For: 0.94.0, 0.96.0

 Attachments: 3443.txt


 For incrementColumnValue() HBASE-3082 adds an optimization to check memstores 
 first, and only if not present in the memstore then check the store files. In 
 the presence of deletes, the above optimization is not reliable.
 If the column is marked as deleted in the memstore, one should not look 
 further into the store files. But currently, the code does so.
 Sample test code outline:
 {code}
 admin.createTable(desc)
 table = HTable.new(conf, tableName)
 table.incrementColumnValue(Bytes.toBytes(row), cf1name, 
 Bytes.toBytes(column), 5);
 admin.flush(tableName)
 sleep(2)
 del = Delete.new(Bytes.toBytes(row))
 table.delete(del)
 table.incrementColumnValue(Bytes.toBytes(row), cf1name, 
 Bytes.toBytes(column), 5);
 get = Get.new(Bytes.toBytes(row))
 keyValues = table.get(get).raw()
 keyValues.each do |keyValue|
   puts Expect 5; Got Value=#{Bytes.toLong(keyValue.getValue())};
 end
 {code}
 The above prints:
 {code}
 Expect 5; Got Value=10
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-3082) For ICV gets, first look in MemStore before reading StoreFiles

2012-04-12 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-3082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252814#comment-13252814
 ] 

Hudson commented on HBASE-3082:
---

Integrated in HBase-0.94 #107 (See 
[https://builds.apache.org/job/HBase-0.94/107/])
HBASE-3443 ICV optimization to look in memstore first and then store files 
(HBASE-3082) does not work when deletes are in the mix (Revision 1325453)

 Result = SUCCESS
larsh : 
Files : 
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java


 For ICV gets, first look in MemStore before reading StoreFiles
 --

 Key: HBASE-3082
 URL: https://issues.apache.org/jira/browse/HBASE-3082
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Reporter: Jonathan Gray
Assignee: Prakash Khemani
 Fix For: 0.90.0

 Attachments: HBASE-3082-FINAL.patch


 For incrementColumnValue operations, it is possible to check MemStore for the 
 column being incremented without sacrificing correctness.  If the column is 
 not found in MemStore, we would then have to do a normal Get that 
 opens/checks all StoreFiles for the given Store.
 In practice, this makes increment operations significantly faster for 
 recently/frequently incremented columns.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5300) Research website symolic links for renaming of HBase book to HBase Reference Guide

2012-04-12 Thread stack (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-5300:
-

Attachment: 5300.txt

Patch to fix two locations that talk about 'HBase book'.

 Research website symolic links for renaming of HBase book to HBase 
 Reference Guide
 --

 Key: HBASE-5300
 URL: https://issues.apache.org/jira/browse/HBASE-5300
 Project: HBase
  Issue Type: Task
Reporter: Doug Meil
Assignee: stack
 Fix For: 0.96.0

 Attachments: 5300.txt


 Creating this task per a conversation I had with stack last week.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Resolved] (HBASE-5300) Research website symolic links for renaming of HBase book to HBase Reference Guide

2012-04-12 Thread stack (Resolved) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-5300.
--

   Resolution: Fixed
Fix Version/s: 0.96.0

I committed attached patch.  Everywhere else seems to talk about Reference 
Guide.  Closing this.  Good on you Doug.

 Research website symolic links for renaming of HBase book to HBase 
 Reference Guide
 --

 Key: HBASE-5300
 URL: https://issues.apache.org/jira/browse/HBASE-5300
 Project: HBase
  Issue Type: Task
Reporter: Doug Meil
Assignee: stack
 Fix For: 0.96.0

 Attachments: 5300.txt


 Creating this task per a conversation I had with stack last week.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5775) ZKUtil doesn't handle deleteRecurisively cleanly

2012-04-12 Thread Lars Hofhansl (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252831#comment-13252831
 ] 

Lars Hofhansl commented on HBASE-5775:
--

Will commit when HadoopQA is done.

 ZKUtil doesn't handle deleteRecurisively cleanly
 

 Key: HBASE-5775
 URL: https://issues.apache.org/jira/browse/HBASE-5775
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: Jesse Yates
Assignee: Jesse Yates
 Fix For: 0.94.0, 0.96.0

 Attachments: java_HBASE-5775.patch


 ZKUtil.deleteNodeRecursively()'s contract says that it handles deletion of 
 the node and all its children. However, nothing is mentioned as to what 
 happens if the node you are attempting to delete doesn't actually exist. 
 Turns out, it throws a null pointer exception. I
 'm proposing that we change the code s.t. it handles the case where the 
 parent is already gone and exits cleanly, rather than failing horribly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5736) ThriftServerRunner.HbaseHandler.mutateRow() does not use ByteBuffer correctly

2012-04-12 Thread Scott Chen (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252845#comment-13252845
 ] 

Scott Chen commented on HBASE-5736:
---

Thanks for doing the 94 patch

 ThriftServerRunner.HbaseHandler.mutateRow() does not use ByteBuffer correctly
 -

 Key: HBASE-5736
 URL: https://issues.apache.org/jira/browse/HBASE-5736
 Project: HBase
  Issue Type: Bug
Reporter: Scott Chen
Assignee: Scott Chen
 Fix For: 0.94.0, 0.96.0

 Attachments: 5736-94.txt, HBASE-5736.D2649.1.patch, 
 HBASE-5736.D2649.2.patch, HBASE-5736.D2649.3.patch


 We have fixed similar bug in
 https://issues.apache.org/jira/browse/HBASE-5507
 It uses ByteBuffer.array() to read the ByteBuffer.
 This will ignore the offset return the whole underlying byte array.
 The bug can be triggered by using framed Transport thrift servers.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5775) ZKUtil doesn't handle deleteRecurisively cleanly

2012-04-12 Thread Hadoop QA (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252861#comment-13252861
 ] 

Hadoop QA commented on HBASE-5775:
--

+1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12522467/java_HBASE-5775.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 3 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

+1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

+1 core tests.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1499//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1499//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1499//console

This message is automatically generated.

 ZKUtil doesn't handle deleteRecurisively cleanly
 

 Key: HBASE-5775
 URL: https://issues.apache.org/jira/browse/HBASE-5775
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: Jesse Yates
Assignee: Jesse Yates
 Fix For: 0.94.0, 0.96.0

 Attachments: java_HBASE-5775.patch


 ZKUtil.deleteNodeRecursively()'s contract says that it handles deletion of 
 the node and all its children. However, nothing is mentioned as to what 
 happens if the node you are attempting to delete doesn't actually exist. 
 Turns out, it throws a null pointer exception. I
 'm proposing that we change the code s.t. it handles the case where the 
 parent is already gone and exits cleanly, rather than failing horribly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Created] (HBASE-5776) HTableMultiplexer

2012-04-12 Thread Liyin Tang (Created) (JIRA)
HTableMultiplexer 
--

 Key: HBASE-5776
 URL: https://issues.apache.org/jira/browse/HBASE-5776
 Project: HBase
  Issue Type: Improvement
Reporter: Liyin Tang
Assignee: Liyin Tang


There is a known issue in HBase client that single slow/dead region server 
could slow down the multiput operations across all the region servers. So the 
HBase client will be as slow as the slowest region server in the cluster. 
 
To solve this problem, HTableMultiplexer will separate the multiput submitting 
threads with the flush threads, which means the multiput operation will be a 
nonblocking operation. 
The submitting thread will shard all the puts into different queues based on 
its destination region server and return immediately. The flush threads will 
flush these puts from each queue to its destination region server. 

Currently the HTableMultiplexer only supports the put operation.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Created] (HBASE-5777) MiniHBaseCluster cannot start multiple region servers

2012-04-12 Thread Jimmy Xiang (Created) (JIRA)
MiniHBaseCluster cannot start multiple region servers
-

 Key: HBASE-5777
 URL: https://issues.apache.org/jira/browse/HBASE-5777
 Project: HBase
  Issue Type: Test
Reporter: Jimmy Xiang


MiniHBaseCluster can try to start multiple region servers.  But all of them 
except one will die in putting up the web UI
because of BindException since HConstants.REGIONSERVER_INFO_PORT_AUTO is set to 
false by default.

This issue will make many unit tests depends on multiple region servers flaky, 
such as TestAdmin.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5777) MiniHBaseCluster cannot start multiple region servers

2012-04-12 Thread Jimmy Xiang (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang updated HBASE-5777:
---

Attachment: hbase-5777.patch

 MiniHBaseCluster cannot start multiple region servers
 -

 Key: HBASE-5777
 URL: https://issues.apache.org/jira/browse/HBASE-5777
 Project: HBase
  Issue Type: Test
Reporter: Jimmy Xiang
 Attachments: hbase-5777.patch


 MiniHBaseCluster can try to start multiple region servers.  But all of them 
 except one will die in putting up the web UI
 because of BindException since HConstants.REGIONSERVER_INFO_PORT_AUTO is set 
 to false by default.
 This issue will make many unit tests depends on multiple region servers 
 flaky, such as TestAdmin.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Assigned] (HBASE-5777) MiniHBaseCluster cannot start multiple region servers

2012-04-12 Thread Jimmy Xiang (Assigned) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang reassigned HBASE-5777:
--

Assignee: Jimmy Xiang

 MiniHBaseCluster cannot start multiple region servers
 -

 Key: HBASE-5777
 URL: https://issues.apache.org/jira/browse/HBASE-5777
 Project: HBase
  Issue Type: Test
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Attachments: hbase-5777.patch


 MiniHBaseCluster can try to start multiple region servers.  But all of them 
 except one will die in putting up the web UI
 because of BindException since HConstants.REGIONSERVER_INFO_PORT_AUTO is set 
 to false by default.
 This issue will make many unit tests depends on multiple region servers 
 flaky, such as TestAdmin.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5777) MiniHBaseCluster cannot start multiple region servers

2012-04-12 Thread Jimmy Xiang (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang updated HBASE-5777:
---

Status: Patch Available  (was: Open)

 MiniHBaseCluster cannot start multiple region servers
 -

 Key: HBASE-5777
 URL: https://issues.apache.org/jira/browse/HBASE-5777
 Project: HBase
  Issue Type: Test
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Attachments: hbase-5777.patch


 MiniHBaseCluster can try to start multiple region servers.  But all of them 
 except one will die in putting up the web UI
 because of BindException since HConstants.REGIONSERVER_INFO_PORT_AUTO is set 
 to false by default.
 This issue will make many unit tests depends on multiple region servers 
 flaky, such as TestAdmin.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5777) MiniHBaseCluster cannot start multiple region servers

2012-04-12 Thread Jimmy Xiang (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang updated HBASE-5777:
---

Description: 
MiniHBaseCluster can try to start multiple region servers.  But all of them 
except one will die in putting up the web UI
because of BindException since HConstants.REGIONSERVER_INFO_PORT_AUTO is set to 
false by default.

This issue will make many unit tests depending on multiple region servers 
flaky, such as TestAdmin.

  was:
MiniHBaseCluster can try to start multiple region servers.  But all of them 
except one will die in putting up the web UI
because of BindException since HConstants.REGIONSERVER_INFO_PORT_AUTO is set to 
false by default.

This issue will make many unit tests depends on multiple region servers flaky, 
such as TestAdmin.


 MiniHBaseCluster cannot start multiple region servers
 -

 Key: HBASE-5777
 URL: https://issues.apache.org/jira/browse/HBASE-5777
 Project: HBase
  Issue Type: Test
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Attachments: hbase-5777.patch


 MiniHBaseCluster can try to start multiple region servers.  But all of them 
 except one will die in putting up the web UI
 because of BindException since HConstants.REGIONSERVER_INFO_PORT_AUTO is set 
 to false by default.
 This issue will make many unit tests depending on multiple region servers 
 flaky, such as TestAdmin.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5777) MiniHBaseCluster cannot start multiple region servers

2012-04-12 Thread stack (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252881#comment-13252881
 ] 

stack commented on HBASE-5777:
--

We have an hbase-site.xml at src/test that is used when we run tests.  It 
disables the UI.  You think we should apply this patch too Jimmy?

 MiniHBaseCluster cannot start multiple region servers
 -

 Key: HBASE-5777
 URL: https://issues.apache.org/jira/browse/HBASE-5777
 Project: HBase
  Issue Type: Test
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Attachments: hbase-5777.patch


 MiniHBaseCluster can try to start multiple region servers.  But all of them 
 except one will die in putting up the web UI
 because of BindException since HConstants.REGIONSERVER_INFO_PORT_AUTO is set 
 to false by default.
 This issue will make many unit tests depending on multiple region servers 
 flaky, such as TestAdmin.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5775) ZKUtil doesn't handle deleteRecurisively cleanly

2012-04-12 Thread Lars Hofhansl (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-5775:
-

  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Committed to 0.94 and 0.96.

 ZKUtil doesn't handle deleteRecurisively cleanly
 

 Key: HBASE-5775
 URL: https://issues.apache.org/jira/browse/HBASE-5775
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: Jesse Yates
Assignee: Jesse Yates
 Fix For: 0.94.0, 0.96.0

 Attachments: java_HBASE-5775.patch


 ZKUtil.deleteNodeRecursively()'s contract says that it handles deletion of 
 the node and all its children. However, nothing is mentioned as to what 
 happens if the node you are attempting to delete doesn't actually exist. 
 Turns out, it throws a null pointer exception. I
 'm proposing that we change the code s.t. it handles the case where the 
 parent is already gone and exits cleanly, rather than failing horribly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5775) ZKUtil doesn't handle deleteRecurisively cleanly

2012-04-12 Thread Lars Hofhansl (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252889#comment-13252889
 ] 

Lars Hofhansl commented on HBASE-5775:
--

Thanks for the patch.

 ZKUtil doesn't handle deleteRecurisively cleanly
 

 Key: HBASE-5775
 URL: https://issues.apache.org/jira/browse/HBASE-5775
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: Jesse Yates
Assignee: Jesse Yates
 Fix For: 0.94.0, 0.96.0

 Attachments: java_HBASE-5775.patch


 ZKUtil.deleteNodeRecursively()'s contract says that it handles deletion of 
 the node and all its children. However, nothing is mentioned as to what 
 happens if the node you are attempting to delete doesn't actually exist. 
 Turns out, it throws a null pointer exception. I
 'm proposing that we change the code s.t. it handles the case where the 
 parent is already gone and exits cleanly, rather than failing horribly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5719) Enhance hbck to sideline overlapped mega regions

2012-04-12 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252890#comment-13252890
 ] 

Hudson commented on HBASE-5719:
---

Integrated in HBase-0.92 #369 (See 
[https://builds.apache.org/job/HBase-0.92/369/])
HBASE-5719 Enhance hbck to sideline overlapped mega regions (Jimmy Xiang) 
(Revision 1325405)

 Result = FAILURE
jmhsieh : 
Files : 
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java
* 
/hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/util/TestRegionSplitCalculator.java


 Enhance hbck to sideline overlapped mega regions
 

 Key: HBASE-5719
 URL: https://issues.apache.org/jira/browse/HBASE-5719
 Project: HBase
  Issue Type: New Feature
  Components: hbck
Affects Versions: 0.94.0, 0.96.0
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0

 Attachments: 5719.patch, 5719_0.90.patch, 5719_0.92.patch, 
 5719_0.94.patch, hbase-5719.patch, hbase-5719_0.90.patch, 
 hbase-5719_0.92.patch, hbase-5719_0.94.patch, hbase-5719_v3-new.patch, 
 hbase-5719_v3.patch


 If there are too many regions in one overlapped group (by default, more than 
 10), hbck currently doesn't merge them since it takes time.
 In this case, we can sideline some regions in the group and break the 
 overlapping to fix the inconsistency.  Later on, sidelined regions can be 
 bulk loaded manually.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5604) M/R tool to replay WAL files

2012-04-12 Thread Lars Hofhansl (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-5604:
-

Fix Version/s: 0.96.0
   0.94.0

 M/R tool to replay WAL files
 

 Key: HBASE-5604
 URL: https://issues.apache.org/jira/browse/HBASE-5604
 Project: HBase
  Issue Type: New Feature
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.94.0, 0.96.0

 Attachments: 5604-v10.txt, 5604-v11.txt, 5604-v4.txt, 5604-v6.txt, 
 5604-v7.txt, 5604-v8.txt, 5604-v9.txt, HLog-5604-v3.txt


 Just an idea I had. Might be useful for restore of a backup using the HLogs.
 This could an M/R (with a mapper per HLog file).
 The tool would get a timerange and a (set of) table(s). We'd pick the right 
 HLogs based on time before the M/R job is started and then have a mapper per 
 HLog file.
 The mapper would then go through the HLog, filter all WALEdits that didn't 
 fit into the time range or are not any of the tables and then uses 
 HFileOutputFormat to generate HFiles.
 Would need to indicate the splits we want, probably from a live table.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5300) Research website symolic links for renaming of HBase book to HBase Reference Guide

2012-04-12 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252894#comment-13252894
 ] 

Hudson commented on HBASE-5300:
---

Integrated in HBase-TRUNK #2747 (See 
[https://builds.apache.org/job/HBase-TRUNK/2747/])
HBASE-5300 Research website symolic links for renaming of HBase book to 
HBase Reference Guide (Revision 1325512)

 Result = SUCCESS
stack : 
Files : 
* /hbase/trunk/src/site/xdoc/acid-semantics.xml
* /hbase/trunk/src/site/xdoc/metrics.xml


 Research website symolic links for renaming of HBase book to HBase 
 Reference Guide
 --

 Key: HBASE-5300
 URL: https://issues.apache.org/jira/browse/HBASE-5300
 Project: HBase
  Issue Type: Task
Reporter: Doug Meil
Assignee: stack
 Fix For: 0.96.0

 Attachments: 5300.txt


 Creating this task per a conversation I had with stack last week.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5604) M/R tool to replay WAL files

2012-04-12 Thread Lars Hofhansl (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252892#comment-13252892
 ] 

Lars Hofhansl commented on HBASE-5604:
--

@Ted: Are you OK with latest patch? If so, I'll commit today.

 M/R tool to replay WAL files
 

 Key: HBASE-5604
 URL: https://issues.apache.org/jira/browse/HBASE-5604
 Project: HBase
  Issue Type: New Feature
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.94.0, 0.96.0

 Attachments: 5604-v10.txt, 5604-v11.txt, 5604-v4.txt, 5604-v6.txt, 
 5604-v7.txt, 5604-v8.txt, 5604-v9.txt, HLog-5604-v3.txt


 Just an idea I had. Might be useful for restore of a backup using the HLogs.
 This could an M/R (with a mapper per HLog file).
 The tool would get a timerange and a (set of) table(s). We'd pick the right 
 HLogs based on time before the M/R job is started and then have a mapper per 
 HLog file.
 The mapper would then go through the HLog, filter all WALEdits that didn't 
 fit into the time range or are not any of the tables and then uses 
 HFileOutputFormat to generate HFiles.
 Would need to indicate the splits we want, probably from a live table.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5776) HTableMultiplexer

2012-04-12 Thread Phabricator (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HBASE-5776:
---

Attachment: D2775.1.patch

Liyin requested code review of [jira][89-fb][HBASE-5776] HTableMultiplexer.
Reviewers: Kannan

  There is a known issue in HBase client that single slow/dead region server 
could slow down the multiput operations across all the region servers. So the 
HBase client will be as slow as the slowest region server in the cluster.

  To solve this problem, HTableMultiplexer will separate the multiput 
submitting threads with the flush threads, which means the multiput operation 
will be a nonblocking operation.
  The submitting thread will shard all the puts into different queues based on 
its destination region server and return immediately. The flush threads will 
flush these puts from each queue to its destination region server.

  Currently the HTableMultiplexer only supports the put operation.

TEST PLAN
  Unit tests

REVISION DETAIL
  https://reviews.facebook.net/D2775

AFFECTED FILES
  src/main/java/org/apache/hadoop/hbase/HConstants.java
  src/main/java/org/apache/hadoop/hbase/client/HConnection.java
  src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
  src/main/java/org/apache/hadoop/hbase/client/HTable.java
  src/main/java/org/apache/hadoop/hbase/client/HTableMultiplexer.java
  src/main/java/org/apache/hadoop/hbase/client/MultiPut.java
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
  src/test/java/org/apache/hadoop/hbase/client/TestHTableMultiplexer.java

MANAGE HERALD DIFFERENTIAL RULES
  https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
  https://reviews.facebook.net/herald/transcript/6303/

Tip: use the X-Herald-Rules header to filter Herald messages in your client.


 HTableMultiplexer 
 --

 Key: HBASE-5776
 URL: https://issues.apache.org/jira/browse/HBASE-5776
 Project: HBase
  Issue Type: Improvement
Reporter: Liyin Tang
Assignee: Liyin Tang
 Attachments: D2775.1.patch


 There is a known issue in HBase client that single slow/dead region server 
 could slow down the multiput operations across all the region servers. So the 
 HBase client will be as slow as the slowest region server in the cluster. 
  
 To solve this problem, HTableMultiplexer will separate the multiput 
 submitting threads with the flush threads, which means the multiput operation 
 will be a nonblocking operation. 
 The submitting thread will shard all the puts into different queues based on 
 its destination region server and return immediately. The flush threads will 
 flush these puts from each queue to its destination region server. 
 Currently the HTableMultiplexer only supports the put operation.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5777) MiniHBaseCluster cannot start multiple region servers

2012-04-12 Thread Jimmy Xiang (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252899#comment-13252899
 ] 

Jimmy Xiang commented on HBASE-5777:


I see.  When I run unit tests in eclipse, the hbase-site.xml at src/test is not 
used.
Maybe we can disable the UI in MiniHBaseCluster too, how about that?

 MiniHBaseCluster cannot start multiple region servers
 -

 Key: HBASE-5777
 URL: https://issues.apache.org/jira/browse/HBASE-5777
 Project: HBase
  Issue Type: Test
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Attachments: hbase-5777.patch


 MiniHBaseCluster can try to start multiple region servers.  But all of them 
 except one will die in putting up the web UI
 because of BindException since HConstants.REGIONSERVER_INFO_PORT_AUTO is set 
 to false by default.
 This issue will make many unit tests depending on multiple region servers 
 flaky, such as TestAdmin.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5776) HTableMultiplexer

2012-04-12 Thread Phabricator (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HBASE-5776:
---

Attachment: D2775.1.patch

Liyin requested code review of [jira][89-fb][HBASE-5776] HTableMultiplexer.
Reviewers: Kannan

  There is a known issue in HBase client that single slow/dead region server 
could slow down the multiput operations across all the region servers. So the 
HBase client will be as slow as the slowest region server in the cluster.

  To solve this problem, HTableMultiplexer will separate the multiput 
submitting threads with the flush threads, which means the multiput operation 
will be a nonblocking operation.
  The submitting thread will shard all the puts into different queues based on 
its destination region server and return immediately. The flush threads will 
flush these puts from each queue to its destination region server.

  Currently the HTableMultiplexer only supports the put operation.

TEST PLAN
  Unit tests

REVISION DETAIL
  https://reviews.facebook.net/D2775

AFFECTED FILES
  src/main/java/org/apache/hadoop/hbase/HConstants.java
  src/main/java/org/apache/hadoop/hbase/client/HConnection.java
  src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
  src/main/java/org/apache/hadoop/hbase/client/HTable.java
  src/main/java/org/apache/hadoop/hbase/client/HTableMultiplexer.java
  src/main/java/org/apache/hadoop/hbase/client/MultiPut.java
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
  src/test/java/org/apache/hadoop/hbase/client/TestHTableMultiplexer.java

MANAGE HERALD DIFFERENTIAL RULES
  https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
  https://reviews.facebook.net/herald/transcript/6303/

Tip: use the X-Herald-Rules header to filter Herald messages in your client.


 HTableMultiplexer 
 --

 Key: HBASE-5776
 URL: https://issues.apache.org/jira/browse/HBASE-5776
 Project: HBase
  Issue Type: Improvement
Reporter: Liyin Tang
Assignee: Liyin Tang
 Attachments: D2775.1.patch, D2775.1.patch


 There is a known issue in HBase client that single slow/dead region server 
 could slow down the multiput operations across all the region servers. So the 
 HBase client will be as slow as the slowest region server in the cluster. 
  
 To solve this problem, HTableMultiplexer will separate the multiput 
 submitting threads with the flush threads, which means the multiput operation 
 will be a nonblocking operation. 
 The submitting thread will shard all the puts into different queues based on 
 its destination region server and return immediately. The flush threads will 
 flush these puts from each queue to its destination region server. 
 Currently the HTableMultiplexer only supports the put operation.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5604) M/R tool to replay WAL files

2012-04-12 Thread Ted Yu (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252916#comment-13252916
 ] 

Ted Yu commented on HBASE-5604:
---

Patch looks good.
Minor comment:
{code}
+  HLog.Entry temp;
+  long i=-1;
{code}
Insert spaces around equal sign above.

 M/R tool to replay WAL files
 

 Key: HBASE-5604
 URL: https://issues.apache.org/jira/browse/HBASE-5604
 Project: HBase
  Issue Type: New Feature
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.94.0, 0.96.0

 Attachments: 5604-v10.txt, 5604-v11.txt, 5604-v4.txt, 5604-v6.txt, 
 5604-v7.txt, 5604-v8.txt, 5604-v9.txt, HLog-5604-v3.txt


 Just an idea I had. Might be useful for restore of a backup using the HLogs.
 This could an M/R (with a mapper per HLog file).
 The tool would get a timerange and a (set of) table(s). We'd pick the right 
 HLogs based on time before the M/R job is started and then have a mapper per 
 HLog file.
 The mapper would then go through the HLog, filter all WALEdits that didn't 
 fit into the time range or are not any of the tables and then uses 
 HFileOutputFormat to generate HFiles.
 Would need to indicate the splits we want, probably from a live table.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Created] (HBASE-5778) Turn on WAL compression by default

2012-04-12 Thread Jean-Daniel Cryans (Created) (JIRA)
Turn on WAL compression by default
--

 Key: HBASE-5778
 URL: https://issues.apache.org/jira/browse/HBASE-5778
 Project: HBase
  Issue Type: Improvement
Reporter: Jean-Daniel Cryans
Priority: Blocker
 Fix For: 0.94.0, 0.96.0


I ran some tests to verify if WAL compression should be turned on by default.

For a use case where it's not very useful (values two order of magnitude bigger 
than the keys), the insert time wasn't different and the CPU usage 15% higher 
(150% CPU usage VS 130% when not compressing the WAL).

When values are smaller than the keys, I saw a 38% improvement for the insert 
run time and CPU usage was 33% higher (600% CPU usage VS 450%). I'm not sure 
WAL compression accounts for all the additional CPU usage, it might just be 
that we're able to insert faster and we spend more time in the MemStore per 
second (because our MemStores are bad when they contain tens of thousands of 
values).

Those are two extremes, but it shows that for the price of some CPU we can save 
a lot. My machines have 2 quads with HT, so I still had a lot of idle CPUs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5778) Turn on WAL compression by default

2012-04-12 Thread Jean-Daniel Cryans (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jean-Daniel Cryans updated HBASE-5778:
--

Attachment: HBASE-5778.patch

I believe only this line is needed.

 Turn on WAL compression by default
 --

 Key: HBASE-5778
 URL: https://issues.apache.org/jira/browse/HBASE-5778
 Project: HBase
  Issue Type: Improvement
Reporter: Jean-Daniel Cryans
Priority: Blocker
 Fix For: 0.94.0, 0.96.0

 Attachments: HBASE-5778.patch


 I ran some tests to verify if WAL compression should be turned on by default.
 For a use case where it's not very useful (values two order of magnitude 
 bigger than the keys), the insert time wasn't different and the CPU usage 15% 
 higher (150% CPU usage VS 130% when not compressing the WAL).
 When values are smaller than the keys, I saw a 38% improvement for the insert 
 run time and CPU usage was 33% higher (600% CPU usage VS 450%). I'm not sure 
 WAL compression accounts for all the additional CPU usage, it might just be 
 that we're able to insert faster and we spend more time in the MemStore per 
 second (because our MemStores are bad when they contain tens of thousands of 
 values).
 Those are two extremes, but it shows that for the price of some CPU we can 
 save a lot. My machines have 2 quads with HT, so I still had a lot of idle 
 CPUs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5773) HtablePool constructor not reading config files in certain cases

2012-04-12 Thread Ioan Eugen Stan (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252921#comment-13252921
 ] 

Ioan Eugen Stan commented on HBASE-5773:


Hi Stack,

0.90 [1] branch has the same type of constructors. Am I looking in the wrong 
place?

https://svn.apache.org/repos/asf/hbase/branches/0.90/src/main/java/org/apache/hadoop/hbase/client/HTablePool.java
  


 HtablePool constructor not reading config files in certain cases
 

 Key: HBASE-5773
 URL: https://issues.apache.org/jira/browse/HBASE-5773
 Project: HBase
  Issue Type: Bug
  Components: client
Affects Versions: 0.90.6, 0.92.1, 0.94.1
Reporter: Ioan Eugen Stan
Priority: Minor
 Fix For: 0.92.2, 0.94.0

 Attachments: different-config-behaviour.patch


 Creating a HtablePool can issue two behaviour depanding on the constructor 
 called. 
 Case 1: loads the configs from hbase-site
   public HTablePool() {
 this(HBaseConfiguration.create(), Integer.MAX_VALUE);
   }
 Calling this with null values for Configuration: 
 public HTablePool(final Configuration config, final int maxSize) {
 this(config, maxSize, null, null);
   }
 will issue:
  public HTablePool(final Configuration config, final int maxSize,
   final HTableInterfaceFactory tableFactory, PoolType poolType) {
 // Make a new configuration instance so I can safely cleanup when
 // done with the pool.
 this.config = config == null ? new Configuration() : config;
 which does not read the hbase-site config files as 
 HBaseConfiguration.create() does. 
 I've tracked this problem to all versions of hbase. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5778) Turn on WAL compression by default

2012-04-12 Thread Todd Lipcon (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252924#comment-13252924
 ] 

Todd Lipcon commented on HBASE-5778:


Do we have this in hbase-default.xml as well? if not, +1

 Turn on WAL compression by default
 --

 Key: HBASE-5778
 URL: https://issues.apache.org/jira/browse/HBASE-5778
 Project: HBase
  Issue Type: Improvement
Reporter: Jean-Daniel Cryans
Priority: Blocker
 Fix For: 0.94.0, 0.96.0

 Attachments: HBASE-5778.patch


 I ran some tests to verify if WAL compression should be turned on by default.
 For a use case where it's not very useful (values two order of magnitude 
 bigger than the keys), the insert time wasn't different and the CPU usage 15% 
 higher (150% CPU usage VS 130% when not compressing the WAL).
 When values are smaller than the keys, I saw a 38% improvement for the insert 
 run time and CPU usage was 33% higher (600% CPU usage VS 450%). I'm not sure 
 WAL compression accounts for all the additional CPU usage, it might just be 
 that we're able to insert faster and we spend more time in the MemStore per 
 second (because our MemStores are bad when they contain tens of thousands of 
 values).
 Those are two extremes, but it shows that for the price of some CPU we can 
 save a lot. My machines have 2 quads with HT, so I still had a lot of idle 
 CPUs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5778) Turn on WAL compression by default

2012-04-12 Thread Jean-Daniel Cryans (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252927#comment-13252927
 ] 

Jean-Daniel Cryans commented on HBASE-5778:
---

It's not in there, do we want it since we turn it on? Or do we act like we 
always had it? :)

 Turn on WAL compression by default
 --

 Key: HBASE-5778
 URL: https://issues.apache.org/jira/browse/HBASE-5778
 Project: HBase
  Issue Type: Improvement
Reporter: Jean-Daniel Cryans
Priority: Blocker
 Fix For: 0.94.0, 0.96.0

 Attachments: HBASE-5778.patch


 I ran some tests to verify if WAL compression should be turned on by default.
 For a use case where it's not very useful (values two order of magnitude 
 bigger than the keys), the insert time wasn't different and the CPU usage 15% 
 higher (150% CPU usage VS 130% when not compressing the WAL).
 When values are smaller than the keys, I saw a 38% improvement for the insert 
 run time and CPU usage was 33% higher (600% CPU usage VS 450%). I'm not sure 
 WAL compression accounts for all the additional CPU usage, it might just be 
 that we're able to insert faster and we spend more time in the MemStore per 
 second (because our MemStores are bad when they contain tens of thousands of 
 values).
 Those are two extremes, but it shows that for the price of some CPU we can 
 save a lot. My machines have 2 quads with HT, so I still had a lot of idle 
 CPUs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




  1   2   >