date:20111011


 [ 
https://issues.apache.org/jira/browse/HBASE-4562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

bluedavy updated HBASE-4562:


Attachment: (was: HBASE-4562.patch)

 When split doing offlineParentInMeta occurs error,it'll cause data loss
 ---

 Key: HBASE-4562
 URL: https://issues.apache.org/jira/browse/HBASE-4562
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.90.4
Reporter: bluedavy
Priority: Blocker
 Fix For: 0.90.5

 Attachments: HBASE-4562-test.report.txt, HBASE-4562.patch


 Follow below steps to replay the problem:
 1. change the SplitTransaction.java as below,just like mock the timeout error.
{code:title=SplitTransaction.java|borderStyle=solid}
   if (!testing) {
 MetaEditor.offlineParentInMeta(server.getCatalogTracker(),
this.parent.getRegionInfo(), a.getRegionInfo(), b.getRegionInfo());
 throw new IOException(some unexpected error in split);
   }
{code} 
 2. update the regionserver code,restart;
 3. create a table  put some data to the table;
 4. split the table;
 5. kill the regionserver hosted the table;
 6. wait some time after master ServerShutdownHandler.process execute,then 
 scan the table,u'll find the data wrote before lost.
 We can fix the bug just use the patch.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4562) When split doing offlineParentInMeta occurs error,it'll cause data loss

2011-10-11 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-4562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

bluedavy updated HBASE-4562:


Attachment: HBASE-4562.patch

 When split doing offlineParentInMeta occurs error,it'll cause data loss
 ---

 Key: HBASE-4562
 URL: https://issues.apache.org/jira/browse/HBASE-4562
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.90.4
Reporter: bluedavy
Priority: Blocker
 Fix For: 0.90.5

 Attachments: HBASE-4562-test.report.txt, HBASE-4562.patch


 Follow below steps to replay the problem:
 1. change the SplitTransaction.java as below,just like mock the timeout error.
{code:title=SplitTransaction.java|borderStyle=solid}
   if (!testing) {
 MetaEditor.offlineParentInMeta(server.getCatalogTracker(),
this.parent.getRegionInfo(), a.getRegionInfo(), b.getRegionInfo());
 throw new IOException(some unexpected error in split);
   }
{code} 
 2. update the regionserver code,restart;
 3. create a table  put some data to the table;
 4. split the table;
 5. kill the regionserver hosted the table;
 6. wait some time after master ServerShutdownHandler.process execute,then 
 scan the table,u'll find the data wrote before lost.
 We can fix the bug just use the patch.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4536) Allow CF to retain deleted rows


[ 
https://issues.apache.org/jira/browse/HBASE-4536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13124723#comment-13124723
 ] 

jirapos...@reviews.apache.org commented on HBASE-4536:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2178/#review2501
---


Elegant solution.


http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java
https://reviews.apache.org/r/2178/#comment5660

Is this changed required by the feature ?



http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Scan.java
https://reviews.apache.org/r/2178/#comment5661

'the' is redundant.



http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
https://reviews.apache.org/r/2178/#comment5662

Timerange would be better than timetravel.



http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
https://reviews.apache.org/r/2178/#comment5663

Would EARLIEST_PUT_TS be a better name ?


- Ted


On 2011-10-11 04:21:07, Lars Hofhansl wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/2178/
bq.  ---
bq.  
bq.  (Updated 2011-10-11 04:21:07)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  HBase timerange Gets and Scans allow to do timetravel in HBase. I.e. 
look at the state of the data at any point in the past, provided the data is 
still around.
bq.  This did not work for deletes, however. Deletes would always mask all puts 
in the past.
bq.  This change adds a flag that can be on HColumnDescriptor to enable 
retention of deleted rows.
bq.  These rows are still subject to TTL and/or VERSIONS.
bq.  
bq.  This changes the following:
bq.  1. There is a new flag on HColumnDescriptor enabling that behavior.
bq.  2. Allow gets/scans with a timerange to retrieve rows hidden by a delete 
marker, if the timerange does not include the delete marker.
bq.  3. Do not unconditionally collect all deleted rows during a compaction.
bq.  4. Allow a raw Scan, which retrieves all delete markers and deleted rows.
bq.  
bq.  The change is small'ish, but the logic is intricate, so please review 
carefully.
bq.  
bq.  
bq.  This addresses bug HBASE-4536.
bq.  https://issues.apache.org/jira/browse/HBASE-4536
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java
 1181616 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Scan.java
 1181616 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ColumnTracker.java
 1181616 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ExplicitColumnTracker.java
 1181616 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
 1181616 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanWildcardColumnTracker.java
 1181616 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
 1181616 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
 1181616 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
 1181616 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestCase.java
 1181616 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestExplicitColumnTracker.java
 1181616 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestKeepDeletes.java
 PRE-CREATION 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestMinVersions.java
 1181616 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestScanWildcardColumnTracker.java
 1181616 
bq.  
bq.  Diff: https://reviews.apache.org/r/2178/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  All tests pass now.
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Lars
bq.  
bq.



 Allow CF to retain deleted rows
 ---

 Key: HBASE-4536
 URL:

[jira] [Assigned] (HBASE-4555) TestShell seems passed, but actually errors seen in test output file

2011-10-11 Thread Mingjie Lai (Assigned) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-4555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mingjie Lai reassigned HBASE-4555:
--

Assignee: Mingjie Lai

 TestShell seems passed, but actually errors seen in test output file
 

 Key: HBASE-4555
 URL: https://issues.apache.org/jira/browse/HBASE-4555
 Project: HBase
  Issue Type: Test
  Components: test
Reporter: Mingjie Lai
Assignee: Mingjie Lai
 Attachments: 4555.diff


 When I was making test cases for 4554, I saw a weird issue that TestShell 
 seems to pass, but actually I saw error messages in the output file.
 {code}
 ---
  T E S T S
 ---
 Running org.apache.hadoop.hbase.client.TestShell
 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 39.252 sec
 Results :
 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
 {code}
 Error messages in org.apache.hadoop.hbase.client.TestShell-output.txt:
 {code}
 ...
   6) Error:
 test_alter_should_support_shortcut_DELETE_alter_specs(Hbase::AdminAlterTableTest):
 ArgumentError: There should be at least one argument but the table name
 
 /home/mlai/git/hbase-private/src/test/ruby/../../main/ruby/hbase/admin.rb:307:in
  `alter'
 ./src/test/ruby/hbase/admin_test.rb:271:in 
 `test_alter_should_support_shortcut_DELETE_alter_specs'
 org/jruby/RubyProc.java:268:in `call'
 org/jruby/RubyKernel.java:2038:in `send'
 org/jruby/RubyArray.java:1572:in `each'
 org/jruby/RubyArray.java:1572:in `each'
 
   7) Error:
 test_split_should_work(Hbase::AdminMethodsTest):
 ArgumentError: wrong number of arguments (1 for 2)
 ./src/test/ruby/hbase/admin_test.rb:99:in `test_split_should_work'
 org/jruby/RubyProc.java:268:in `call'
 org/jruby/RubyKernel.java:2038:in `send'
 org/jruby/RubyArray.java:1572:in `each'
 org/jruby/RubyArray.java:1572:in `each'
 
 192 tests, 259 assertions, 1 failures, 6 errors
 Done with tests! Shutting down the cluster...
 2011-10-07 16:46:14,760 INFO  [main] hbase.HBaseTestingUtility(551): Shutting 
 down minicluster
 2011-10-07 16:46:14,760 DEBUG [main] util.JVMClusterUtil(214): Shutting down 
 HBase Cluster
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4555) TestShell seems passed, but actually errors seen in test output file

2011-10-11 Thread Mingjie Lai (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-4555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mingjie Lai updated HBASE-4555:
---

Attachment: 4555.diff

Patch 1.

 TestShell seems passed, but actually errors seen in test output file
 

 Key: HBASE-4555
 URL: https://issues.apache.org/jira/browse/HBASE-4555
 Project: HBase
  Issue Type: Test
  Components: test
Reporter: Mingjie Lai
Assignee: Mingjie Lai
 Attachments: 4555.diff


 When I was making test cases for 4554, I saw a weird issue that TestShell 
 seems to pass, but actually I saw error messages in the output file.
 {code}
 ---
  T E S T S
 ---
 Running org.apache.hadoop.hbase.client.TestShell
 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 39.252 sec
 Results :
 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
 {code}
 Error messages in org.apache.hadoop.hbase.client.TestShell-output.txt:
 {code}
 ...
   6) Error:
 test_alter_should_support_shortcut_DELETE_alter_specs(Hbase::AdminAlterTableTest):
 ArgumentError: There should be at least one argument but the table name
 
 /home/mlai/git/hbase-private/src/test/ruby/../../main/ruby/hbase/admin.rb:307:in
  `alter'
 ./src/test/ruby/hbase/admin_test.rb:271:in 
 `test_alter_should_support_shortcut_DELETE_alter_specs'
 org/jruby/RubyProc.java:268:in `call'
 org/jruby/RubyKernel.java:2038:in `send'
 org/jruby/RubyArray.java:1572:in `each'
 org/jruby/RubyArray.java:1572:in `each'
 
   7) Error:
 test_split_should_work(Hbase::AdminMethodsTest):
 ArgumentError: wrong number of arguments (1 for 2)
 ./src/test/ruby/hbase/admin_test.rb:99:in `test_split_should_work'
 org/jruby/RubyProc.java:268:in `call'
 org/jruby/RubyKernel.java:2038:in `send'
 org/jruby/RubyArray.java:1572:in `each'
 org/jruby/RubyArray.java:1572:in `each'
 
 192 tests, 259 assertions, 1 failures, 6 errors
 Done with tests! Shutting down the cluster...
 2011-10-07 16:46:14,760 INFO  [main] hbase.HBaseTestingUtility(551): Shutting 
 down minicluster
 2011-10-07 16:46:14,760 DEBUG [main] util.JVMClusterUtil(214): Shutting down 
 HBase Cluster
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4555) TestShell seems passed, but actually errors seen in test output file


[ 
https://issues.apache.org/jira/browse/HBASE-4555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13124729#comment-13124729
 ] 

Mingjie Lai commented on HBASE-4555:


HBase shell unit cases are in ruby. Ruby unit test runner 
Test::Unit::AutoRunner.run only prints unit tests assertion failures to console 
by default. So it actually swallows all the assertion errors from ruby, and 
junit cannot notice any shell test failure. 

I checked the source code of Test::Unit::AutoRunner.run, and it returns a 
boolean to indicate test result successfully or not. The attached patch just 
checks the return value and throw a runtime exception so TestShell can get to 
know there is something wrong by test errors. The drawback is that we won't see 
detailed failure info unless we check the output file. 

{code}
---
 T E S T S
---
Running org.apache.hadoop.hbase.client.TestShell
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 52.802 sec  
FAILURE!

Results :

Tests in error: 
  testRunShellTests(org.apache.hadoop.hbase.client.TestShell): (RuntimeError) 
Shell unit tests failed. Check output file for details.

Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
{code}

The perfect solution would be that ruby unit tests can pass test results to 
junit, so junit can display failures nicely.

After the patch, we will see TestShell failures we've never seen. And they need 
to be fixed. 



 TestShell seems passed, but actually errors seen in test output file
 

 Key: HBASE-4555
 URL: https://issues.apache.org/jira/browse/HBASE-4555
 Project: HBase
  Issue Type: Test
  Components: test
Reporter: Mingjie Lai
 Attachments: 4555.diff


 When I was making test cases for 4554, I saw a weird issue that TestShell 
 seems to pass, but actually I saw error messages in the output file.
 {code}
 ---
  T E S T S
 ---
 Running org.apache.hadoop.hbase.client.TestShell
 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 39.252 sec
 Results :
 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
 {code}
 Error messages in org.apache.hadoop.hbase.client.TestShell-output.txt:
 {code}
 ...
   6) Error:
 test_alter_should_support_shortcut_DELETE_alter_specs(Hbase::AdminAlterTableTest):
 ArgumentError: There should be at least one argument but the table name
 
 /home/mlai/git/hbase-private/src/test/ruby/../../main/ruby/hbase/admin.rb:307:in
  `alter'
 ./src/test/ruby/hbase/admin_test.rb:271:in 
 `test_alter_should_support_shortcut_DELETE_alter_specs'
 org/jruby/RubyProc.java:268:in `call'
 org/jruby/RubyKernel.java:2038:in `send'
 org/jruby/RubyArray.java:1572:in `each'
 org/jruby/RubyArray.java:1572:in `each'
 
   7) Error:
 test_split_should_work(Hbase::AdminMethodsTest):
 ArgumentError: wrong number of arguments (1 for 2)
 ./src/test/ruby/hbase/admin_test.rb:99:in `test_split_should_work'
 org/jruby/RubyProc.java:268:in `call'
 org/jruby/RubyKernel.java:2038:in `send'
 org/jruby/RubyArray.java:1572:in `each'
 org/jruby/RubyArray.java:1572:in `each'
 
 192 tests, 259 assertions, 1 failures, 6 errors
 Done with tests! Shutting down the cluster...
 2011-10-07 16:46:14,760 INFO  [main] hbase.HBaseTestingUtility(551): Shutting 
 down minicluster
 2011-10-07 16:46:14,760 DEBUG [main] util.JVMClusterUtil(214): Shutting down 
 HBase Cluster
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4335) Splits can create temporary holes in .META. that confuse clients and regionservers

2011-10-11 Thread jirapos...@reviews.apache.org (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13124728#comment-13124728
]

Lars Hofhansl commented on HBASE-4335:
--

Results :

Tests run: 1031, Failures: 0, Errors: 0, Skipped: 16

[INFO]
[INFO] BUILD SUCCESSFUL
[INFO]

Splits can create temporary holes in .META. that confuse clients and
regionservers
--

Key: HBASE-4335
URL: https://issues.apache.org/jira/browse/HBASE-4335
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.90.4
Reporter: Joe Pallas
Assignee: Lars Hofhansl
Priority: Critical
Fix For: 0.92.0

Attachments: 4335-v2.txt, 4335-v3.txt, 4335-v4.txt, 4335-v5.txt,
4335.txt

When a SplitTransaction is performed, three updates are done to .META.:
1. The parent region is marked as splitting (and hence offline)
2. The first daughter region is added (same start key as parent)
3. The second daughter region is added (split key is start key)
(later, the original parent region is deleted, but that's not important to
this discussion)
Steps 2 and 3 are actually done concurrently by
SplitTransaction.DaughterOpener threads. While the master is notified when a
split is complete, the only visibility that clients have is whether the
daughter regions have appeared in .META.
If the second daughter is added to .META. first, then .META. will contain the
(offline) parent region followed by the second daughter region. If the
client looks up a key that is greater than (or equal to) the split, the
client will find the second daughter region and use it. If the key is less
than the split key, the client will find the parent region and see that it is
offline, triggering a retry.
If the first daughter is added to .META. before the second daughter, there is
a window during which .META. has a hole: the first daughter effectively hides
the parent region (same start key), but there is no entry for the second
daughter. A region lookup will find the first daughter for all keys in the
parent's range, but the first daughter does not include keys at or beyond the
split key.
See HBASE-4333 and HBASE-4334 for details on how this causes problems and
suggestions for mitigating this in the client and regionserver.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4536) Allow CF to retain deleted rows


[ 
https://issues.apache.org/jira/browse/HBASE-4536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13124733#comment-13124733
 ] 

jirapos...@reviews.apache.org commented on HBASE-4536:
--



bq.  On 2011-10-11 06:02:38, Ted Yu wrote:
bq.   
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java,
 line 106
bq.   https://reviews.apache.org/r/2178/diff/6/?file=49393#file49393line106
bq.  
bq.   Timerange would be better than timetravel.

Agreed.


bq.  On 2011-10-11 06:02:38, Ted Yu wrote:
bq.   
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java,
 line 321
bq.   https://reviews.apache.org/r/2178/diff/6/?file=49389#file49389line321
bq.  
bq.   Is this changed required by the feature ?

No... But it didn't make sense before. Setting minVersions equal to maxVersions 
is the same as not having TTL.


bq.  On 2011-10-11 06:02:38, Ted Yu wrote:
bq.   
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java,
 line 127
bq.   https://reviews.apache.org/r/2178/diff/6/?file=49396#file49396line127
bq.  
bq.   Would EARLIEST_PUT_TS be a better name ?

Probably.


- Lars


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2178/#review2501
---


On 2011-10-11 04:21:07, Lars Hofhansl wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/2178/
bq.  ---
bq.  
bq.  (Updated 2011-10-11 04:21:07)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  HBase timerange Gets and Scans allow to do timetravel in HBase. I.e. 
look at the state of the data at any point in the past, provided the data is 
still around.
bq.  This did not work for deletes, however. Deletes would always mask all puts 
in the past.
bq.  This change adds a flag that can be on HColumnDescriptor to enable 
retention of deleted rows.
bq.  These rows are still subject to TTL and/or VERSIONS.
bq.  
bq.  This changes the following:
bq.  1. There is a new flag on HColumnDescriptor enabling that behavior.
bq.  2. Allow gets/scans with a timerange to retrieve rows hidden by a delete 
marker, if the timerange does not include the delete marker.
bq.  3. Do not unconditionally collect all deleted rows during a compaction.
bq.  4. Allow a raw Scan, which retrieves all delete markers and deleted rows.
bq.  
bq.  The change is small'ish, but the logic is intricate, so please review 
carefully.
bq.  
bq.  
bq.  This addresses bug HBASE-4536.
bq.  https://issues.apache.org/jira/browse/HBASE-4536
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java
 1181616 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Scan.java
 1181616 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ColumnTracker.java
 1181616 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ExplicitColumnTracker.java
 1181616 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
 1181616 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanWildcardColumnTracker.java
 1181616 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
 1181616 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
 1181616 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
 1181616 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestCase.java
 1181616 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestExplicitColumnTracker.java
 1181616 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestKeepDeletes.java
 PRE-CREATION 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestMinVersions.java
 1181616 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestScanWildcardColumnTracker.java
 1181616 
bq.  
bq.  Diff: https://reviews.apache.org/r/2178/diff
bq.  
bq.  
bq.  Testing

[jira] [Commented] (HBASE-4335) Splits can create temporary holes in .META. that confuse clients and regionservers

2011-10-11 Thread jirapos...@reviews.apache.org (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13124734#comment-13124734
]

Ted Yu commented on HBASE-4335:
---

Pass.
+1.

Splits can create temporary holes in .META. that confuse clients and
regionservers
--

Attachments: 4335-v2.txt, 4335-v3.txt, 4335-v4.txt, 4335-v5.txt,
4335.txt

[jira] [Commented] (HBASE-4536) Allow CF to retain deleted rows

[
https://issues.apache.org/jira/browse/HBASE-4536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13124735#comment-13124735
]

jirapos...@reviews.apache.org commented on HBASE-4536:
--

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2178/
---

(Updated 2011-10-11 06:20:01.186435)

Review request for hbase.

Changes
---

o Addressed Ted's point.
o Also added this flag to the shell.

Summary
---

HBase timerange Gets and Scans allow to do timetravel in HBase. I.e. look at
the state of the data at any point in the past, provided the data is still
around.
This did not work for deletes, however. Deletes would always mask all puts in
the past.
This change adds a flag that can be on HColumnDescriptor to enable retention of
deleted rows.
These rows are still subject to TTL and/or VERSIONS.

This changes the following:
1. There is a new flag on HColumnDescriptor enabling that behavior.
2. Allow gets/scans with a timerange to retrieve rows hidden by a delete
marker, if the timerange does not include the delete marker.
3. Do not unconditionally collect all deleted rows during a compaction.
4. Allow a raw Scan, which retrieves all delete markers and deleted rows.

The change is small'ish, but the logic is intricate, so please review carefully.

This addresses bug HBASE-4536.
https://issues.apache.org/jira/browse/HBASE-4536

Diffs (updated)
-

http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java
1181616

http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Scan.java
1181616

http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ColumnTracker.java
1181616

http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ExplicitColumnTracker.java
1181616

http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
1181616

http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanWildcardColumnTracker.java
1181616

http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
1181616

http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
1181616

http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
1181616
http://svn.apache.org/repos/asf/hbase/trunk/src/main/ruby/hbase/admin.rb
1181616

http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestCase.java
1181616

http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestExplicitColumnTracker.java
1181616

http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestKeepDeletes.java
PRE-CREATION

http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestMinVersions.java
1181616

http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestScanWildcardColumnTracker.java
1181616

Diff: https://reviews.apache.org/r/2178/diff

Testing
---

All tests pass now.

Thanks,

Lars

Allow CF to retain deleted rows
---

Key: HBASE-4536
URL: https://issues.apache.org/jira/browse/HBASE-4536
Project: HBase
Issue Type: Sub-task
Components: regionserver
Affects Versions: 0.92.0
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Fix For: 0.94.0

Parent allows for a cluster to retain rows for a TTL or keep a minimum number
of versions.
However, if a client deletes a row all version older than the delete tomb
stone will be remove at the next major compaction (and even at memstore flush
- see HBASE-4241).
There should be a way to retain those version to guard against software error.
I see two options here:
1. Add a new flag HColumnDescriptor. Something like RETAIN_DELETED.
2. Folds this into the parent change. I.e. keep minimum-number-of-versions of
versions even past the delete marker.
#1 would allow for more flexibility. #2 comes somewhat naturally with parent
(from a user viewpoint)
Comments? Any other options?

[jira] [Commented] (HBASE-4536) Allow CF to retain deleted rows

2011-10-11 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13124736#comment-13124736
 ] 

jirapos...@reviews.apache.org commented on HBASE-4536:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2178/#review2503
---



http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java
https://reviews.apache.org/r/2178/#comment5667

So minVersions == maxVersions == 1 would no longer be accepted ...


- Ted


On 2011-10-11 06:20:01, Lars Hofhansl wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/2178/
bq.  ---
bq.  
bq.  (Updated 2011-10-11 06:20:01)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  HBase timerange Gets and Scans allow to do timetravel in HBase. I.e. 
look at the state of the data at any point in the past, provided the data is 
still around.
bq.  This did not work for deletes, however. Deletes would always mask all puts 
in the past.
bq.  This change adds a flag that can be on HColumnDescriptor to enable 
retention of deleted rows.
bq.  These rows are still subject to TTL and/or VERSIONS.
bq.  
bq.  This changes the following:
bq.  1. There is a new flag on HColumnDescriptor enabling that behavior.
bq.  2. Allow gets/scans with a timerange to retrieve rows hidden by a delete 
marker, if the timerange does not include the delete marker.
bq.  3. Do not unconditionally collect all deleted rows during a compaction.
bq.  4. Allow a raw Scan, which retrieves all delete markers and deleted rows.
bq.  
bq.  The change is small'ish, but the logic is intricate, so please review 
carefully.
bq.  
bq.  
bq.  This addresses bug HBASE-4536.
bq.  https://issues.apache.org/jira/browse/HBASE-4536
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java
 1181616 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Scan.java
 1181616 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ColumnTracker.java
 1181616 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ExplicitColumnTracker.java
 1181616 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
 1181616 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanWildcardColumnTracker.java
 1181616 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
 1181616 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
 1181616 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
 1181616 
bq.http://svn.apache.org/repos/asf/hbase/trunk/src/main/ruby/hbase/admin.rb 
1181616 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestCase.java
 1181616 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestExplicitColumnTracker.java
 1181616 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestKeepDeletes.java
 PRE-CREATION 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestMinVersions.java
 1181616 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestScanWildcardColumnTracker.java
 1181616 
bq.  
bq.  Diff: https://reviews.apache.org/r/2178/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  All tests pass now.
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Lars
bq.  
bq.



 Allow CF to retain deleted rows
 ---

 Key: HBASE-4536
 URL: https://issues.apache.org/jira/browse/HBASE-4536
 Project: HBase
  Issue Type: Sub-task
  Components: regionserver
Affects Versions: 0.92.0
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.94.0


 Parent allows for a cluster to retain rows for a TTL or keep a minimum number 
 of versions.
 However, if a client deletes a row all version older than the delete tomb 
 stone will be remove at the next major compaction (and

[jira] [Commented] (HBASE-4555) TestShell seems passed, but actually errors seen in test output file


[ 
https://issues.apache.org/jira/browse/HBASE-4555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13124739#comment-13124739
 ] 

Ted Yu commented on HBASE-4555:
---

Interesting.
Next step would be fixing the failures now that TestShell consistently hangs on 
Jenkins.
Actually I don't see its output file here:
https://builds.apache.org/view/G-L/view/HBase/job/HBase-TRUNK/2314/artifact/trunk/target/surefire-reports/
and org.apache.hadoop.hbase.client.TestShell.txt is empty :-(

 TestShell seems passed, but actually errors seen in test output file
 

 Key: HBASE-4555
 URL: https://issues.apache.org/jira/browse/HBASE-4555
 Project: HBase
  Issue Type: Test
  Components: test
Reporter: Mingjie Lai
Assignee: Mingjie Lai
 Attachments: 4555.diff


 When I was making test cases for 4554, I saw a weird issue that TestShell 
 seems to pass, but actually I saw error messages in the output file.
 {code}
 ---
  T E S T S
 ---
 Running org.apache.hadoop.hbase.client.TestShell
 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 39.252 sec
 Results :
 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
 {code}
 Error messages in org.apache.hadoop.hbase.client.TestShell-output.txt:
 {code}
 ...
   6) Error:
 test_alter_should_support_shortcut_DELETE_alter_specs(Hbase::AdminAlterTableTest):
 ArgumentError: There should be at least one argument but the table name
 
 /home/mlai/git/hbase-private/src/test/ruby/../../main/ruby/hbase/admin.rb:307:in
  `alter'
 ./src/test/ruby/hbase/admin_test.rb:271:in 
 `test_alter_should_support_shortcut_DELETE_alter_specs'
 org/jruby/RubyProc.java:268:in `call'
 org/jruby/RubyKernel.java:2038:in `send'
 org/jruby/RubyArray.java:1572:in `each'
 org/jruby/RubyArray.java:1572:in `each'
 
   7) Error:
 test_split_should_work(Hbase::AdminMethodsTest):
 ArgumentError: wrong number of arguments (1 for 2)
 ./src/test/ruby/hbase/admin_test.rb:99:in `test_split_should_work'
 org/jruby/RubyProc.java:268:in `call'
 org/jruby/RubyKernel.java:2038:in `send'
 org/jruby/RubyArray.java:1572:in `each'
 org/jruby/RubyArray.java:1572:in `each'
 
 192 tests, 259 assertions, 1 failures, 6 errors
 Done with tests! Shutting down the cluster...
 2011-10-07 16:46:14,760 INFO  [main] hbase.HBaseTestingUtility(551): Shutting 
 down minicluster
 2011-10-07 16:46:14,760 DEBUG [main] util.JVMClusterUtil(214): Shutting down 
 HBase Cluster
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4562) When split doing offlineParentInMeta occurs error,it'll cause data loss


 [ 
https://issues.apache.org/jira/browse/HBASE-4562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

bluedavy updated HBASE-4562:


Attachment: HBASE-4562fortrunk.patch
HBASE-4562for0.92.patch

 When split doing offlineParentInMeta occurs error,it'll cause data loss
 ---

 Key: HBASE-4562
 URL: https://issues.apache.org/jira/browse/HBASE-4562
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.90.4
Reporter: bluedavy
Priority: Blocker
 Fix For: 0.90.5

 Attachments: HBASE-4562-test.report.txt, HBASE-4562.patch, 
 HBASE-4562for0.92.patch, HBASE-4562fortrunk.patch


 Follow below steps to replay the problem:
 1. change the SplitTransaction.java as below,just like mock the timeout error.
{code:title=SplitTransaction.java|borderStyle=solid}
   if (!testing) {
 MetaEditor.offlineParentInMeta(server.getCatalogTracker(),
this.parent.getRegionInfo(), a.getRegionInfo(), b.getRegionInfo());
 throw new IOException(some unexpected error in split);
   }
{code} 
 2. update the regionserver code,restart;
 3. create a table  put some data to the table;
 4. split the table;
 5. kill the regionserver hosted the table;
 6. wait some time after master ServerShutdownHandler.process execute,then 
 scan the table,u'll find the data wrote before lost.
 We can fix the bug just use the patch.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4563) When split doing this.parent.close(false) occurs error,it'll cause the splited region cann't write read


 [ 
https://issues.apache.org/jira/browse/HBASE-4563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

bluedavy updated HBASE-4563:


Attachment: HBASE-4563fortrunk.patch
HBASE-4563for0.92.patch

 When split doing this.parent.close(false) occurs error,it'll cause the 
 splited region cann't write  read
 -

 Key: HBASE-4563
 URL: https://issues.apache.org/jira/browse/HBASE-4563
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.90.4, 0.92.0
Reporter: bluedavy
Priority: Blocker
 Fix For: 0.90.5

 Attachments: HBASE-4563-test.report.txt, HBASE-4563.patch, 
 HBASE-4563for0.92.patch, HBASE-4563fortrunk.patch


 Follow below steps to replay the problem:
 1. change the SplitTransaction.java as below,just like mock the hdfs error.
{code:title=SplitTransaction.java|borderStyle=solid}
   ListStoreFile hstoreFilesToSplit = this.parent.close(false);
   throw new IOException(some unexpected error in close store files);
{code} 
 2. update the regionserver code,restart;
 3. create a table  put some data to the table;
 4. split the table;
 5. scan the table,then it'll fail.
 We can fix the bug just use the patch.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4563) When split doing this.parent.close(false) occurs error,it'll cause the splited region cann't write read


 [ 
https://issues.apache.org/jira/browse/HBASE-4563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

bluedavy updated HBASE-4563:


Affects Version/s: 0.92.0

 When split doing this.parent.close(false) occurs error,it'll cause the 
 splited region cann't write  read
 -

 Key: HBASE-4563
 URL: https://issues.apache.org/jira/browse/HBASE-4563
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.90.4, 0.92.0
Reporter: bluedavy
Priority: Blocker
 Fix For: 0.90.5

 Attachments: HBASE-4563-test.report.txt, HBASE-4563.patch, 
 HBASE-4563for0.92.patch, HBASE-4563fortrunk.patch


 Follow below steps to replay the problem:
 1. change the SplitTransaction.java as below,just like mock the hdfs error.
{code:title=SplitTransaction.java|borderStyle=solid}
   ListStoreFile hstoreFilesToSplit = this.parent.close(false);
   throw new IOException(some unexpected error in close store files);
{code} 
 2. update the regionserver code,restart;
 3. create a table  put some data to the table;
 4. split the table;
 5. scan the table,then it'll fail.
 We can fix the bug just use the patch.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4562) When split doing offlineParentInMeta occurs error,it'll cause data loss


[ 
https://issues.apache.org/jira/browse/HBASE-4562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13124742#comment-13124742
 ] 

Ted Yu commented on HBASE-4562:
---

Please run patch for 0.92 or TRUNK (hopefully both) through unit tests.
Thanks

 When split doing offlineParentInMeta occurs error,it'll cause data loss
 ---

 Key: HBASE-4562
 URL: https://issues.apache.org/jira/browse/HBASE-4562
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.90.4
Reporter: bluedavy
Priority: Blocker
 Fix For: 0.90.5

 Attachments: HBASE-4562-test.report.txt, HBASE-4562.patch, 
 HBASE-4562for0.92.patch, HBASE-4562fortrunk.patch


 Follow below steps to replay the problem:
 1. change the SplitTransaction.java as below,just like mock the timeout error.
{code:title=SplitTransaction.java|borderStyle=solid}
   if (!testing) {
 MetaEditor.offlineParentInMeta(server.getCatalogTracker(),
this.parent.getRegionInfo(), a.getRegionInfo(), b.getRegionInfo());
 throw new IOException(some unexpected error in split);
   }
{code} 
 2. update the regionserver code,restart;
 3. create a table  put some data to the table;
 4. split the table;
 5. kill the regionserver hosted the table;
 6. wait some time after master ServerShutdownHandler.process execute,then 
 scan the table,u'll find the data wrote before lost.
 We can fix the bug just use the patch.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4562) When split doing offlineParentInMeta encounters error, it'll cause data loss

2011-10-11 Thread Ted Yu (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-4562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-4562:
--

Summary: When split doing offlineParentInMeta encounters error, it'll cause 
data loss  (was: When split doing offlineParentInMeta occurs error,it'll cause 
data loss)

 When split doing offlineParentInMeta encounters error, it'll cause data loss
 

 Key: HBASE-4562
 URL: https://issues.apache.org/jira/browse/HBASE-4562
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.90.4
Reporter: bluedavy
Priority: Blocker
 Fix For: 0.90.5

 Attachments: HBASE-4562-test.report.txt, HBASE-4562.patch, 
 HBASE-4562for0.92.patch, HBASE-4562fortrunk.patch


 Follow below steps to replay the problem:
 1. change the SplitTransaction.java as below,just like mock the timeout error.
{code:title=SplitTransaction.java|borderStyle=solid}
   if (!testing) {
 MetaEditor.offlineParentInMeta(server.getCatalogTracker(),
this.parent.getRegionInfo(), a.getRegionInfo(), b.getRegionInfo());
 throw new IOException(some unexpected error in split);
   }
{code} 
 2. update the regionserver code,restart;
 3. create a table  put some data to the table;
 4. split the table;
 5. kill the regionserver hosted the table;
 6. wait some time after master ServerShutdownHandler.process execute,then 
 scan the table,u'll find the data wrote before lost.
 We can fix the bug just use the patch.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4556) Fix all incorrect uses of InternalScanner.next(...)


[ 
https://issues.apache.org/jira/browse/HBASE-4556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13124745#comment-13124745
 ] 

Lars Hofhansl commented on HBASE-4556:
--

I think we could use some tests that (somehow) simulate InternalScanners or 
RegionScanners, where next returns false while still delivering KVs in the 
current iteration.

 Fix all incorrect uses of InternalScanner.next(...)
 ---

 Key: HBASE-4556
 URL: https://issues.apache.org/jira/browse/HBASE-4556
 Project: HBase
  Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl

 There are cases all over the code where InternalScanner.next(...) is not used 
 correctly.
 I see this a lot:
 {code}
 while(scanner.next(...)) {
 }
 {code}
 The correct pattern is:
 {code}
 boolean more = false;
 do {
more = scanner.next(...);
 } while (more);
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4563) When split doing this.parent.close(false) occurs error,it'll cause the splited region cann't write read


[ 
https://issues.apache.org/jira/browse/HBASE-4563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13124751#comment-13124751
 ] 

Ted Yu commented on HBASE-4563:
---

Please run through unit tests for 0.92 patch.

 When split doing this.parent.close(false) occurs error,it'll cause the 
 splited region cann't write  read
 -

 Key: HBASE-4563
 URL: https://issues.apache.org/jira/browse/HBASE-4563
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.90.4, 0.92.0
Reporter: bluedavy
Priority: Blocker
 Fix For: 0.90.5

 Attachments: HBASE-4563-test.report.txt, HBASE-4563.patch, 
 HBASE-4563for0.92.patch, HBASE-4563fortrunk.patch


 Follow below steps to replay the problem:
 1. change the SplitTransaction.java as below,just like mock the hdfs error.
{code:title=SplitTransaction.java|borderStyle=solid}
   ListStoreFile hstoreFilesToSplit = this.parent.close(false);
   throw new IOException(some unexpected error in close store files);
{code} 
 2. update the regionserver code,restart;
 3. create a table  put some data to the table;
 4. split the table;
 5. scan the table,then it'll fail.
 We can fix the bug just use the patch.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4536) Allow CF to retain deleted rows

[
https://issues.apache.org/jira/browse/HBASE-4536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13124753#comment-13124753
]

Ted Yu commented on HBASE-4536:
---

For the case where minVersions == maxVersions == n, since we know the
semantics, shall we change the values for minVersions and log our decision
instead of throwing IllegalArgumentException ?

Allow CF to retain deleted rows
---

[jira] [Commented] (HBASE-4489) Better key splitting in RegionSplitter


[ 
https://issues.apache.org/jira/browse/HBASE-4489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13124756#comment-13124756
 ] 

Ted Yu commented on HBASE-4489:
---

Please add license to TestRegionSplitter.java

 Better key splitting in RegionSplitter
 --

 Key: HBASE-4489
 URL: https://issues.apache.org/jira/browse/HBASE-4489
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.90.4
Reporter: Dave Revell
Assignee: Dave Revell
 Attachments: HBASE-4489-branch0.90-v1.patch, 
 HBASE-4489-branch0.90-v2.patch, HBASE-4489-trunk-v1.patch, 
 HBASE-4489-trunk-v2.patch


 The RegionSplitter utility allows users to create a pre-split table from the 
 command line or do a rolling split on an existing table. It supports 
 pluggable split algorithms that implement the SplitAlgorithm interface. The 
 only/default SplitAlgorithm is one that assumes keys fall in the range from 
 ASCII string  to ASCII string 7FFF. This is not a sane 
 default, and seems useless to most users. Users are likely to be surprised by 
 the fact that all the region splits occur in in the byte range of ASCII 
 characters.
 A better default split algorithm would be one that evenly divides the space 
 of all bytes, which is what this patch does. Making a table with five regions 
 would split at \x33\x33..., \x66\x66, \x99\x99..., \xCC\xCC..., and 
 \xFF\xFF.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4555) TestShell seems passed, but actually errors seen in test output file


[ 
https://issues.apache.org/jira/browse/HBASE-4555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13124759#comment-13124759
 ] 

Mingjie Lai commented on HBASE-4555:


@ted. I checked an earlier build 2310 and you can see the errors:

https://builds.apache.org/view/G-L/view/HBase/job/HBase-TRUNK/2310/artifact/trunk/target/surefire-reports/org.apache.hadoop.hbase.client.TestShell-output.txt

 TestShell seems passed, but actually errors seen in test output file
 

 Key: HBASE-4555
 URL: https://issues.apache.org/jira/browse/HBASE-4555
 Project: HBase
  Issue Type: Test
  Components: test
Reporter: Mingjie Lai
Assignee: Mingjie Lai
 Attachments: 4555.diff


 When I was making test cases for 4554, I saw a weird issue that TestShell 
 seems to pass, but actually I saw error messages in the output file.
 {code}
 ---
  T E S T S
 ---
 Running org.apache.hadoop.hbase.client.TestShell
 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 39.252 sec
 Results :
 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
 {code}
 Error messages in org.apache.hadoop.hbase.client.TestShell-output.txt:
 {code}
 ...
   6) Error:
 test_alter_should_support_shortcut_DELETE_alter_specs(Hbase::AdminAlterTableTest):
 ArgumentError: There should be at least one argument but the table name
 
 /home/mlai/git/hbase-private/src/test/ruby/../../main/ruby/hbase/admin.rb:307:in
  `alter'
 ./src/test/ruby/hbase/admin_test.rb:271:in 
 `test_alter_should_support_shortcut_DELETE_alter_specs'
 org/jruby/RubyProc.java:268:in `call'
 org/jruby/RubyKernel.java:2038:in `send'
 org/jruby/RubyArray.java:1572:in `each'
 org/jruby/RubyArray.java:1572:in `each'
 
   7) Error:
 test_split_should_work(Hbase::AdminMethodsTest):
 ArgumentError: wrong number of arguments (1 for 2)
 ./src/test/ruby/hbase/admin_test.rb:99:in `test_split_should_work'
 org/jruby/RubyProc.java:268:in `call'
 org/jruby/RubyKernel.java:2038:in `send'
 org/jruby/RubyArray.java:1572:in `each'
 org/jruby/RubyArray.java:1572:in `each'
 
 192 tests, 259 assertions, 1 failures, 6 errors
 Done with tests! Shutting down the cluster...
 2011-10-07 16:46:14,760 INFO  [main] hbase.HBaseTestingUtility(551): Shutting 
 down minicluster
 2011-10-07 16:46:14,760 DEBUG [main] util.JVMClusterUtil(214): Shutting down 
 HBase Cluster
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4536) Allow CF to retain deleted rows

[
https://issues.apache.org/jira/browse/HBASE-4536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13124768#comment-13124768
]

Lars Hofhansl commented on HBASE-4536:
--

You mean set TTL to FOREVER and minVersions to 0? Sure that would work.

On the other hand it indicates that the user did not understand how minVersions
works, so I could see an error as well.

Allow CF to retain deleted rows
---

[jira] [Commented] (HBASE-4555) TestShell seems passed, but actually errors seen in test output file


[ 
https://issues.apache.org/jira/browse/HBASE-4555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13124770#comment-13124770
 ] 

Ted Yu commented on HBASE-4555:
---

Thanks for the finding, Mingjie.
I am afraid the symptom for builds 2314 and beyond is different.
We'd better reproduce the hanging TestShell and see where it gets stuck.

 TestShell seems passed, but actually errors seen in test output file
 

 Key: HBASE-4555
 URL: https://issues.apache.org/jira/browse/HBASE-4555
 Project: HBase
  Issue Type: Test
  Components: test
Reporter: Mingjie Lai
Assignee: Mingjie Lai
 Attachments: 4555.diff


 When I was making test cases for 4554, I saw a weird issue that TestShell 
 seems to pass, but actually I saw error messages in the output file.
 {code}
 ---
  T E S T S
 ---
 Running org.apache.hadoop.hbase.client.TestShell
 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 39.252 sec
 Results :
 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
 {code}
 Error messages in org.apache.hadoop.hbase.client.TestShell-output.txt:
 {code}
 ...
   6) Error:
 test_alter_should_support_shortcut_DELETE_alter_specs(Hbase::AdminAlterTableTest):
 ArgumentError: There should be at least one argument but the table name
 
 /home/mlai/git/hbase-private/src/test/ruby/../../main/ruby/hbase/admin.rb:307:in
  `alter'
 ./src/test/ruby/hbase/admin_test.rb:271:in 
 `test_alter_should_support_shortcut_DELETE_alter_specs'
 org/jruby/RubyProc.java:268:in `call'
 org/jruby/RubyKernel.java:2038:in `send'
 org/jruby/RubyArray.java:1572:in `each'
 org/jruby/RubyArray.java:1572:in `each'
 
   7) Error:
 test_split_should_work(Hbase::AdminMethodsTest):
 ArgumentError: wrong number of arguments (1 for 2)
 ./src/test/ruby/hbase/admin_test.rb:99:in `test_split_should_work'
 org/jruby/RubyProc.java:268:in `call'
 org/jruby/RubyKernel.java:2038:in `send'
 org/jruby/RubyArray.java:1572:in `each'
 org/jruby/RubyArray.java:1572:in `each'
 
 192 tests, 259 assertions, 1 failures, 6 errors
 Done with tests! Shutting down the cluster...
 2011-10-07 16:46:14,760 INFO  [main] hbase.HBaseTestingUtility(551): Shutting 
 down minicluster
 2011-10-07 16:46:14,760 DEBUG [main] util.JVMClusterUtil(214): Shutting down 
 HBase Cluster
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4536) Allow CF to retain deleted rows

[
https://issues.apache.org/jira/browse/HBASE-4536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13124773#comment-13124773
]

Ted Yu commented on HBASE-4536:
---

Personally I would love the software which can interpret my expression
correctly even though I don't fully know how it works :-)
It's your call what to do here, Lars.

Allow CF to retain deleted rows
---

[jira] [Commented] (HBASE-4555) TestShell seems passed, but actually errors seen in test output file

2011-10-11 Thread ramkrishna.s.vasudevan (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13124774#comment-13124774
 ] 

Mingjie Lai commented on HBASE-4555:


I agree. Hanging TestShell should be another issue. This one is about 
discovering hidden TestShell assertion failures. 

 TestShell seems passed, but actually errors seen in test output file
 

 Key: HBASE-4555
 URL: https://issues.apache.org/jira/browse/HBASE-4555
 Project: HBase
  Issue Type: Test
  Components: test
Reporter: Mingjie Lai
Assignee: Mingjie Lai
 Attachments: 4555.diff


 When I was making test cases for 4554, I saw a weird issue that TestShell 
 seems to pass, but actually I saw error messages in the output file.
 {code}
 ---
  T E S T S
 ---
 Running org.apache.hadoop.hbase.client.TestShell
 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 39.252 sec
 Results :
 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
 {code}
 Error messages in org.apache.hadoop.hbase.client.TestShell-output.txt:
 {code}
 ...
   6) Error:
 test_alter_should_support_shortcut_DELETE_alter_specs(Hbase::AdminAlterTableTest):
 ArgumentError: There should be at least one argument but the table name
 
 /home/mlai/git/hbase-private/src/test/ruby/../../main/ruby/hbase/admin.rb:307:in
  `alter'
 ./src/test/ruby/hbase/admin_test.rb:271:in 
 `test_alter_should_support_shortcut_DELETE_alter_specs'
 org/jruby/RubyProc.java:268:in `call'
 org/jruby/RubyKernel.java:2038:in `send'
 org/jruby/RubyArray.java:1572:in `each'
 org/jruby/RubyArray.java:1572:in `each'
 
   7) Error:
 test_split_should_work(Hbase::AdminMethodsTest):
 ArgumentError: wrong number of arguments (1 for 2)
 ./src/test/ruby/hbase/admin_test.rb:99:in `test_split_should_work'
 org/jruby/RubyProc.java:268:in `call'
 org/jruby/RubyKernel.java:2038:in `send'
 org/jruby/RubyArray.java:1572:in `each'
 org/jruby/RubyArray.java:1572:in `each'
 
 192 tests, 259 assertions, 1 failures, 6 errors
 Done with tests! Shutting down the cluster...
 2011-10-07 16:46:14,760 INFO  [main] hbase.HBaseTestingUtility(551): Shutting 
 down minicluster
 2011-10-07 16:46:14,760 DEBUG [main] util.JVMClusterUtil(214): Shutting down 
 HBase Cluster
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4335) Splits can create temporary holes in .META. that confuse clients and regionservers

2011-10-11 Thread Jonathan Hsieh (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13125037#comment-13125037
]

Jonathan Hsieh commented on HBASE-4335:
---

lgtm

Splits can create temporary holes in .META. that confuse clients and
regionservers
--

Attachments: 4335-v2.txt, 4335-v3.txt, 4335-v4.txt, 4335-v5.txt,
4335.txt

[jira] [Updated] (HBASE-1744) Thrift server to match the new java api.

2011-10-11 Thread Bob Copeland (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bob Copeland updated HBASE-1744:


Attachment: HBASE-1744.8.patch

Sure -- v8 is v7 + the TDeleteType addition.

 Thrift server to match the new java api.
 

 Key: HBASE-1744
 URL: https://issues.apache.org/jira/browse/HBASE-1744
 Project: HBase
  Issue Type: Improvement
  Components: thrift
Reporter: Tim Sell
Assignee: Tim Sell
Priority: Critical
 Fix For: 0.94.0

 Attachments: 
 0001-thrift2-enable-usage-of-.deleteColumns-for-thrift.patch, 
 HBASE-1744.2.patch, HBASE-1744.3.patch, HBASE-1744.4.patch, 
 HBASE-1744.5.patch, HBASE-1744.6.patch, HBASE-1744.7.patch, 
 HBASE-1744.8.patch, HBASE-1744.preview.1.patch, thriftexperiment.patch


 This mutateRows, etc.. is a little confusing compared to the new cleaner java 
 client.
 Thinking of ways to make a thrift client that is just as elegant. something 
 like:
 void put(1:Bytes table, 2:TPut put) throws (1:IOError io)
 with:
 struct TColumn {
   1:Bytes family,
   2:Bytes qualifier,
   3:i64 timestamp
 }
 struct TPut {
   1:Bytes row,
   2:mapTColumn, Bytes values
 }
 This creates more verbose rpc  than if the columns in TPut were just 
 mapBytes, mapBytes, Bytes, but that is harder to fit timestamps into and 
 still be intuitive from say python.
 Presumably the goal of a thrift gateway is to be easy first.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4335) Splits can create temporary holes in .META. that confuse clients and regionservers

[
https://issues.apache.org/jira/browse/HBASE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13125061#comment-13125061
]

ramkrishna.s.vasudevan commented on HBASE-4335:
---

+1. Your test case comments looks good. May be you can also take a look at
HBASE-4562 and HBASE-4563 as it is related to Splits.

Splits can create temporary holes in .META. that confuse clients and
regionservers
--

Attachments: 4335-v2.txt, 4335-v3.txt, 4335-v4.txt, 4335-v5.txt,
4335.txt

[jira] [Commented] (HBASE-4489) Better key splitting in RegionSplitter

2011-10-11 Thread Jonathan Hsieh (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-4489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13125118#comment-13125118
]

Jonathan Hsieh commented on HBASE-4489:
---

@Dave

Part of me really just would prefer decouple rollingSplit from the presplit
min/max value selection -- maybe change this in to two programs -- a custom
presplit table generator program that handles key bounds, and a separate
rollingSplit program that just splits based on given key ranges.

I thought that there was agreement that we would keep MD5StringSplit as default
for 0.90. It looks like the default was changed to UniformSplit from
MD5StringSplit in both patches. While I generally agree with your point #3,
it is a in 0.90 and would be a compatibility problem for anyone who depends on
it. Would it make sense to change the default in trunk/0.92 (I'm fine with
that) but leave 0.90.x as is?

Nice functional test. Did you consider just doing a unit test on the split
algorithm along with the cluster spinning functional test? I believe
HBaseAdmin.create(HTableDescriptor htd,byte startKeys[][]) is well tested and
would make the non @Ignored portions quicker. I can see how you need this
setup for testing rollingSplit.

Interesting div 0 bug. More testing, less surprises!

Any reason why in testCreatePressplitTable you go to -0x71, 0x81 .. -0x11
instead of just going to 0x8f, 0x9f .. 0xff? Though more verbose, I think it
is easier to read and follow if you use positive hex and cast all of them
with (byte), or write out single longs and convert?

Better key splitting in RegionSplitter
--

Key: HBASE-4489
URL: https://issues.apache.org/jira/browse/HBASE-4489
Project: HBase
Issue Type: Improvement
Affects Versions: 0.90.4
Reporter: Dave Revell
Assignee: Dave Revell
Attachments: HBASE-4489-branch0.90-v1.patch,
HBASE-4489-branch0.90-v2.patch, HBASE-4489-trunk-v1.patch,
HBASE-4489-trunk-v2.patch

The RegionSplitter utility allows users to create a pre-split table from the
command line or do a rolling split on an existing table. It supports
pluggable split algorithms that implement the SplitAlgorithm interface. The
only/default SplitAlgorithm is one that assumes keys fall in the range from
ASCII string to ASCII string 7FFF. This is not a sane
default, and seems useless to most users. Users are likely to be surprised by
the fact that all the region splits occur in in the byte range of ASCII
characters.
A better default split algorithm would be one that evenly divides the space
of all bytes, which is what this patch does. Making a table with five regions
would split at \x33\x33..., \x66\x66, \x99\x99..., \xCC\xCC..., and
\xFF\xFF.

[jira] [Commented] (HBASE-4102) atomicAppend: A put that appends to the latest version of a cell; i.e. reads current value then adds the bytes offered by the client to the tail and writes out a new en

[
https://issues.apache.org/jira/browse/HBASE-4102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13125134#comment-13125134
]

Lars Hofhansl commented on HBASE-4102:
--

All tests pass.

atomicAppend: A put that appends to the latest version of a cell; i.e. reads
current value then adds the bytes offered by the client to the tail and
writes out a new entry
---

Key: HBASE-4102
URL: https://issues.apache.org/jira/browse/HBASE-4102
Project: HBase
Issue Type: New Feature
Reporter: stack
Assignee: Lars Hofhansl
Attachments: 4102-v1.txt, 4102.txt

Its come up a few times that clients want to add to an existing cell rather
than make a new cell each time. At our place, the frontend keeps a list of
urls a user has visited -- their md5s -- and updates it as user progresses.
Rather than read, modify client-side, then write new value back to hbase, it
would be sweet if could do it all in one operation in hbase server. TSDB
aims to be space efficient. Rather than pay the cost of the KV wrapper per
metric, it would rather have a KV for an interval an in this KV have a value
that is all the metrics for the period.
It could be done as a coprocessor but this feels more like a fundamental
feature.
Benoît suggests that atomicAppend take a flag to indicate whether or not the
client wants to see the resulting cell; often a client won't want to see the
result and in this case, why pay the price formulating and delivering a
response that client just drops.

[jira] [Commented] (HBASE-4102) atomicAppend: A put that appends to the latest version of a cell; i.e. reads current value then adds the bytes offered by the client to the tail and writes out a new en

2011-10-11 Thread ramkrishna.s.vasudevan (Updated) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-4102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13125144#comment-13125144
]

Ted Yu commented on HBASE-4102:
---

+1 on patch v1.

atomicAppend: A put that appends to the latest version of a cell; i.e. reads
current value then adds the bytes offered by the client to the tail and
writes out a new entry
---

Key: HBASE-4102
URL: https://issues.apache.org/jira/browse/HBASE-4102
Project: HBase
Issue Type: New Feature
Reporter: stack
Assignee: Lars Hofhansl
Attachments: 4102-v1.txt, 4102.txt

[jira] [Updated] (HBASE-4558) Refactor TestOpenedRegionHandler and TestOpenRegionHandler.


 [ 
https://issues.apache.org/jira/browse/HBASE-4558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-4558:
--

Fix Version/s: 0.92.0
   Status: Patch Available  (was: Open)

 Refactor TestOpenedRegionHandler and TestOpenRegionHandler.
 ---

 Key: HBASE-4558
 URL: https://issues.apache.org/jira/browse/HBASE-4558
 Project: HBase
  Issue Type: Improvement
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Minor
 Fix For: 0.92.0

 Attachments: HBASE-4558_1.patch


 This is an improvement task taken up to refactor TestOpenedRegionandler and 
 TestOpenRegionHandler so that MockServer and MockRegionServerServices can be 
 accessed from a common utility package.
 If we do this then one of the testcases in TestOpenedRegionHandler need not 
 start up a cluster and also moving it into a common package will help in 
 mocking the server for future testcases.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4558) Refactor TestOpenedRegionHandler and TestOpenRegionHandler.

2011-10-11 Thread ramkrishna.s.vasudevan (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-4558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-4558:
--

Attachment: HBASE-4558_1.patch

Refactors MockServer and MockRegionServer to org.apache.hadoop.hbase.util 
package.
TestOpenedRegionHandler.testShouldNotCompeleteOpenedRegionSuccessfullyIfVersionMismatches()
 now will not use the mini cluster except for minizkCluster.  

 Refactor TestOpenedRegionHandler and TestOpenRegionHandler.
 ---

 Key: HBASE-4558
 URL: https://issues.apache.org/jira/browse/HBASE-4558
 Project: HBase
  Issue Type: Improvement
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Minor
 Fix For: 0.92.0

 Attachments: HBASE-4558_1.patch


 This is an improvement task taken up to refactor TestOpenedRegionandler and 
 TestOpenRegionHandler so that MockServer and MockRegionServerServices can be 
 accessed from a common utility package.
 If we do this then one of the testcases in TestOpenedRegionHandler need not 
 start up a cluster and also moving it into a common package will help in 
 mocking the server for future testcases.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4540) OpenedRegionHandler is not enforcing atomicity of the operation it is performing

2011-10-11 Thread ramkrishna.s.vasudevan (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-4540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-4540:
--

Attachment: HBASE-4540_90.patch

Contains patch for 0.90.x version.  Moving MockServer and MockRegionServices to 
test.util package has also been done. Contains similar changes as in trunk 
version.

 OpenedRegionHandler is not enforcing atomicity of the operation it is 
 performing
 

 Key: HBASE-4540
 URL: https://issues.apache.org/jira/browse/HBASE-4540
 Project: HBase
  Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 0.92.0

 Attachments: HBASE-4540_1.patch, HBASE-4540_90.patch


 - OpenedRegionHandler has not yet deleted the znode of the region R1 opened 
 by RS1.
 - RS1 goes down.
 - Servershutdownhandler assigns the region R1 to RS2.
 - The znode of R1 is moved to OFFLINE state by master or OPENING state by 
 RS2 if RS2 has started opening the region.
 - Now the first OpenedRegionHandler tries to delete the znode thinking its 
 in OPENED state but fails.
 - Though it fails it removes the node from RIT and adds RS1 as the owner of 
 R1 in master's memory.
 - Now when RS2 completes opening the region the master is not able to open 
 the region as already the reigon has been deleted from RIT.
 {code}
 Master
 ==
 2011-10-05 20:49:45,301 INFO 
 org.apache.hadoop.hbase.master.handler.ServerShutdownHandler: Finished 
 processing of shutdown of linux146,60020,1317827727647
 2011-10-05 20:49:54,177 DEBUG org.apache.hadoop.hbase.master.HMaster: Not 
 running balancer because 1 region(s) in transition: 
 {3e69d628a8bd8e9b7c5e7a2a6e03aad9=t1,,1317827883842.3e69d628a8bd8e9b7c5e7a2a6e03aad9.
  state=PENDING_OPEN, ts=1317827985272, server=linux76,60020,1317827746847}
 2011-10-05 20:49:57,720 DEBUG 
 org.apache.hadoop.hbase.master.AssignmentManager: Handling 
 transition=M_ZK_REGION_OFFLINE, server=linux76,6,1317827742012, 
 region=3e69d628a8bd8e9b7c5e7a2a6e03aad9
 2011-10-05 20:50:14,501 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 master:6-0x132d3dc13090023 Deleting existing unassigned node for 
 3e69d628a8bd8e9b7c5e7a2a6e03aad9 that is in expected state RS_ZK_REGION_OPENED
 2011-10-05 20:50:14,505 WARN org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 master:6-0x132d3dc13090023 Attempting to delete unassigned node 
 3e69d628a8bd8e9b7c5e7a2a6e03aad9 in RS_ZK_REGION_OPENED state but node is in 
 RS_ZK_REGION_OPENING state
 After the region is opened in RS2
 =
 2011-10-05 20:50:48,066 DEBUG 
 org.apache.hadoop.hbase.master.AssignmentManager: Handling 
 transition=RS_ZK_REGION_OPENING, server=linux76,60020,1317827746847, 
 region=3e69d628a8bd8e9b7c5e7a2a6e03aad9, which is more than 15 seconds late
 2011-10-05 20:50:48,290 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received OPENING for region 
 3e69d628a8bd8e9b7c5e7a2a6e03aad9 from server linux76,60020,1317827746847 but 
 region was in  the state null and not in expected PENDING_OPEN or OPENING 
 states
 2011-10-05 20:50:53,743 DEBUG 
 org.apache.hadoop.hbase.master.AssignmentManager: Handling 
 transition=RS_ZK_REGION_OPENING, server=linux76,60020,1317827746847, 
 region=3e69d628a8bd8e9b7c5e7a2a6e03aad9
 2011-10-05 20:50:54,182 DEBUG org.apache.hadoop.hbase.master.CatalogJanitor: 
 Scanned 1 catalog row(s) and gc'd 0 unreferenced parent region(s)
 2011-10-05 20:50:54,397 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received OPENING for region 
 3e69d628a8bd8e9b7c5e7a2a6e03aad9 from server linux76,60020,1317827746847 but 
 region was in  the state null and not in expected PENDING_OPEN or OPENING 
 states
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4540) OpenedRegionHandler is not enforcing atomicity of the operation it is performing

2011-10-11 Thread ramkrishna.s.vasudevan (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13125174#comment-13125174
 ] 

ramkrishna.s.vasudevan commented on HBASE-4540:
---

Testcases results will let you know tomorrow.  

 OpenedRegionHandler is not enforcing atomicity of the operation it is 
 performing
 

 Key: HBASE-4540
 URL: https://issues.apache.org/jira/browse/HBASE-4540
 Project: HBase
  Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 0.92.0

 Attachments: HBASE-4540_1.patch, HBASE-4540_90.patch


 - OpenedRegionHandler has not yet deleted the znode of the region R1 opened 
 by RS1.
 - RS1 goes down.
 - Servershutdownhandler assigns the region R1 to RS2.
 - The znode of R1 is moved to OFFLINE state by master or OPENING state by 
 RS2 if RS2 has started opening the region.
 - Now the first OpenedRegionHandler tries to delete the znode thinking its 
 in OPENED state but fails.
 - Though it fails it removes the node from RIT and adds RS1 as the owner of 
 R1 in master's memory.
 - Now when RS2 completes opening the region the master is not able to open 
 the region as already the reigon has been deleted from RIT.
 {code}
 Master
 ==
 2011-10-05 20:49:45,301 INFO 
 org.apache.hadoop.hbase.master.handler.ServerShutdownHandler: Finished 
 processing of shutdown of linux146,60020,1317827727647
 2011-10-05 20:49:54,177 DEBUG org.apache.hadoop.hbase.master.HMaster: Not 
 running balancer because 1 region(s) in transition: 
 {3e69d628a8bd8e9b7c5e7a2a6e03aad9=t1,,1317827883842.3e69d628a8bd8e9b7c5e7a2a6e03aad9.
  state=PENDING_OPEN, ts=1317827985272, server=linux76,60020,1317827746847}
 2011-10-05 20:49:57,720 DEBUG 
 org.apache.hadoop.hbase.master.AssignmentManager: Handling 
 transition=M_ZK_REGION_OFFLINE, server=linux76,6,1317827742012, 
 region=3e69d628a8bd8e9b7c5e7a2a6e03aad9
 2011-10-05 20:50:14,501 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 master:6-0x132d3dc13090023 Deleting existing unassigned node for 
 3e69d628a8bd8e9b7c5e7a2a6e03aad9 that is in expected state RS_ZK_REGION_OPENED
 2011-10-05 20:50:14,505 WARN org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 master:6-0x132d3dc13090023 Attempting to delete unassigned node 
 3e69d628a8bd8e9b7c5e7a2a6e03aad9 in RS_ZK_REGION_OPENED state but node is in 
 RS_ZK_REGION_OPENING state
 After the region is opened in RS2
 =
 2011-10-05 20:50:48,066 DEBUG 
 org.apache.hadoop.hbase.master.AssignmentManager: Handling 
 transition=RS_ZK_REGION_OPENING, server=linux76,60020,1317827746847, 
 region=3e69d628a8bd8e9b7c5e7a2a6e03aad9, which is more than 15 seconds late
 2011-10-05 20:50:48,290 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received OPENING for region 
 3e69d628a8bd8e9b7c5e7a2a6e03aad9 from server linux76,60020,1317827746847 but 
 region was in  the state null and not in expected PENDING_OPEN or OPENING 
 states
 2011-10-05 20:50:53,743 DEBUG 
 org.apache.hadoop.hbase.master.AssignmentManager: Handling 
 transition=RS_ZK_REGION_OPENING, server=linux76,60020,1317827746847, 
 region=3e69d628a8bd8e9b7c5e7a2a6e03aad9
 2011-10-05 20:50:54,182 DEBUG org.apache.hadoop.hbase.master.CatalogJanitor: 
 Scanned 1 catalog row(s) and gc'd 0 unreferenced parent region(s)
 2011-10-05 20:50:54,397 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received OPENING for region 
 3e69d628a8bd8e9b7c5e7a2a6e03aad9 from server linux76,60020,1317827746847 but 
 region was in  the state null and not in expected PENDING_OPEN or OPENING 
 states
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4573) [book] book.xml - adding examples in KeyValue section


 [ 
https://issues.apache.org/jira/browse/HBASE-4573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-4573:
-

Status: Patch Available  (was: Open)

 [book] book.xml - adding examples in KeyValue section
 -

 Key: HBASE-4573
 URL: https://issues.apache.org/jira/browse/HBASE-4573
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
 Attachments: book_HBASE_4573.xml.patch


 book.xml
 * In Arch/Region/Store/KeyValue, adding a sub-section for examples of how a 
 Put is turned into a KeyValue.
 * This is a follow-up from yesterday's patch where it added documentation on 
 KeyValue in the arch section.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4573) [book] book.xml - adding examples in KeyValue section


 [ 
https://issues.apache.org/jira/browse/HBASE-4573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-4573:
-

Attachment: book_HBASE_4573.xml.patch

 [book] book.xml - adding examples in KeyValue section
 -

 Key: HBASE-4573
 URL: https://issues.apache.org/jira/browse/HBASE-4573
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
 Attachments: book_HBASE_4573.xml.patch


 book.xml
 * In Arch/Region/Store/KeyValue, adding a sub-section for examples of how a 
 Put is turned into a KeyValue.
 * This is a follow-up from yesterday's patch where it added documentation on 
 KeyValue in the arch section.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4573) [book] book.xml - adding examples in KeyValue section

2011-10-11 Thread Karthik Ranganathan (Commented) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-4573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-4573:
-

Resolution: Fixed
Status: Resolved  (was: Patch Available)

 [book] book.xml - adding examples in KeyValue section
 -

 Key: HBASE-4573
 URL: https://issues.apache.org/jira/browse/HBASE-4573
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
 Attachments: book_HBASE_4573.xml.patch


 book.xml
 * In Arch/Region/Store/KeyValue, adding a sub-section for examples of how a 
 Put is turned into a KeyValue.
 * This is a follow-up from yesterday's patch where it added documentation on 
 KeyValue in the arch section.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4489) Better key splitting in RegionSplitter

2011-10-11 Thread Dave Revell (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-4489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13125196#comment-13125196
]

Dave Revell commented on HBASE-4489:

@Ted: will add license in patch -v3.

@Jonathan:
* We did agree to leave MD5StringSplit as the default in 0.90, I'll fix that
* I have no objection to turning rollingsplit into a different utility. That
seems out of scope here though; my intent in this ticket was to make
RegionSplitter do something sane.
* Unit tests are a good idea, I'll add some
* Using positive hex with (byte) casting is a good idea, I'll change that

Better key splitting in RegionSplitter
--

[jira] [Commented] (HBASE-4102) atomicAppend: A put that appends to the latest version of a cell; i.e. reads current value then adds the bytes offered by the client to the tail and writes out a new en

2011-10-11 Thread Jonathan Gray (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-4102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13125225#comment-13125225
]

Jonathan Gray commented on HBASE-4102:
--

This is really nice Lars. I'd love to see integration with RWCC and to somehow
unify the code with Increment. But I'm okay with committing this and filing a
follow-up JIRA.

I'm also going to backport this into my local 92 branch but I think it should
only be committed to trunk. Let's put all the polish on before putting it in
an official release.

Nice work!

atomicAppend: A put that appends to the latest version of a cell; i.e. reads
current value then adds the bytes offered by the client to the tail and
writes out a new entry
---

Key: HBASE-4102
URL: https://issues.apache.org/jira/browse/HBASE-4102
Project: HBase
Issue Type: New Feature
Reporter: stack
Assignee: Lars Hofhansl
Attachments: 4102-v1.txt, 4102.txt

[jira] [Commented] (HBASE-4572) Multithread compaction in regionserver


[ 
https://issues.apache.org/jira/browse/HBASE-4572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13125240#comment-13125240
 ] 

Karthik Ranganathan commented on HBASE-4572:


Compactions are already multi-threaded (I am talking about 0.92 and trunk), see 
CompactSplitThread.java and look for conf settings like 
hbase.regionserver.thread.compaction.large and 
hbase.regionserver.thread.compaction.small. There are now 2 queues - a queue 
for large compactions and another for small ones (defined by the amount of data 
it has to read in order to compact). Each can be serviced by a configurable 
number of threads.

Does this work for you?

 Multithread compaction in regionserver
 --

 Key: HBASE-4572
 URL: https://issues.apache.org/jira/browse/HBASE-4572
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Affects Versions: 0.90.4
Reporter: Aaron Guo
 Fix For: 0.92.0


 RegionServer only have one thread doing region compaction staff.
 Could We adjust the RegionServer compation thread num and priority online. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4573) [book] book.xml - adding examples in KeyValue section

2011-10-11 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13125296#comment-13125296
 ] 

Hudson commented on HBASE-4573:
---

Integrated in HBase-TRUNK #2317 (See 
[https://builds.apache.org/job/HBase-TRUNK/2317/])
HBASE-4573 book.xml, Put to KeyValue examples.

dmeil : 
Files : 
* /hbase/trunk/src/docbkx/book.xml


 [book] book.xml - adding examples in KeyValue section
 -

 Key: HBASE-4573
 URL: https://issues.apache.org/jira/browse/HBASE-4573
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
 Attachments: book_HBASE_4573.xml.patch


 book.xml
 * In Arch/Region/Store/KeyValue, adding a sub-section for examples of how a 
 Put is turned into a KeyValue.
 * This is a follow-up from yesterday's patch where it added documentation on 
 KeyValue in the arch section.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4422) Move block cache parameters and references into single CacheConf class


[ 
https://issues.apache.org/jira/browse/HBASE-4422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13125366#comment-13125366
 ] 

jirapos...@reviews.apache.org commented on HBASE-4422:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2089/
---

(Updated 2011-10-11 20:35:51.722038)


Review request for hbase, Dhruba Borthakur, Michael Stack, and Li Pi.


Changes
---

Updated version rebased on latest 92 branch.  Two new test failures after 
rebasing...  TestStoreFile.testBloomTypes and TestHFile.testTFileFeatures.

It seems like I'm somehow corrupting data or dropping a flag somewhere?  Not 
really sure.


Summary
---

Creates a new CacheConfig class and moves almost everything block cache related 
into this single class.  Adding new configuration params and booleans and such 
should be much better.

All tests are NOT passing yet, still working on it, but wanted to have 
something up today.  Basically code complete but broken :)


This addresses bug HBASE-4422.
https://issues.apache.org/jira/browse/HBASE-4422


Diffs (updated)
-

  /src/main/java/org/apache/hadoop/hbase/io/HalfStoreFileReader.java 1182021 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileReader.java 
1182021 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileWriter.java 
1182021 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/BlockCache.java 1182021 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java PRE-CREATION 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/DoubleBlockCache.java 1182021 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java 1182021 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/HFilePrettyPrinter.java 
1182021 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV1.java 1182021 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java 1182021 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV1.java 1182021 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java 1182021 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java 1182021 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/SimpleBlockCache.java 1182021 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/slab/SingleSizeCache.java 
1182021 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/slab/SlabCache.java 1182021 
  /src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java 
1182021 
  /src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 
1182021 
  /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1182021 
  /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 
1182021 
  /src/main/java/org/apache/hadoop/hbase/regionserver/Store.java 1182021 
  /src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java 1182021 
  /src/main/java/org/apache/hadoop/hbase/util/BloomFilterFactory.java 1182021 
  /src/main/java/org/apache/hadoop/hbase/util/CompressionTest.java 1182021 
  /src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java 
1182021 
  /src/test/java/org/apache/hadoop/hbase/io/TestHalfStoreFileReader.java 
1182021 
  /src/test/java/org/apache/hadoop/hbase/io/hfile/RandomSeek.java 1182021 
  /src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java 1182021 
  /src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFile.java 1182021 
  /src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlockIndex.java 
1182021 
  /src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFilePerformance.java 
1182021 
  /src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileReaderV1.java 
1182021 
  /src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileSeek.java 1182021 
  /src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileWriterV2.java 
1182021 
  /src/test/java/org/apache/hadoop/hbase/io/hfile/TestReseekTo.java 1182021 
  /src/test/java/org/apache/hadoop/hbase/io/hfile/TestSeekTo.java 1182021 
  /src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java 
1182021 
  
/src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFiles.java 
1182021 
  /src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksRead.java 
1182021 
  /src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactSelection.java 
1182021 
  
/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompoundBloomFilter.java
 1182021 
  /src/test/java/org/apache/hadoop/hbase/regionserver/TestFSErrorsExposed.java 
1182021 
  /src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java 1182021 
  /src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java 
1182021 
  
/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFileBlockCacheSummary.java
 1182021

[jira] [Updated] (HBASE-4102) atomicAppend: A put that appends to the latest version of a cell; i.e. reads current value then adds the bytes offered by the client to the tail and writes out a new entr

2011-10-11 Thread Lars Hofhansl (Updated) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-4102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Lars Hofhansl updated HBASE-4102:
-

Fix Version/s: 0.94.0

atomicAppend: A put that appends to the latest version of a cell; i.e. reads
current value then adds the bytes offered by the client to the tail and
writes out a new entry
---

Key: HBASE-4102
URL: https://issues.apache.org/jira/browse/HBASE-4102
Project: HBase
Issue Type: New Feature
Reporter: stack
Assignee: Lars Hofhansl
Fix For: 0.94.0

Attachments: 4102-v1.txt, 4102.txt

[jira] [Commented] (HBASE-4102) atomicAppend: A put that appends to the latest version of a cell; i.e. reads current value then adds the bytes offered by the client to the tail and writes out a new en

2011-10-11 Thread jirapos...@reviews.apache.org (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-4102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13125374#comment-13125374
]

Lars Hofhansl commented on HBASE-4102:
--

Agree with trunk only (otherwise never get 0.92 out of the door).

bq. Let's put all the polish on before putting it in an official release.

So you're saying the RWCC integration and refactoring should go in before this
is committed to trunk?
I would offer to do the RWCC integration right away, but this seems like a
special case where data in the memstore is changed in place (same as
Increment), and I am not sure about the RWCC semantics in that case.

I'd say we put this in as is, and then think about RWCC and refactoring for
this and Increment.

atomicAppend: A put that appends to the latest version of a cell; i.e. reads
current value then adds the bytes offered by the client to the tail and
writes out a new entry
---

Key: HBASE-4102
URL: https://issues.apache.org/jira/browse/HBASE-4102
Project: HBase
Issue Type: New Feature
Reporter: stack
Assignee: Lars Hofhansl
Fix For: 0.94.0

Attachments: 4102-v1.txt, 4102.txt

[jira] [Commented] (HBASE-4554) Allow set/unset coprocessor table attributes from shell.


[ 
https://issues.apache.org/jira/browse/HBASE-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13125397#comment-13125397
 ] 

jirapos...@reviews.apache.org commented on HBASE-4554:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2350/
---

Review request for hbase and Gary Helmling.


Summary
---

Features:
- Support adding table level coprocessors -- regionobserver and endpoint -- 
from shell.
- Support removing table attributes from shell.

Introduced shell comands:

  hbase alter 't1', METHOD = 'table_att', \

'coprocessor$1'='|org.apache.hadoop.hbase.coprocessor.SimpleRegionObserver||', 
\
'coprocessor$2'='hdfs://foo.jar|com.foo.FooRegionObserver|1001|'

  hbase alter 't1', METHOD = 'table_att_unset', NAME = 'MAX_FILESIZE'
  hbase alter 't1', METHOD = 'table_att_unset', NAME = 'coprocessor$1'


This addresses bug HBASE-4554.
https://issues.apache.org/jira/browse/HBASE-4554


Diffs
-

  src/main/java/org/apache/hadoop/hbase/HConstants.java 3af1bf0 
  src/main/ruby/hbase/admin.rb b244ffe 
  src/main/ruby/shell/commands/alter.rb ee3668d 
  src/test/ruby/hbase/admin_test.rb 5e491e4 

Diff: https://reviews.apache.org/r/2350/diff


Testing
---

Test passed after applied HBase-4555 patch locally. 


Thanks,

Mingjie



 Allow set/unset coprocessor table attributes from shell.
 

 Key: HBASE-4554
 URL: https://issues.apache.org/jira/browse/HBASE-4554
 Project: HBase
  Issue Type: Improvement
  Components: coprocessors
Reporter: Mingjie Lai
Assignee: Mingjie Lai
 Fix For: 0.92.0


 Table/region level coprocessor -- RegionObserver -- can be configured by 
 setting a HTD's attribute which matches Coprocessor$*. 
 Current shell -- alter -- cannot support to set/unset a table's arbitrary 
 attribute. We need it in order to configure region level coprocessors to a 
 table. 
 Proposed new shell:
 {code}
 hbase shell  alter 't1', METHOD = 'table_att', COPROCESSOR$1 = 
 'hdfs://cp/foo.jar|org.apache.hadoop.hbase.sample|1|'
 hbase shell  describe 't1'
  {NAME = 't1', COPROCESSOR$1 = 
 'hdfs://cp/foo.jar|org.apache.hadoop.hbase.sample|1|', MAX_FILESIZE = 
 '134217728', ...}
 hbase shell  alter 't1', METHOD = 'table_att_unset', COPROCESSOR$1
 hbase shell  describe 't1'
  {NAME = 't1', MAX_FILESIZE = '134217728', ...}
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HBASE-4335) Splits can create temporary holes in .META. that confuse clients and regionservers

2011-10-11 Thread Lars Hofhansl (Resolved) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Lars Hofhansl resolved HBASE-4335.
--

Resolution: Fixed
Hadoop Flags: Reviewed

Integrated into 0.92 and trunk.

Splits can create temporary holes in .META. that confuse clients and
regionservers
--

Attachments: 4335-v2.txt, 4335-v3.txt, 4335-v4.txt, 4335-v5.txt,
4335.txt

[jira] [Commented] (HBASE-4536) Allow CF to retain deleted rows

2011-10-11 Thread jirapos...@reviews.apache.org (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-4536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13125399#comment-13125399
]

Lars Hofhansl commented on HBASE-4536:
--

I think I'd like to keep the error, to avoid surprises.

And some more eyes should look at this...
@jgray, @stack wanna have a look too?

Allow CF to retain deleted rows
---

[jira] [Commented] (HBASE-4422) Move block cache parameters and references into single CacheConf class


[ 
https://issues.apache.org/jira/browse/HBASE-4422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13125410#comment-13125410
 ] 

jirapos...@reviews.apache.org commented on HBASE-4422:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2089/#review2520
---


There still seem to be some ZK-related changes in the patch

-this.conf.setLong(hbase.zookeeper.recoverable.waittime, 1000);
+this.conf.setLong(zookeeper.recovery.retry.intervalmill, 100);
+this.conf.setLong(zookeeper.recovery.retry, 1);

Also, for some reason the patch does not apply cleanly to StoreFile.java (the 
hunk that deletes getBlockCache), but that is easily fixed manually.

Debugging TestStoreFile and TestHFile...

- Mikhail


On 2011-10-11 20:35:51, Jonathan Gray wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/2089/
bq.  ---
bq.  
bq.  (Updated 2011-10-11 20:35:51)
bq.  
bq.  
bq.  Review request for hbase, Dhruba Borthakur, Michael Stack, and Li Pi.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  Creates a new CacheConfig class and moves almost everything block cache 
related into this single class.  Adding new configuration params and booleans 
and such should be much better.
bq.  
bq.  All tests are NOT passing yet, still working on it, but wanted to have 
something up today.  Basically code complete but broken :)
bq.  
bq.  
bq.  This addresses bug HBASE-4422.
bq.  https://issues.apache.org/jira/browse/HBASE-4422
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq./src/main/java/org/apache/hadoop/hbase/io/HalfStoreFileReader.java 
1182021 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileReader.java 
1182021 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileWriter.java 
1182021 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/BlockCache.java 1182021 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java 
PRE-CREATION 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/DoubleBlockCache.java 
1182021 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java 1182021 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFilePrettyPrinter.java 
1182021 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV1.java 
1182021 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java 
1182021 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV1.java 
1182021 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java 
1182021 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java 
1182021 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/SimpleBlockCache.java 
1182021 
bq.
/src/main/java/org/apache/hadoop/hbase/io/hfile/slab/SingleSizeCache.java 
1182021 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/slab/SlabCache.java 
1182021 
bq./src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java 
1182021 
bq.
/src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 
1182021 
bq./src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1182021 
bq./src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 
1182021 
bq./src/main/java/org/apache/hadoop/hbase/regionserver/Store.java 1182021 
bq./src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java 
1182021 
bq./src/main/java/org/apache/hadoop/hbase/util/BloomFilterFactory.java 
1182021 
bq./src/main/java/org/apache/hadoop/hbase/util/CompressionTest.java 1182021 
bq./src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java 
1182021 
bq./src/test/java/org/apache/hadoop/hbase/io/TestHalfStoreFileReader.java 
1182021 
bq./src/test/java/org/apache/hadoop/hbase/io/hfile/RandomSeek.java 1182021 
bq./src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java 
1182021 
bq./src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFile.java 1182021 
bq./src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlockIndex.java 
1182021 
bq.
/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFilePerformance.java 
1182021 
bq./src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileReaderV1.java 
1182021 
bq./src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileSeek.java 
1182021 
bq./src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileWriterV2.java 
1182021 
bq./src/test/java/org/apache/hadoop/hbase/io/hfile/TestReseekTo.java 
1182021 
bq./src/test/java/org/apache/hadoop/hbase/io/hfile/TestSeekTo.java 1182021 
bq.

[jira] [Updated] (HBASE-4489) Better key splitting in RegionSplitter

2011-10-11 Thread Dave Revell (Updated) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-4489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Dave Revell updated HBASE-4489:
---

Attachment: HBASE-4489-branch0.90-v3.patch
HBASE-4489-trunk-v3.patch

v3 patches with suggested fixes.

Better key splitting in RegionSplitter
--

[jira] [Created] (HBASE-4574) Node deleted but still in RIT printed too often

2011-10-11 Thread Jean-Daniel Cryans (Created) (JIRA)

Node deleted but still in RIT printed too often
-

 Key: HBASE-4574
 URL: https://issues.apache.org/jira/browse/HBASE-4574
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Jean-Daniel Cryans
 Fix For: 0.92.0


Looking at the 0.92 master logs, I see I often get this message:

bq. WARN org.apache.hadoop.hbase.master.AssignmentManager: Node deleted but 
still in RIT: TestTable,blah. state=OPEN, ts=1318369648361, server=blah

The issue seems to be due to a race between OpenedRegionHandler and watchers in 
AssignmentManager. Specifically, ORH first deletes the znode then deletes the 
in-memory RIT data structure (via regionOnline). Between the two steps a 
watcher is triggered and if it arrives first then it will see the region still 
in RIT.

If the message is really supposed to be a warning then in the current form it's 
useless as people will see this message 99% of the time because of this race.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-4575) Inconsistent naming for ZK config parameters

2011-10-11 Thread Jonathan Gray (Created) (JIRA)

Inconsistent naming for ZK config parameters


 Key: HBASE-4575
 URL: https://issues.apache.org/jira/browse/HBASE-4575
 Project: HBase
  Issue Type: Bug
  Components: test, zookeeper
Affects Versions: 0.92.0, 0.94.0
Reporter: Jonathan Gray
Assignee: Jonathan Gray
 Fix For: 0.92.0


I've found some misnaming of certain ZK config options.  Make them consistent.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4422) Move block cache parameters and references into single CacheConf class

2011-10-11 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13125437#comment-13125437
 ] 

jirapos...@reviews.apache.org commented on HBASE-4422:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2089/
---

(Updated 2011-10-11 22:18:32.037217)


Review request for hbase, Dhruba Borthakur, Michael Stack, and Li Pi.


Changes
---

Lucky number 13!!!  This is the one!

Fixes failing TestHFile / TestStoreFile.  Thanks Mikhail.


Summary
---

Creates a new CacheConfig class and moves almost everything block cache related 
into this single class.  Adding new configuration params and booleans and such 
should be much better.

All tests are NOT passing yet, still working on it, but wanted to have 
something up today.  Basically code complete but broken :)


This addresses bug HBASE-4422.
https://issues.apache.org/jira/browse/HBASE-4422


Diffs (updated)
-

  /src/main/java/org/apache/hadoop/hbase/io/HalfStoreFileReader.java 1182021 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileReader.java 
1182021 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileWriter.java 
1182021 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/BlockCache.java 1182021 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java PRE-CREATION 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/DoubleBlockCache.java 1182021 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java 1182021 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/HFilePrettyPrinter.java 
1182021 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV1.java 1182021 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java 1182021 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV1.java 1182021 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java 1182021 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java 1182021 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/SimpleBlockCache.java 1182021 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/slab/SingleSizeCache.java 
1182021 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/slab/SlabCache.java 1182021 
  /src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java 
1182021 
  /src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 
1182021 
  /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1182021 
  /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 
1182021 
  /src/main/java/org/apache/hadoop/hbase/regionserver/Store.java 1182021 
  /src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java 1182021 
  /src/main/java/org/apache/hadoop/hbase/util/BloomFilterFactory.java 1182021 
  /src/main/java/org/apache/hadoop/hbase/util/CompressionTest.java 1182021 
  /src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java 
1182021 
  /src/test/java/org/apache/hadoop/hbase/io/TestHalfStoreFileReader.java 
1182021 
  /src/test/java/org/apache/hadoop/hbase/io/hfile/RandomSeek.java 1182021 
  /src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java 1182021 
  /src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFile.java 1182021 
  /src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlockIndex.java 
1182021 
  /src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFilePerformance.java 
1182021 
  /src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileReaderV1.java 
1182021 
  /src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileSeek.java 1182021 
  /src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileWriterV2.java 
1182021 
  /src/test/java/org/apache/hadoop/hbase/io/hfile/TestReseekTo.java 1182021 
  /src/test/java/org/apache/hadoop/hbase/io/hfile/TestSeekTo.java 1182021 
  /src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java 
1182021 
  
/src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFiles.java 
1182021 
  /src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksRead.java 
1182021 
  /src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactSelection.java 
1182021 
  
/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompoundBloomFilter.java
 1182021 
  /src/test/java/org/apache/hadoop/hbase/regionserver/TestFSErrorsExposed.java 
1182021 
  /src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java 1182021 
  /src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java 
1182021 
  
/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFileBlockCacheSummary.java
 1182021 
  /src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALReplay.java 
1182021 

Diff: https://reviews.apache.org/r/2089/diff


Testing
---

[jira] [Commented] (HBASE-4555) TestShell seems passed, but actually errors seen in test output file

2011-10-11 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13125466#comment-13125466
 ] 

jirapos...@reviews.apache.org commented on HBASE-4555:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2351/
---

Review request for hbase.


Summary
---

HBase shell unit cases are in ruby. Ruby unit test runner 
Test::Unit::AutoRunner.run only prints unit tests assertion failures to console 
by default. So it actually swallows all the assertion errors from ruby, and 
junit cannot notice any shell test failure. 

Fixed the test_runner.rb to chech the return value and throw a runtime 
exception, so that TestShell can get to know there is something wrong by test 
errors. 

Also fixed 7 previously existing failed test cases. 

In addition, after HBase-1730, there's no need to disable a table before 
altering a table. Changed the test cases accordingly. 


This addresses bug HBase-4555.
https://issues.apache.org/jira/browse/HBase-4555


Diffs
-

  src/test/ruby/hbase/admin_test.rb 5e491e4 
  src/test/ruby/tests_runner.rb 0dbc5ce 

Diff: https://reviews.apache.org/r/2351/diff


Testing
---

TestShell passed locally. 


Thanks,

Mingjie



 TestShell seems passed, but actually errors seen in test output file
 

 Key: HBASE-4555
 URL: https://issues.apache.org/jira/browse/HBASE-4555
 Project: HBase
  Issue Type: Test
  Components: test
Reporter: Mingjie Lai
Assignee: Mingjie Lai
 Attachments: 4555.diff


 When I was making test cases for 4554, I saw a weird issue that TestShell 
 seems to pass, but actually I saw error messages in the output file.
 {code}
 ---
  T E S T S
 ---
 Running org.apache.hadoop.hbase.client.TestShell
 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 39.252 sec
 Results :
 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
 {code}
 Error messages in org.apache.hadoop.hbase.client.TestShell-output.txt:
 {code}
 ...
   6) Error:
 test_alter_should_support_shortcut_DELETE_alter_specs(Hbase::AdminAlterTableTest):
 ArgumentError: There should be at least one argument but the table name
 
 /home/mlai/git/hbase-private/src/test/ruby/../../main/ruby/hbase/admin.rb:307:in
  `alter'
 ./src/test/ruby/hbase/admin_test.rb:271:in 
 `test_alter_should_support_shortcut_DELETE_alter_specs'
 org/jruby/RubyProc.java:268:in `call'
 org/jruby/RubyKernel.java:2038:in `send'
 org/jruby/RubyArray.java:1572:in `each'
 org/jruby/RubyArray.java:1572:in `each'
 
   7) Error:
 test_split_should_work(Hbase::AdminMethodsTest):
 ArgumentError: wrong number of arguments (1 for 2)
 ./src/test/ruby/hbase/admin_test.rb:99:in `test_split_should_work'
 org/jruby/RubyProc.java:268:in `call'
 org/jruby/RubyKernel.java:2038:in `send'
 org/jruby/RubyArray.java:1572:in `each'
 org/jruby/RubyArray.java:1572:in `each'
 
 192 tests, 259 assertions, 1 failures, 6 errors
 Done with tests! Shutting down the cluster...
 2011-10-07 16:46:14,760 INFO  [main] hbase.HBaseTestingUtility(551): Shutting 
 down minicluster
 2011-10-07 16:46:14,760 DEBUG [main] util.JVMClusterUtil(214): Shutting down 
 HBase Cluster
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-4576) [book] moved Cluster Replication chapter to OpsMgt, other nit

2011-10-11 Thread Doug Meil (Created) (JIRA)

[book] moved Cluster Replication chapter to OpsMgt, other nit
-

 Key: HBASE-4576
 URL: https://issues.apache.org/jira/browse/HBASE-4576
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Trivial


book.xml
* There was a placeholder chapter for Cluster Replication that just linked to 
replication.html.  I moved this to a section in the Ops Mgt chapter.
* Nit:  in Schema Design Smackdown sub-section, changed Winner to 
Preference.  I got a little carried away when I created that entry and I'm 
toning it down a tad.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4576) [book] moved Cluster Replication chapter to OpsMgt, other nit


 [ 
https://issues.apache.org/jira/browse/HBASE-4576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-4576:
-

Resolution: Fixed
Status: Resolved  (was: Patch Available)

 [book] moved Cluster Replication chapter to OpsMgt, other nit
 -

 Key: HBASE-4576
 URL: https://issues.apache.org/jira/browse/HBASE-4576
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Trivial
 Attachments: docbkx_HBASE_4576.patch


 book.xml
 * There was a placeholder chapter for Cluster Replication that just linked to 
 replication.html.  I moved this to a section in the Ops Mgt chapter.
 * Nit:  in Schema Design Smackdown sub-section, changed Winner to 
 Preference.  I got a little carried away when I created that entry and I'm 
 toning it down a tad.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4576) [book] moved Cluster Replication chapter to OpsMgt, other nit

2011-10-11 Thread Nicolas Spiegelberg (Commented) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-4576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-4576:
-

Attachment: docbkx_HBASE_4576.patch

 [book] moved Cluster Replication chapter to OpsMgt, other nit
 -

 Key: HBASE-4576
 URL: https://issues.apache.org/jira/browse/HBASE-4576
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Trivial
 Attachments: docbkx_HBASE_4576.patch


 book.xml
 * There was a placeholder chapter for Cluster Replication that just linked to 
 replication.html.  I moved this to a section in the Ops Mgt chapter.
 * Nit:  in Schema Design Smackdown sub-section, changed Winner to 
 Preference.  I got a little carried away when I created that entry and I'm 
 toning it down a tad.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4489) Better key splitting in RegionSplitter


[ 
https://issues.apache.org/jira/browse/HBASE-4489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13125474#comment-13125474
 ] 

Nicolas Spiegelberg commented on HBASE-4489:


1. If you want to save space, you should probably switch to a UINT32 for the 
key range instead of the current INT64.  This should scale for up to 2 million 
regions.

2. We (FB) have this specified for MD5StringSplit elsewhere, but I think an 
mandatory requirement if your goal is better key splitting is to provide the 
hash function that, given a user-specified key, hash it and return what you 
need to insert into HBase.  Maybe add a generateKey() function to 
SplitAlgorithm interface.  Worst case is we make it an identity operation.  

Talked with StumbleUpon  Salesforce guys last week about making this something 
that users can enable and the HBase Client would transparently do for them.  
However, step one is giving users the APIs they need to manually do proper key 
distribution.

 Better key splitting in RegionSplitter
 --

 Key: HBASE-4489
 URL: https://issues.apache.org/jira/browse/HBASE-4489
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.90.4
Reporter: Dave Revell
Assignee: Dave Revell
 Attachments: HBASE-4489-branch0.90-v1.patch, 
 HBASE-4489-branch0.90-v2.patch, HBASE-4489-branch0.90-v3.patch, 
 HBASE-4489-trunk-v1.patch, HBASE-4489-trunk-v2.patch, 
 HBASE-4489-trunk-v3.patch


 The RegionSplitter utility allows users to create a pre-split table from the 
 command line or do a rolling split on an existing table. It supports 
 pluggable split algorithms that implement the SplitAlgorithm interface. The 
 only/default SplitAlgorithm is one that assumes keys fall in the range from 
 ASCII string  to ASCII string 7FFF. This is not a sane 
 default, and seems useless to most users. Users are likely to be surprised by 
 the fact that all the region splits occur in in the byte range of ASCII 
 characters.
 A better default split algorithm would be one that evenly divides the space 
 of all bytes, which is what this patch does. Making a table with five regions 
 would split at \x33\x33..., \x66\x66, \x99\x99..., \xCC\xCC..., and 
 \xFF\xFF.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4576) [book] moved Cluster Replication chapter to OpsMgt, other nit


 [ 
https://issues.apache.org/jira/browse/HBASE-4576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-4576:
-

Status: Patch Available  (was: Open)

 [book] moved Cluster Replication chapter to OpsMgt, other nit
 -

 Key: HBASE-4576
 URL: https://issues.apache.org/jira/browse/HBASE-4576
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Trivial
 Attachments: docbkx_HBASE_4576.patch


 book.xml
 * There was a placeholder chapter for Cluster Replication that just linked to 
 replication.html.  I moved this to a section in the Ops Mgt chapter.
 * Nit:  in Schema Design Smackdown sub-section, changed Winner to 
 Preference.  I got a little carried away when I created that entry and I'm 
 toning it down a tad.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4556) Fix all incorrect uses of InternalScanner.next(...)


[ 
https://issues.apache.org/jira/browse/HBASE-4556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13125475#comment-13125475
 ] 

Lars Hofhansl commented on HBASE-4556:
--

Another pattern to use would be:

{code}
while(scanner.next(kvs) || !kvs.isEmpty()) {
...
}
{code}



 Fix all incorrect uses of InternalScanner.next(...)
 ---

 Key: HBASE-4556
 URL: https://issues.apache.org/jira/browse/HBASE-4556
 Project: HBase
  Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl

 There are cases all over the code where InternalScanner.next(...) is not used 
 correctly.
 I see this a lot:
 {code}
 while(scanner.next(...)) {
 }
 {code}
 The correct pattern is:
 {code}
 boolean more = false;
 do {
more = scanner.next(...);
 } while (more);
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-4577) Region server reports storefileSizeMB bigger than storefileUncompressedSizeMB

2011-10-11 Thread Jean-Daniel Cryans (Created) (JIRA)

Region server reports storefileSizeMB bigger than storefileUncompressedSizeMB
-

 Key: HBASE-4577
 URL: https://issues.apache.org/jira/browse/HBASE-4577
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Jean-Daniel Cryans
Priority: Minor
 Fix For: 0.92.0


Minor issue while looking at the RS metrics:

bq. numberOfStorefiles=8, storefileUncompressedSizeMB=2418, 
storefileSizeMB=2420, compressionRatio=1.0008

I guess there's a truncation somewhere when it's adding the numbers up.

FWIW there's no compression on that table.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4556) Fix all incorrect uses of InternalScanner.next(...)

2011-10-11 Thread Lars Hofhansl (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-4556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-4556:
-

Attachment: 4556.txt

Looked through all uses of InternalScanner.next(...).

 Fix all incorrect uses of InternalScanner.next(...)
 ---

 Key: HBASE-4556
 URL: https://issues.apache.org/jira/browse/HBASE-4556
 Project: HBase
  Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Attachments: 4556.txt


 There are cases all over the code where InternalScanner.next(...) is not used 
 correctly.
 I see this a lot:
 {code}
 while(scanner.next(...)) {
 }
 {code}
 The correct pattern is:
 {code}
 boolean more = false;
 do {
more = scanner.next(...);
 } while (more);
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4536) Allow CF to retain deleted rows

2011-10-11 Thread Lars Hofhansl (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-4536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-4536:
-

Issue Type: New Feature  (was: Sub-task)
Parent: (was: HBASE-4071)

 Allow CF to retain deleted rows
 ---

 Key: HBASE-4536
 URL: https://issues.apache.org/jira/browse/HBASE-4536
 Project: HBase
  Issue Type: New Feature
  Components: regionserver
Affects Versions: 0.92.0
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.94.0


 Parent allows for a cluster to retain rows for a TTL or keep a minimum number 
 of versions.
 However, if a client deletes a row all version older than the delete tomb 
 stone will be remove at the next major compaction (and even at memstore flush 
 - see HBASE-4241).
 There should be a way to retain those version to guard against software error.
 I see two options here:
 1. Add a new flag HColumnDescriptor. Something like RETAIN_DELETED.
 2. Folds this into the parent change. I.e. keep minimum-number-of-versions of 
 versions even past the delete marker.
 #1 would allow for more flexibility. #2 comes somewhat naturally with parent 
 (from a user viewpoint)
 Comments? Any other options?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4489) Better key splitting in RegionSplitter

2011-10-11 Thread Dave Revell (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-4489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13125481#comment-13125481
]

Dave Revell commented on HBASE-4489:

@Nicolas,

In response to your #1:
If you want to save space, you should probably switch to a UINT32 for the
key range instead of the current INT64. This should scale for up to 2 million
regions.

Users can use whatever length of keys they want. RegionSplitter just chooses
the region boundaries, which has no effect on space consumption.

In response to your #2:
Every cryptographic hash function distributes its values uniformly across the
space of byte strings of length N. So that makes UniformSplit a sensible
default when using MD5 hashes or SHA1 hashes or whatever else, as long as
they're not converted to ASCII.

The goal here is sane default behavior that makes sense for typical use cases.
Evenly dividing the key space accomplishes that goal. At least that's *my* goal.

Better key splitting in RegionSplitter
--

[jira] [Commented] (HBASE-4536) Allow CF to retain deleted rows

2011-10-11 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13125482#comment-13125482
 ] 

Lars Hofhansl commented on HBASE-4536:
--

Converted to feature.

 Allow CF to retain deleted rows
 ---

 Key: HBASE-4536
 URL: https://issues.apache.org/jira/browse/HBASE-4536
 Project: HBase
  Issue Type: New Feature
  Components: regionserver
Affects Versions: 0.92.0
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.94.0


 Parent allows for a cluster to retain rows for a TTL or keep a minimum number 
 of versions.
 However, if a client deletes a row all version older than the delete tomb 
 stone will be remove at the next major compaction (and even at memstore flush 
 - see HBASE-4241).
 There should be a way to retain those version to guard against software error.
 I see two options here:
 1. Add a new flag HColumnDescriptor. Something like RETAIN_DELETED.
 2. Folds this into the parent change. I.e. keep minimum-number-of-versions of 
 versions even past the delete marker.
 #1 would allow for more flexibility. #2 comes somewhat naturally with parent 
 (from a user viewpoint)
 Comments? Any other options?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4555) TestShell seems passed, but actually errors seen in test output file


[ 
https://issues.apache.org/jira/browse/HBASE-4555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13125487#comment-13125487
 ] 

jirapos...@reviews.apache.org commented on HBASE-4555:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2351/#review2521
---

Ship it!


- Michael


On 2011-10-11 23:02:27, Mingjie Lai wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/2351/
bq.  ---
bq.  
bq.  (Updated 2011-10-11 23:02:27)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  HBase shell unit cases are in ruby. Ruby unit test runner 
Test::Unit::AutoRunner.run only prints unit tests assertion failures to console 
by default. So it actually swallows all the assertion errors from ruby, and 
junit cannot notice any shell test failure. 
bq.  
bq.  Fixed the test_runner.rb to chech the return value and throw a runtime 
exception, so that TestShell can get to know there is something wrong by test 
errors. 
bq.  
bq.  Also fixed 7 previously existing failed test cases. 
bq.  
bq.  In addition, after HBase-1730, there's no need to disable a table before 
altering a table. Changed the test cases accordingly. 
bq.  
bq.  
bq.  This addresses bug HBase-4555.
bq.  https://issues.apache.org/jira/browse/HBase-4555
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.src/test/ruby/hbase/admin_test.rb 5e491e4 
bq.src/test/ruby/tests_runner.rb 0dbc5ce 
bq.  
bq.  Diff: https://reviews.apache.org/r/2351/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  TestShell passed locally. 
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Mingjie
bq.  
bq.



 TestShell seems passed, but actually errors seen in test output file
 

 Key: HBASE-4555
 URL: https://issues.apache.org/jira/browse/HBASE-4555
 Project: HBase
  Issue Type: Test
  Components: test
Reporter: Mingjie Lai
Assignee: Mingjie Lai
 Attachments: 4555.diff


 When I was making test cases for 4554, I saw a weird issue that TestShell 
 seems to pass, but actually I saw error messages in the output file.
 {code}
 ---
  T E S T S
 ---
 Running org.apache.hadoop.hbase.client.TestShell
 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 39.252 sec
 Results :
 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
 {code}
 Error messages in org.apache.hadoop.hbase.client.TestShell-output.txt:
 {code}
 ...
   6) Error:
 test_alter_should_support_shortcut_DELETE_alter_specs(Hbase::AdminAlterTableTest):
 ArgumentError: There should be at least one argument but the table name
 
 /home/mlai/git/hbase-private/src/test/ruby/../../main/ruby/hbase/admin.rb:307:in
  `alter'
 ./src/test/ruby/hbase/admin_test.rb:271:in 
 `test_alter_should_support_shortcut_DELETE_alter_specs'
 org/jruby/RubyProc.java:268:in `call'
 org/jruby/RubyKernel.java:2038:in `send'
 org/jruby/RubyArray.java:1572:in `each'
 org/jruby/RubyArray.java:1572:in `each'
 
   7) Error:
 test_split_should_work(Hbase::AdminMethodsTest):
 ArgumentError: wrong number of arguments (1 for 2)
 ./src/test/ruby/hbase/admin_test.rb:99:in `test_split_should_work'
 org/jruby/RubyProc.java:268:in `call'
 org/jruby/RubyKernel.java:2038:in `send'
 org/jruby/RubyArray.java:1572:in `each'
 org/jruby/RubyArray.java:1572:in `each'
 
 192 tests, 259 assertions, 1 failures, 6 errors
 Done with tests! Shutting down the cluster...
 2011-10-07 16:46:14,760 INFO  [main] hbase.HBaseTestingUtility(551): Shutting 
 down minicluster
 2011-10-07 16:46:14,760 DEBUG [main] util.JVMClusterUtil(214): Shutting down 
 HBase Cluster
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-4578) NPE when altering a table that has moving regions

2011-10-11 Thread Jean-Daniel Cryans (Created) (JIRA)

NPE when altering a table that has moving regions
-

 Key: HBASE-4578
 URL: https://issues.apache.org/jira/browse/HBASE-4578
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Jean-Daniel Cryans
Priority: Critical
 Fix For: 0.92.0


I'm still not a 100% sure on the source of this error, but here's what I was 
able to get twice while altering a table that was doing a bunch of splits:

{quote}

2011-10-11 23:48:59,344 INFO 
org.apache.hadoop.hbase.master.handler.SplitRegionHandler: Handled SPLIT 
report); 
parent=TestTable,0002608338,1318376880454.a75d6815fdfc513fb1c8aabe086c6763. 
daughter 
a=TestTable,0002608338,1318376938764.ef170ff6cd8695dc8aec92e542dc9ac1.daughter 
b=TestTable,0003301408,1318376938764.36eb2530341bd46888ede312c5559b5d.
2011-10-11 23:49:09,579 DEBUG 
org.apache.hadoop.hbase.master.handler.TableEventHandler: Ignoring table not 
disabled exception for supporting online schema changes.
2011-10-11 23:49:09,580 INFO 
org.apache.hadoop.hbase.master.handler.TableEventHandler: Handling table 
operation C_M_MODIFY_TABLE on table TestTable
2011-10-11 23:49:09,612 INFO org.apache.hadoop.hbase.util.FSUtils: 
TableInfoPath = hdfs://sv4r11s38:9100/hbase/TestTable/.tableinfo tmpPath = 
hdfs://sv4r11s38:9100/hbase/TestTable/.tmp/.tableinfo.1318376949612
2011-10-11 23:49:09,692 INFO org.apache.hadoop.hbase.util.FSUtils: 
TableDescriptor stored. TableInfoPath = 
hdfs://sv4r11s38:9100/hbase/TestTable/.tableinfo
2011-10-11 23:49:09,693 INFO org.apache.hadoop.hbase.util.FSUtils: Updated 
tableinfo=hdfs://sv4r11s38:9100/hbase/TestTable/.tableinfo to blah
2011-10-11 23:49:09,695 INFO 
org.apache.hadoop.hbase.master.handler.TableEventHandler: Bucketing regions by 
region server...
2011-10-11 23:49:09,695 DEBUG org.apache.hadoop.hbase.client.MetaScanner: 
Scanning .META. starting at row=TestTable,,00 for max=2147483647 
rows
2011-10-11 23:49:09,709 DEBUG 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: 
The connection to hconnection-0x132f043bbde02e9 has been closed.
2011-10-11 23:49:09,709 ERROR org.apache.hadoop.hbase.executor.EventHandler: 
Caught throwable while processing event C_M_MODIFY_TABLE
java.lang.NullPointerException
at java.util.TreeMap.getEntry(TreeMap.java:324)
at java.util.TreeMap.containsKey(TreeMap.java:209)
at 
org.apache.hadoop.hbase.master.handler.TableEventHandler.reOpenAllRegions(TableEventHandler.java:114)
at 
org.apache.hadoop.hbase.master.handler.TableEventHandler.process(TableEventHandler.java:90)
at 
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:168)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)

{quote}

The first time the shell reported that all the regions were updated correctly, 
the second time it got stuck for a while:

{quote}

6/14 regions updated.
0/14 regions updated.
...
0/14 regions updated.
2/16 regions updated.
...
2/16 regions updated.
8/9 regions updated.
...
8/9 regions updated.
{quote}

After which I killed it, redid the alter and it worked.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4555) TestShell seems passed, but actually errors seen in test output file

2011-10-11 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13125491#comment-13125491
 ] 

jirapos...@reviews.apache.org commented on HBASE-4555:
--



bq.  On 2011-10-11 23:50:18, Michael Stack wrote:
bq.  

Mind adding this patch to JIRA so I can apply it Mingjie (You need to add it 
because you need to click the Grant license to ASF for inclusion in ASF works 
(as per the Apache License §5) button since you wrote the patch.  Good stuff.


- Michael


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2351/#review2521
---


On 2011-10-11 23:02:27, Mingjie Lai wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/2351/
bq.  ---
bq.  
bq.  (Updated 2011-10-11 23:02:27)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  HBase shell unit cases are in ruby. Ruby unit test runner 
Test::Unit::AutoRunner.run only prints unit tests assertion failures to console 
by default. So it actually swallows all the assertion errors from ruby, and 
junit cannot notice any shell test failure. 
bq.  
bq.  Fixed the test_runner.rb to chech the return value and throw a runtime 
exception, so that TestShell can get to know there is something wrong by test 
errors. 
bq.  
bq.  Also fixed 7 previously existing failed test cases. 
bq.  
bq.  In addition, after HBase-1730, there's no need to disable a table before 
altering a table. Changed the test cases accordingly. 
bq.  
bq.  
bq.  This addresses bug HBase-4555.
bq.  https://issues.apache.org/jira/browse/HBase-4555
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.src/test/ruby/hbase/admin_test.rb 5e491e4 
bq.src/test/ruby/tests_runner.rb 0dbc5ce 
bq.  
bq.  Diff: https://reviews.apache.org/r/2351/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  TestShell passed locally. 
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Mingjie
bq.  
bq.



 TestShell seems passed, but actually errors seen in test output file
 

 Key: HBASE-4555
 URL: https://issues.apache.org/jira/browse/HBASE-4555
 Project: HBase
  Issue Type: Test
  Components: test
Reporter: Mingjie Lai
Assignee: Mingjie Lai
 Attachments: 4555.diff


 When I was making test cases for 4554, I saw a weird issue that TestShell 
 seems to pass, but actually I saw error messages in the output file.
 {code}
 ---
  T E S T S
 ---
 Running org.apache.hadoop.hbase.client.TestShell
 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 39.252 sec
 Results :
 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
 {code}
 Error messages in org.apache.hadoop.hbase.client.TestShell-output.txt:
 {code}
 ...
   6) Error:
 test_alter_should_support_shortcut_DELETE_alter_specs(Hbase::AdminAlterTableTest):
 ArgumentError: There should be at least one argument but the table name
 
 /home/mlai/git/hbase-private/src/test/ruby/../../main/ruby/hbase/admin.rb:307:in
  `alter'
 ./src/test/ruby/hbase/admin_test.rb:271:in 
 `test_alter_should_support_shortcut_DELETE_alter_specs'
 org/jruby/RubyProc.java:268:in `call'
 org/jruby/RubyKernel.java:2038:in `send'
 org/jruby/RubyArray.java:1572:in `each'
 org/jruby/RubyArray.java:1572:in `each'
 
   7) Error:
 test_split_should_work(Hbase::AdminMethodsTest):
 ArgumentError: wrong number of arguments (1 for 2)
 ./src/test/ruby/hbase/admin_test.rb:99:in `test_split_should_work'
 org/jruby/RubyProc.java:268:in `call'
 org/jruby/RubyKernel.java:2038:in `send'
 org/jruby/RubyArray.java:1572:in `each'
 org/jruby/RubyArray.java:1572:in `each'
 
 192 tests, 259 assertions, 1 failures, 6 errors
 Done with tests! Shutting down the cluster...
 2011-10-07 16:46:14,760 INFO  [main] hbase.HBaseTestingUtility(551): Shutting 
 down minicluster
 2011-10-07 16:46:14,760 DEBUG [main] util.JVMClusterUtil(214): Shutting down 
 HBase Cluster
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4335) Splits can create temporary holes in .META. that confuse clients and regionservers

[
https://issues.apache.org/jira/browse/HBASE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13125492#comment-13125492
]

Hudson commented on HBASE-4335:
---

Integrated in HBase-0.92 #55 (See
[https://builds.apache.org/job/HBase-0.92/55/])
Integrating HBASE-4335

larsh :
Files :
* /hbase/branches/0.92/CHANGES.txt
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java
*
/hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestEndToEndSplitTransaction.java
*
/hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransaction.java

Splits can create temporary holes in .META. that confuse clients and
regionservers
--

Attachments: 4335-v2.txt, 4335-v3.txt, 4335-v4.txt, 4335-v5.txt,
4335.txt

[jira] [Commented] (HBASE-4489) Better key splitting in RegionSplitter

2011-10-11 Thread Dave Revell (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-4489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13125495#comment-13125495
]

Dave Revell commented on HBASE-4489:

@Nicolas, did you know HBase keys are lexicographically ordered?
http://wiki.apache.org/hadoop/Hbase/DataModel

Otherwise I can't see how variables (integer, long) are relevant to the problem.

Better key splitting in RegionSplitter
--

[jira] [Created] (HBASE-4579) Compactions that can't run are requested forever

2011-10-11 Thread Jean-Daniel Cryans (Created) (JIRA)

Compactions that can't run are requested forever


 Key: HBASE-4579
 URL: https://issues.apache.org/jira/browse/HBASE-4579
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Jean-Daniel Cryans
Priority: Critical
 Fix For: 0.92.0


Another bug I'm not so sure what's going on. I see this in my log:

{quote}
2011-10-12 00:23:43,435 DEBUG org.apache.hadoop.hbase.regionserver.Store: info: 
no store files to compact
2011-10-12 00:23:44,335 DEBUG org.apache.hadoop.hbase.regionserver.Store: info: 
no store files to compact
2011-10-12 00:23:45,236 DEBUG org.apache.hadoop.hbase.regionserver.Store: info: 
no store files to compact
2011-10-12 00:23:46,136 DEBUG org.apache.hadoop.hbase.regionserver.Store: info: 
no store files to compact
2011-10-12 00:23:47,036 DEBUG org.apache.hadoop.hbase.regionserver.Store: info: 
no store files to compact
2011-10-12 00:23:47,936 DEBUG org.apache.hadoop.hbase.regionserver.Store: info: 
no store files to compact
{quote}

It spams for a while, and a little later instead I get:

{quote}
2011-10-12 00:26:52,139 DEBUG org.apache.hadoop.hbase.regionserver.Store: 
Skipped compaction of info.  Only 2 file(s) of size 176.4m have met compaction 
criteria.
2011-10-12 00:26:53,040 DEBUG org.apache.hadoop.hbase.regionserver.Store: 
Skipped compaction of info.  Only 2 file(s) of size 176.4m have met compaction 
criteria.
2011-10-12 00:26:53,940 DEBUG org.apache.hadoop.hbase.regionserver.Store: 
Skipped compaction of info.  Only 2 file(s) of size 176.4m have met compaction 
criteria.
2011-10-12 00:26:54,840 DEBUG org.apache.hadoop.hbase.regionserver.Store: 
Skipped compaction of info.  Only 2 file(s) of size 176.4m have met compaction 
criteria.
2011-10-12 00:26:55,741 DEBUG org.apache.hadoop.hbase.regionserver.Store: 
Skipped compaction of info.  Only 2 file(s) of size 176.4m have met compaction 
criteria.
2011-10-12 00:26:56,641 DEBUG org.apache.hadoop.hbase.regionserver.Store: 
Skipped compaction of info.  Only 2 file(s) of size 176.4m have met compaction 
criteria.
{quote}

I believe I also saw something like that for flushes, but the region was 
closing so at least I know why it was spamming (would be nice if it just 
unrequested the flush):

{quote}
2011-10-12 00:26:40,693 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: 
Flush requested on 
TestTable,0038168581,1318378894213.2beb8a1e29382a8d3e90a88b9662e5f5.
2011-10-12 00:26:40,694 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: NOT 
flushing memstore for region 
TestTable,0038168581,1318378894213.2beb8a1e29382a8d3e90a88b9662e5f5., 
flushing=false, writesEnabled=false
2011-10-12 00:26:40,733 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: 
Flush requested on 
TestTable,0038168581,1318378894213.2beb8a1e29382a8d3e90a88b9662e5f5.
2011-10-12 00:26:40,733 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: NOT 
flushing memstore for region 
TestTable,0038168581,1318378894213.2beb8a1e29382a8d3e90a88b9662e5f5., 
flushing=false, writesEnabled=false
2011-10-12 00:26:40,873 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: 
Flush requested on 
TestTable,0038168581,1318378894213.2beb8a1e29382a8d3e90a88b9662e5f5.
2011-10-12 00:26:40,873 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: NOT 
flushing memstore for region 
TestTable,0038168581,1318378894213.2beb8a1e29382a8d3e90a88b9662e5f5., 
flushing=false, writesEnabled=false
2011-10-12 00:26:40,873 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: 
Flush requested on 
TestTable,0038168581,1318378894213.2beb8a1e29382a8d3e90a88b9662e5f5.
2011-10-12 00:26:40,873 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: NOT 
flushing memstore for region 
TestTable,0038168581,1318378894213.2beb8a1e29382a8d3e90a88b9662e5f5., 
flushing=false, writesEnabled=false
2011-10-12 00:26:40,921 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: 
Flush requested on 
TestTable,0038168581,1318378894213.2beb8a1e29382a8d3e90a88b9662e5f5.
2011-10-12 00:26:40,922 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: NOT 
flushing memstore for region 
TestTable,0038168581,1318378894213.2beb8a1e29382a8d3e90a88b9662e5f5., 
flushing=false, writesEnabled=false
2011-10-12 00:26:40,923 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: 
Flush requested on 
TestTable,0038168581,1318378894213.2beb8a1e29382a8d3e90a88b9662e5f5.
2011-10-12 00:26:40,923 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: NOT 
flushing memstore for region 
TestTable,0038168581,1318378894213.2beb8a1e29382a8d3e90a88b9662e5f5., 
flushing=false, writesEnabled=false
2011-10-12 00:26:40,923 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: 
Flush requested on 
TestTable,0038168581,1318378894213.2beb8a1e29382a8d3e90a88b9662e5f5.
2011-10-12 00:26:40,923 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: NOT 
flushing memstore for region

[jira] [Commented] (HBASE-4457) Starting region server on non-default info port is resulting in broken URL's in master UI

2011-10-11 Thread Jean-Daniel Cryans (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13125509#comment-13125509
 ] 

Jean-Daniel Cryans commented on HBASE-4457:
---

A temporary fix could be done by using the configuration directly instead of 
asking the region server. Most of the time the master and RSs share that same 
file so it should cover a lot more cases than the current code does.

 Starting region server on non-default info port is resulting in broken URL's 
 in master UI
 -

 Key: HBASE-4457
 URL: https://issues.apache.org/jira/browse/HBASE-4457
 Project: HBase
  Issue Type: Bug
  Components: master
Affects Versions: 0.92.0
Reporter: Praveen Patibandla
Priority: Minor
  Labels: newbie
 Fix For: 0.92.0

 Attachments: 4457-V1.patch, 4457.patch


 When  hbase.regionserver.info.port is set to non-default port, Master UI 
 has broken URL's in the region server table because it's hard coded to 
 default port.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-4580) Create some invalid zk nodes when a clean cluster start.

2011-10-11 Thread gaojinchao (Created) (JIRA)

Create some invalid zk nodes when a clean cluster start.


 Key: HBASE-4580
 URL: https://issues.apache.org/jira/browse/HBASE-4580
 Project: HBase
  Issue Type: Bug
  Components: master
Affects Versions: 0.92.0
Reporter: gaojinchao
 Fix For: 0.92.0


The below logs said that we created a invalid zk node when restarted a cluster.
it mistakenly believed that the regions belong to a dead server.

2011-10-11 05:05:29,127 INFO org.apache.hadoop.hbase.master.HMaster: Meta 
updated status = true
2011-10-11 05:05:29,127 INFO org.apache.hadoop.hbase.master.HMaster: ROOT/Meta 
already up-to date with new HRI.
2011-10-11 05:05:29,151 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
master:58198-0x132f23a9a38 Creating (or updating) unassigned node for 
771d63e9327383159553619a4f2dc74f with OFFLINE state
2011-10-11 05:05:29,161 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
master:58198-0x132f23a9a38 Creating (or updating) unassigned node for 
3cf860dd323fe6360f571aeafc129f95 with OFFLINE state
2011-10-11 05:05:29,170 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
master:58198-0x132f23a9a38 Creating (or updating) unassigned node for 
4065350214452a9d5c55243c734bef08 with OFFLINE state
2011-10-11 05:05:29,178 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
master:58198-0x132f23a9a38 Creating (or updating) unassigned node for 
4e81613f82a39fc6e5e89f96e7b3ccc4 with OFFLINE state
2011-10-11 05:05:29,187 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
master:58198-0x132f23a9a38 Creating (or updating) unassigned node for 
e21b9e1545a28953aba0098fda5c9cd9 with OFFLINE state
2011-10-11 05:05:29,195 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
master:58198-0x132f23a9a38 Creating (or updating) unassigned node for 
5cd9f55eecd43d088bbd505f6795131f with OFFLINE state
2011-10-11 05:05:29,229 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
master:58198-0x132f23a9a38 Creating (or updating) unassigned node for 
db5f641452a70b09b85a92970e4198c7 with OFFLINE state
2011-10-11 05:05:29,237 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
master:58198-0x132f23a9a38 Creating (or updating) unassigned node for 
a7b20a653919e7f41bfb2ed349af7d21 with OFFLINE state
2011-10-11 05:05:29,253 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
master:58198-0x132f23a9a38 Creating (or updating) unassigned node for 
c9385619425f737eab1a6624d2e097a8 with OFFLINE state

// we cleaned all zk nodes.
2011-10-11 05:05:29,262 INFO org.apache.hadoop.hbase.master.AssignmentManager: 
Clean cluster startup. Assigning userregions
2011-10-11 05:05:29,262 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
master:58198-0x132f23a9a38 Deleting any existing unassigned nodes
2011-10-11 05:05:29,367 INFO org.apache.hadoop.hbase.master.AssignmentManager: 
Bulk assigning 9 region(s) across 1 server(s), retainAssignment=true
2011-10-11 05:05:29,369 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: 
Timeout-on-RIT=9000
2011-10-11 05:05:29,369 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: 
Bulk assigning 9 region(s) to C3S3,54366,1318323920153
2011-10-11 05:05:29,369 INFO org.apache.hadoop.hbase.master.AssignmentManager: 
Bulk assigning done
2011-10-11 05:05:29,371 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
master:58198-0x132f23a9a38 Async create of unassigned node for 
771d63e9327383159553619a4f2dc74f with OFFLINE state
2011-10-11 05:05:29,371 INFO org.apache.hadoop.hbase.master.HMaster: Master has 
completed initialization
2011-10-11 05:05:29,371 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
master:58198-0x132f23a9a38 Async create of unassigned node for 
3cf860dd323fe6360f571aeafc129f95 with OFFLINE state
2011-10-11 05:05:29,371 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
master:58198-0x132f23a9a38 Async create of unassigned node for 
4065350214452a9d5c55243c734bef08 with OFFLINE state
2011-10-11 05:05:29,371 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
master:58198-0x132f23a9a38 Async create of unassigned node for 
4e81613f82a39fc6e5e89f96e7b3ccc4 with OFFLINE state
2011-10-11 05:05:29,371 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
master:58198-0x132f23a9a38 Async create of unassigned node for 
e21b9e1545a28953aba0098fda5c9cd9 with OFFLINE state
2011-10-11 05:05:29,372 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
master:58198-0x132f23a9a38 Async create of unassigned node for 
5cd9f55eecd43d088bbd505f6795131f with OFFLINE state
2011-10-11 05:05:29,372 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
master:58198-0x132f23a9a38 Async create of unassigned node for 
db5f641452a70b09b85a92970e4198c7 with OFFLINE state
2011-10-11 05:05:29,372 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
master:58198-0x132f23a9a38 Async create of unassigned node for

[jira] [Assigned] (HBASE-4580) Create some invalid zk nodes when a clean cluster start.

2011-10-11 Thread gaojinchao (Assigned) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-4580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

gaojinchao reassigned HBASE-4580:
-

Assignee: gaojinchao

 Create some invalid zk nodes when a clean cluster start.
 

 Key: HBASE-4580
 URL: https://issues.apache.org/jira/browse/HBASE-4580
 Project: HBase
  Issue Type: Bug
  Components: master
Affects Versions: 0.92.0
Reporter: gaojinchao
Assignee: gaojinchao
 Fix For: 0.92.0


 The below logs said that we created a invalid zk node when restarted a 
 cluster.
 it mistakenly believed that the regions belong to a dead server.
 2011-10-11 05:05:29,127 INFO org.apache.hadoop.hbase.master.HMaster: Meta 
 updated status = true
 2011-10-11 05:05:29,127 INFO org.apache.hadoop.hbase.master.HMaster: 
 ROOT/Meta already up-to date with new HRI.
 2011-10-11 05:05:29,151 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 master:58198-0x132f23a9a38 Creating (or updating) unassigned node for 
 771d63e9327383159553619a4f2dc74f with OFFLINE state
 2011-10-11 05:05:29,161 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 master:58198-0x132f23a9a38 Creating (or updating) unassigned node for 
 3cf860dd323fe6360f571aeafc129f95 with OFFLINE state
 2011-10-11 05:05:29,170 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 master:58198-0x132f23a9a38 Creating (or updating) unassigned node for 
 4065350214452a9d5c55243c734bef08 with OFFLINE state
 2011-10-11 05:05:29,178 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 master:58198-0x132f23a9a38 Creating (or updating) unassigned node for 
 4e81613f82a39fc6e5e89f96e7b3ccc4 with OFFLINE state
 2011-10-11 05:05:29,187 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 master:58198-0x132f23a9a38 Creating (or updating) unassigned node for 
 e21b9e1545a28953aba0098fda5c9cd9 with OFFLINE state
 2011-10-11 05:05:29,195 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 master:58198-0x132f23a9a38 Creating (or updating) unassigned node for 
 5cd9f55eecd43d088bbd505f6795131f with OFFLINE state
 2011-10-11 05:05:29,229 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 master:58198-0x132f23a9a38 Creating (or updating) unassigned node for 
 db5f641452a70b09b85a92970e4198c7 with OFFLINE state
 2011-10-11 05:05:29,237 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 master:58198-0x132f23a9a38 Creating (or updating) unassigned node for 
 a7b20a653919e7f41bfb2ed349af7d21 with OFFLINE state
 2011-10-11 05:05:29,253 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 master:58198-0x132f23a9a38 Creating (or updating) unassigned node for 
 c9385619425f737eab1a6624d2e097a8 with OFFLINE state
 // we cleaned all zk nodes.
 2011-10-11 05:05:29,262 INFO 
 org.apache.hadoop.hbase.master.AssignmentManager: Clean cluster startup. 
 Assigning userregions
 2011-10-11 05:05:29,262 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 master:58198-0x132f23a9a38 Deleting any existing unassigned nodes
 2011-10-11 05:05:29,367 INFO 
 org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 9 region(s) 
 across 1 server(s), retainAssignment=true
 2011-10-11 05:05:29,369 DEBUG 
 org.apache.hadoop.hbase.master.AssignmentManager: Timeout-on-RIT=9000
 2011-10-11 05:05:29,369 DEBUG 
 org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 9 region(s) 
 to C3S3,54366,1318323920153
 2011-10-11 05:05:29,369 INFO 
 org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done
 2011-10-11 05:05:29,371 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 master:58198-0x132f23a9a38 Async create of unassigned node for 
 771d63e9327383159553619a4f2dc74f with OFFLINE state
 2011-10-11 05:05:29,371 INFO org.apache.hadoop.hbase.master.HMaster: Master 
 has completed initialization
 2011-10-11 05:05:29,371 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 master:58198-0x132f23a9a38 Async create of unassigned node for 
 3cf860dd323fe6360f571aeafc129f95 with OFFLINE state
 2011-10-11 05:05:29,371 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 master:58198-0x132f23a9a38 Async create of unassigned node for 
 4065350214452a9d5c55243c734bef08 with OFFLINE state
 2011-10-11 05:05:29,371 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 master:58198-0x132f23a9a38 Async create of unassigned node for 
 4e81613f82a39fc6e5e89f96e7b3ccc4 with OFFLINE state
 2011-10-11 05:05:29,371 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 master:58198-0x132f23a9a38 Async create of unassigned node for 
 e21b9e1545a28953aba0098fda5c9cd9 with OFFLINE state
 2011-10-11 05:05:29,372 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 master:58198-0x132f23a9a38 Async create of unassigned node for 
 5cd9f55eecd43d088bbd505f6795131f with OFFLINE state
 2011-10-11 05:05:29,372 DEBUG

[jira] [Commented] (HBASE-4047) [Coprocessors] Generic external process host

2011-10-11 Thread Andrew Purtell (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13125523#comment-13125523
 ] 

Andrew Purtell commented on HBASE-4047:
---

Getting to this a bit late, thinking about design.

Here are some possible motivating cases:

  - A hot value cache implemented in C/C++

  - Indexing and search with Lucene indexes hosted on a colocated (impl 
bundled/linked with the external coprocessor and private to it) R+W distributed 
FS like Gluster

  - Support something we are building internally that requires efficient hand 
off of HFiles between processes for compaction strategy override.

Suggestions welcome, preferably useful to real activities you may be 
undertaking.


 [Coprocessors] Generic external process host
 

 Key: HBASE-4047
 URL: https://issues.apache.org/jira/browse/HBASE-4047
 Project: HBase
  Issue Type: New Feature
  Components: coprocessors
Reporter: Andrew Purtell
Assignee: Andrew Purtell

 Where HBase coprocessors deviate substantially from the design (as I 
 understand it) of Google's BigTable coprocessors is we've reimagined it as a 
 framework for internal extension. In contrast BigTable coprocessors run as 
 separate processes colocated with tablet servers. The essential trade off is 
 between performance, flexibility and possibility, and the ability to control 
 and enforce resource usage.
 Since the initial design of HBase coprocessors some additional considerations 
 are in play:
 - Developing computational frameworks sitting directly on top of HBase hosted 
 in coprocessor(s);
 - Introduction of the map reduce next generation (mrng) resource management 
 model, and the probability that limits will be enforced via cgroups at the OS 
 level after this is generally available, e.g. when RHEL 6 deployments are 
 common;
 - The possibility of deployment of HBase onto mrng-enabled Hadoop clusters 
 via the mrng resource manager and a HBase-specific application controller.
 Therefore we should consider developing a coprocessor that is a generic host 
 for another coprocessor, but one that forks a child process, loads the target 
 coprocessor into the child, establishes a bidirectional pipe and uses an 
 eventing model and umbilical protocol to provide for the coprocessor loaded 
 into the child the same semantics as if it was loaded internally to the 
 parent, and (eventually) use available resource management capabilities on 
 the platform -- perhaps via the mrng resource controller or directly with 
 cgroups -- to limit the child as desired by system administrators or the 
 application designer.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-1744) Thrift server to match the new java api.

2011-10-11 Thread Tim Sell (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tim Sell updated HBASE-1744:


Attachment: HBASE-1744.9.patch

Looks good. I added another patch which also sets TDelete to use DELETE_COLUMNS 
by default and added some tests.

 Thrift server to match the new java api.
 

 Key: HBASE-1744
 URL: https://issues.apache.org/jira/browse/HBASE-1744
 Project: HBase
  Issue Type: Improvement
  Components: thrift
Reporter: Tim Sell
Assignee: Tim Sell
Priority: Critical
 Fix For: 0.94.0

 Attachments: 
 0001-thrift2-enable-usage-of-.deleteColumns-for-thrift.patch, 
 HBASE-1744.2.patch, HBASE-1744.3.patch, HBASE-1744.4.patch, 
 HBASE-1744.5.patch, HBASE-1744.6.patch, HBASE-1744.7.patch, 
 HBASE-1744.8.patch, HBASE-1744.9.patch, HBASE-1744.preview.1.patch, 
 thriftexperiment.patch


 This mutateRows, etc.. is a little confusing compared to the new cleaner java 
 client.
 Thinking of ways to make a thrift client that is just as elegant. something 
 like:
 void put(1:Bytes table, 2:TPut put) throws (1:IOError io)
 with:
 struct TColumn {
   1:Bytes family,
   2:Bytes qualifier,
   3:i64 timestamp
 }
 struct TPut {
   1:Bytes row,
   2:mapTColumn, Bytes values
 }
 This creates more verbose rpc  than if the columns in TPut were just 
 mapBytes, mapBytes, Bytes, but that is harder to fit timestamps into and 
 still be intuitive from say python.
 Presumably the goal of a thrift gateway is to be easy first.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4576) [book] moved Cluster Replication chapter to OpsMgt, other nit


[ 
https://issues.apache.org/jira/browse/HBASE-4576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13125540#comment-13125540
 ] 

Hudson commented on HBASE-4576:
---

Integrated in HBase-TRUNK #2318 (See 
[https://builds.apache.org/job/HBase-TRUNK/2318/])
HBASE-4576 book.xml,ops_mgt.xml - moved cluster repl into OpsMgt chapter.

dmeil : 
Files : 
* /hbase/trunk/src/docbkx/book.xml
* /hbase/trunk/src/docbkx/ops_mgt.xml


 [book] moved Cluster Replication chapter to OpsMgt, other nit
 -

 Key: HBASE-4576
 URL: https://issues.apache.org/jira/browse/HBASE-4576
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Trivial
 Attachments: docbkx_HBASE_4576.patch


 book.xml
 * There was a placeholder chapter for Cluster Replication that just linked to 
 replication.html.  I moved this to a section in the Ops Mgt chapter.
 * Nit:  in Schema Design Smackdown sub-section, changed Winner to 
 Preference.  I got a little carried away when I created that entry and I'm 
 toning it down a tad.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4335) Splits can create temporary holes in .META. that confuse clients and regionservers

2011-10-11 Thread jirapos...@reviews.apache.org (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13125539#comment-13125539
]

Hudson commented on HBASE-4335:
---

Integrated in HBase-TRUNK #2318 (See
[https://builds.apache.org/job/HBase-TRUNK/2318/])
Integrating HBASE-4335

larsh :
Files :
* /hbase/trunk/CHANGES.txt
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java
*
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestEndToEndSplitTransaction.java
*
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransaction.java

Splits can create temporary holes in .META. that confuse clients and
regionservers
--

Attachments: 4335-v2.txt, 4335-v3.txt, 4335-v4.txt, 4335-v5.txt,
4335.txt

[jira] [Commented] (HBASE-4422) Move block cache parameters and references into single CacheConf class


[ 
https://issues.apache.org/jira/browse/HBASE-4422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13125543#comment-13125543
 ] 

jirapos...@reviews.apache.org commented on HBASE-4422:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2089/#review2526
---

Ship it!


1032 unit tests passed, 0 failed, 16 skipped (I still don't know why Maven 
skips tests and whether it is a problem -- any idea?)
I think this patch is good to go.

- Mikhail


On 2011-10-11 22:18:32, Jonathan Gray wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/2089/
bq.  ---
bq.  
bq.  (Updated 2011-10-11 22:18:32)
bq.  
bq.  
bq.  Review request for hbase, Dhruba Borthakur, Michael Stack, and Li Pi.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  Creates a new CacheConfig class and moves almost everything block cache 
related into this single class.  Adding new configuration params and booleans 
and such should be much better.
bq.  
bq.  All tests are NOT passing yet, still working on it, but wanted to have 
something up today.  Basically code complete but broken :)
bq.  
bq.  
bq.  This addresses bug HBASE-4422.
bq.  https://issues.apache.org/jira/browse/HBASE-4422
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq./src/main/java/org/apache/hadoop/hbase/io/HalfStoreFileReader.java 
1182021 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileReader.java 
1182021 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileWriter.java 
1182021 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/BlockCache.java 1182021 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java 
PRE-CREATION 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/DoubleBlockCache.java 
1182021 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java 1182021 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFilePrettyPrinter.java 
1182021 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV1.java 
1182021 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java 
1182021 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV1.java 
1182021 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java 
1182021 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java 
1182021 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/SimpleBlockCache.java 
1182021 
bq.
/src/main/java/org/apache/hadoop/hbase/io/hfile/slab/SingleSizeCache.java 
1182021 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/slab/SlabCache.java 
1182021 
bq./src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java 
1182021 
bq.
/src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 
1182021 
bq./src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1182021 
bq./src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 
1182021 
bq./src/main/java/org/apache/hadoop/hbase/regionserver/Store.java 1182021 
bq./src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java 
1182021 
bq./src/main/java/org/apache/hadoop/hbase/util/BloomFilterFactory.java 
1182021 
bq./src/main/java/org/apache/hadoop/hbase/util/CompressionTest.java 1182021 
bq./src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java 
1182021 
bq./src/test/java/org/apache/hadoop/hbase/io/TestHalfStoreFileReader.java 
1182021 
bq./src/test/java/org/apache/hadoop/hbase/io/hfile/RandomSeek.java 1182021 
bq./src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java 
1182021 
bq./src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFile.java 1182021 
bq./src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlockIndex.java 
1182021 
bq.
/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFilePerformance.java 
1182021 
bq./src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileReaderV1.java 
1182021 
bq./src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileSeek.java 
1182021 
bq./src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileWriterV2.java 
1182021 
bq./src/test/java/org/apache/hadoop/hbase/io/hfile/TestReseekTo.java 
1182021 
bq./src/test/java/org/apache/hadoop/hbase/io/hfile/TestSeekTo.java 1182021 
bq.
/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java 
1182021 
bq.
/src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFiles.java 
1182021 
bq./src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksRead.java 
1182021 
bq.

[jira] [Resolved] (HBASE-4572) Multithread compaction in regionserver

2011-10-11 Thread Aaron Guo (Resolved) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-4572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aaron Guo resolved HBASE-4572.
--

Resolution: Duplicate

 Multithread compaction in regionserver
 --

 Key: HBASE-4572
 URL: https://issues.apache.org/jira/browse/HBASE-4572
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Affects Versions: 0.90.4
Reporter: Aaron Guo
 Fix For: 0.92.0


 RegionServer only have one thread doing region compaction staff.
 Could We adjust the RegionServer compation thread num and priority online. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4572) Multithread compaction in regionserver

2011-10-11 Thread Aaron Guo (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13125547#comment-13125547
 ] 

Aaron Guo commented on HBASE-4572:
--

yes, thanks.

 Multithread compaction in regionserver
 --

 Key: HBASE-4572
 URL: https://issues.apache.org/jira/browse/HBASE-4572
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Affects Versions: 0.90.4
Reporter: Aaron Guo
 Fix For: 0.92.0


 RegionServer only have one thread doing region compaction staff.
 Could We adjust the RegionServer compation thread num and priority online. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4555) TestShell seems passed, but actually errors seen in test output file

2011-10-11 Thread Mingjie Lai (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-4555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mingjie Lai updated HBASE-4555:
---

Attachment: HBase-4555-trunk-0.92.patch

 TestShell seems passed, but actually errors seen in test output file
 

 Key: HBASE-4555
 URL: https://issues.apache.org/jira/browse/HBASE-4555
 Project: HBase
  Issue Type: Test
  Components: test
Reporter: Mingjie Lai
Assignee: Mingjie Lai
 Attachments: 4555.diff, HBase-4555-trunk-0.92.patch


 When I was making test cases for 4554, I saw a weird issue that TestShell 
 seems to pass, but actually I saw error messages in the output file.
 {code}
 ---
  T E S T S
 ---
 Running org.apache.hadoop.hbase.client.TestShell
 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 39.252 sec
 Results :
 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
 {code}
 Error messages in org.apache.hadoop.hbase.client.TestShell-output.txt:
 {code}
 ...
   6) Error:
 test_alter_should_support_shortcut_DELETE_alter_specs(Hbase::AdminAlterTableTest):
 ArgumentError: There should be at least one argument but the table name
 
 /home/mlai/git/hbase-private/src/test/ruby/../../main/ruby/hbase/admin.rb:307:in
  `alter'
 ./src/test/ruby/hbase/admin_test.rb:271:in 
 `test_alter_should_support_shortcut_DELETE_alter_specs'
 org/jruby/RubyProc.java:268:in `call'
 org/jruby/RubyKernel.java:2038:in `send'
 org/jruby/RubyArray.java:1572:in `each'
 org/jruby/RubyArray.java:1572:in `each'
 
   7) Error:
 test_split_should_work(Hbase::AdminMethodsTest):
 ArgumentError: wrong number of arguments (1 for 2)
 ./src/test/ruby/hbase/admin_test.rb:99:in `test_split_should_work'
 org/jruby/RubyProc.java:268:in `call'
 org/jruby/RubyKernel.java:2038:in `send'
 org/jruby/RubyArray.java:1572:in `each'
 org/jruby/RubyArray.java:1572:in `each'
 
 192 tests, 259 assertions, 1 failures, 6 errors
 Done with tests! Shutting down the cluster...
 2011-10-07 16:46:14,760 INFO  [main] hbase.HBaseTestingUtility(551): Shutting 
 down minicluster
 2011-10-07 16:46:14,760 DEBUG [main] util.JVMClusterUtil(214): Shutting down 
 HBase Cluster
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4555) TestShell seems passed, but actually errors seen in test output file


[ 
https://issues.apache.org/jira/browse/HBASE-4555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13125554#comment-13125554
 ] 

Mingjie Lai commented on HBASE-4555:


Final patch attached. Thanks. 

 TestShell seems passed, but actually errors seen in test output file
 

 Key: HBASE-4555
 URL: https://issues.apache.org/jira/browse/HBASE-4555
 Project: HBase
  Issue Type: Test
  Components: test
Reporter: Mingjie Lai
Assignee: Mingjie Lai
 Attachments: 4555.diff, HBase-4555-trunk-0.92.patch


 When I was making test cases for 4554, I saw a weird issue that TestShell 
 seems to pass, but actually I saw error messages in the output file.
 {code}
 ---
  T E S T S
 ---
 Running org.apache.hadoop.hbase.client.TestShell
 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 39.252 sec
 Results :
 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
 {code}
 Error messages in org.apache.hadoop.hbase.client.TestShell-output.txt:
 {code}
 ...
   6) Error:
 test_alter_should_support_shortcut_DELETE_alter_specs(Hbase::AdminAlterTableTest):
 ArgumentError: There should be at least one argument but the table name
 
 /home/mlai/git/hbase-private/src/test/ruby/../../main/ruby/hbase/admin.rb:307:in
  `alter'
 ./src/test/ruby/hbase/admin_test.rb:271:in 
 `test_alter_should_support_shortcut_DELETE_alter_specs'
 org/jruby/RubyProc.java:268:in `call'
 org/jruby/RubyKernel.java:2038:in `send'
 org/jruby/RubyArray.java:1572:in `each'
 org/jruby/RubyArray.java:1572:in `each'
 
   7) Error:
 test_split_should_work(Hbase::AdminMethodsTest):
 ArgumentError: wrong number of arguments (1 for 2)
 ./src/test/ruby/hbase/admin_test.rb:99:in `test_split_should_work'
 org/jruby/RubyProc.java:268:in `call'
 org/jruby/RubyKernel.java:2038:in `send'
 org/jruby/RubyArray.java:1572:in `each'
 org/jruby/RubyArray.java:1572:in `each'
 
 192 tests, 259 assertions, 1 failures, 6 errors
 Done with tests! Shutting down the cluster...
 2011-10-07 16:46:14,760 INFO  [main] hbase.HBaseTestingUtility(551): Shutting 
 down minicluster
 2011-10-07 16:46:14,760 DEBUG [main] util.JVMClusterUtil(214): Shutting down 
 HBase Cluster
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4558) Refactor TestOpenedRegionHandler and TestOpenRegionHandler.