date:20111004

2011-10-04 Thread ramkrishna.s.vasudevan (Updated) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-4536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13119932#comment-13119932
]

Lars Hofhansl commented on HBASE-4536:
--

There's a fourth scan type:
4. Memstore flush (see HBASE-4241). This behaves exactly like #2.

There is a pathetic scenario where CF has minversions set and a client only
write delete markers. As they are not counted as versions, they would never be
removed (not even in a major compaction).
I suppose to guard against that case we could remove all trailing delete
markers (that are not followed by any surviving kvs).

Allow CF to retain deleted rows
---

Key: HBASE-4536
URL: https://issues.apache.org/jira/browse/HBASE-4536
Project: HBase
Issue Type: Sub-task
Components: regionserver
Affects Versions: 0.92.0
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Fix For: 0.92.0, 0.94.0

Parent allows for a cluster to retain rows for a TTL or keep a minimum number
of versions.
However, if a client deletes a row all version older than the delete tomb
stone will be remove at the next major compaction (and even at memstore flush
- see HBASE-4241).
There should be a way to retain those version to guard against software error.
I see two options here:
1. Add a new flag HColumnDescriptor. Something like RETAIN_DELETED.
2. Folds this into the parent change. I.e. keep minimum-number-of-versions of
versions even past the delete marker.
#1 would allow for more flexibility. #2 comes somewhat naturally with parent
(from a user viewpoint)
Comments? Any other options?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4479) TestMasterFailover failure in Hbase-0.92#17


 [ 
https://issues.apache.org/jira/browse/HBASE-4479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-4479:
--

Attachment: HBASE-4479.patch

Kindly review the patch.  If it is fine i can commit it.
I have run all the testcases.  And all are working fine.

 TestMasterFailover failure in Hbase-0.92#17
 ---

 Key: HBASE-4479
 URL: https://issues.apache.org/jira/browse/HBASE-4479
 Project: HBase
  Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Minor
 Attachments: HBASE-4479.patch


 When the master restarted it was not able to get any servers online and the 
 restart was a clean restart.
 Hence there were no regions to assign.
 Hence the retainAssignment tries to get one of the regions and uses 
 RANDOM.getInt(size).  Here size is 0.
 So ideally 0 is not accepted here.  Hence we have got an exception making the 
 master not to come up and the test case timeout.
 Though we need to see if really no regions was expected when the master came 
 up, but this JIRA's intent is to deal such scenario where the size can be 0.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4479) TestMasterFailover failure in Hbase-0.92#17


[ 
https://issues.apache.org/jira/browse/HBASE-4479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120040#comment-13120040
 ] 

Ted Yu commented on HBASE-4479:
---

+1 on patch. 

 TestMasterFailover failure in Hbase-0.92#17
 ---

 Key: HBASE-4479
 URL: https://issues.apache.org/jira/browse/HBASE-4479
 Project: HBase
  Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Minor
 Attachments: HBASE-4479.patch


 When the master restarted it was not able to get any servers online and the 
 restart was a clean restart.
 Hence there were no regions to assign.
 Hence the retainAssignment tries to get one of the regions and uses 
 RANDOM.getInt(size).  Here size is 0.
 So ideally 0 is not accepted here.  Hence we have got an exception making the 
 master not to come up and the test case timeout.
 Though we need to see if really no regions was expected when the master came 
 up, but this JIRA's intent is to deal such scenario where the size can be 0.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4528) The put operation can release the rowlock before sync-ing the Hlog


[ 
https://issues.apache.org/jira/browse/HBASE-4528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120170#comment-13120170
 ] 

Ted Yu commented on HBASE-4528:
---

See the following in HLog.syncer(long txid) which is called by sync():
{code}
} catch (IOException e) {
  LOG.fatal(Could not sync. Requesting close of hlog, e);
  requestLogRoll();
  throw e;
}
{code}

 The put operation can release the rowlock before sync-ing the Hlog
 --

 Key: HBASE-4528
 URL: https://issues.apache.org/jira/browse/HBASE-4528
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Reporter: dhruba borthakur
Assignee: dhruba borthakur
 Attachments: appendNoSyncPut1.txt, appendNoSyncPut2.txt


 This allows for better throughput when there are hot rows. A single row 
 update improves from 100 puts/sec/server to 5000 puts/sec/server.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4531) hbase-4454 failsafe broke mvn site; back it out or fix

2011-10-04 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-4531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-4531:
-

Resolution: Fixed
Status: Resolved  (was: Patch Available)

Applied v3

 hbase-4454 failsafe broke mvn site; back it out or fix
 --

 Key: HBASE-4531
 URL: https://issues.apache.org/jira/browse/HBASE-4531
 Project: HBase
  Issue Type: Bug
Reporter: stack
Assignee: Akash Ashok
Priority: Blocker
 Fix For: 0.92.0

 Attachments: HBase-4531-v2.patch, HBase-4531-v2.patch, 
 HBase-4531-v3.patch, HBase-4531.patch


 mvn site is broke in head of trunk.  If I back out the last pom change it 
 works again:
 {code}
 
 r1177168 | stack | 2011-09-29 05:42:13 + (Thu, 29 Sep 2011) | 1 line
 HBASE-4454 Add failsafe plugin to build and rename integration tests
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4489) Better key splitting in RegionSplitter

2011-10-04 Thread Dave Revell (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-4489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120277#comment-13120277
]

Dave Revell commented on HBASE-4489:

I haven't forgotten about this. I'll write the tests as soon as I can make
time, hopefully by the end of the week.

Better key splitting in RegionSplitter
--

Key: HBASE-4489
URL: https://issues.apache.org/jira/browse/HBASE-4489
Project: HBase
Issue Type: Improvement
Affects Versions: 0.90.4
Reporter: Dave Revell
Assignee: Dave Revell
Attachments: HBASE-4489-branch0.90-v1.patch, HBASE-4489-trunk-v1.patch

The RegionSplitter utility allows users to create a pre-split table from the
command line or do a rolling split on an existing table. It supports
pluggable split algorithms that implement the SplitAlgorithm interface. The
only/default SplitAlgorithm is one that assumes keys fall in the range from
ASCII string to ASCII string 7FFF. This is not a sane
default, and seems useless to most users. Users are likely to be surprised by
the fact that all the region splits occur in in the byte range of ASCII
characters.
A better default split algorithm would be one that evenly divides the space
of all bytes, which is what this patch does. Making a table with five regions
would split at \x33\x33..., \x66\x66, \x99\x99..., \xCC\xCC..., and
\xFF\xFF.

[jira] [Reopened] (HBASE-4447) Allow hbase.version to be passed in as command-line argument

2011-10-04 Thread Eric Yang (Reopened) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-4447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Yang reopened HBASE-4447:
--


 Allow hbase.version to be passed in as command-line argument
 

 Key: HBASE-4447
 URL: https://issues.apache.org/jira/browse/HBASE-4447
 Project: HBase
  Issue Type: Improvement
  Components: build
Affects Versions: 0.92.0
Reporter: Joep Rottinghuis
Assignee: Joep Rottinghuis
 Fix For: 0.92.0

 Attachments: HBASE-4447-0.92.patch


 Currently the build always produces the jars and tarball according to the 
 version baked into the POM.
 When we modify this to allow the version to be passed in as a command-line 
 argument, it can still default to the same behavior, yet give the flexibility 
 for an internal build to tag on own version.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4447) Allow hbase.version to be passed in as command-line argument

2011-10-04 Thread Eric Yang (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120344#comment-13120344
 ] 

Eric Yang commented on HBASE-4447:
--

Maven requires fixed version number, or the maven repository will be tainted 
with ${hbase.version} instead of 0.92.x.  The proper method to control version 
number is to use Versions Maven Plugin.  see: 

http://mojo.codehaus.org/versions-maven-plugin/

This patch should be reverted.

 Allow hbase.version to be passed in as command-line argument
 

 Key: HBASE-4447
 URL: https://issues.apache.org/jira/browse/HBASE-4447
 Project: HBase
  Issue Type: Improvement
  Components: build
Affects Versions: 0.92.0
Reporter: Joep Rottinghuis
Assignee: Joep Rottinghuis
 Fix For: 0.92.0

 Attachments: HBASE-4447-0.92.patch


 Currently the build always produces the jars and tarball according to the 
 version baked into the POM.
 When we modify this to allow the version to be passed in as a command-line 
 argument, it can still default to the same behavior, yet give the flexibility 
 for an internal build to tag on own version.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4520) Better handling of Bloom filter type discrepancy between HFile and CF config


[ 
https://issues.apache.org/jira/browse/HBASE-4520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120353#comment-13120353
 ] 

jirapos...@reviews.apache.org commented on HBASE-4520:
--



bq.  On 2011-10-03 16:57:44, Michael Stack wrote:
bq.   +1
bq.   
bq.   Had a sense of deja vu reviewing so maybe this is part of another patch 
already?  Maybe not.  Should go in regardless.

@Michael or Ted: would you mind committing this then? :)


- Mikhail


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2143/#review2266
---


On 2011-10-02 08:05:42, Mikhail Bautin wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/2143/
bq.  ---
bq.  
bq.  (Updated 2011-10-02 08:05:42)
bq.  
bq.  
bq.  Review request for hbase, Michael Stack and Jonathan Gray.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  Modify StoreFile to make it clear where Bloom filter type settings come 
from. We have two sources of truth: (1) HFile; and (2) CF configuration. (1) 
takes precedence in the reader, and (2) takes precedence in the writer.
bq.  
bq.  
bq.  This addresses bug HBASE-4520.
bq.  https://issues.apache.org/jira/browse/HBASE-4520
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java 
fa629c7 
bq.  
bq.  Diff: https://reviews.apache.org/r/2143/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  Running unit tests. I will post the results here -- please don't commit 
this yet.
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Mikhail
bq.  
bq.



 Better handling of Bloom filter type discrepancy between HFile and CF config
 

 Key: HBASE-4520
 URL: https://issues.apache.org/jira/browse/HBASE-4520
 Project: HBase
  Issue Type: Improvement
Reporter: Mikhail Bautin
Assignee: Mikhail Bautin
Priority: Minor
 Fix For: 0.94.0


 Modify StoreFile to make it clear where Bloom filter type settings come from. 
 We have two sources of truth: (1) HFile; and (2) CF configuration. (1) takes 
 precedence in the reader, and (2) takes precedence in the writer.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4520) Better handling of Bloom filter type discrepancy between HFile and CF config


 [ 
https://issues.apache.org/jira/browse/HBASE-4520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-4520:
-

Attachment: 
HBASE-4520_Better_handling_of_Bloom_filt-20111002010126-16a0d7ba.patch

Copied from review board.

 Better handling of Bloom filter type discrepancy between HFile and CF config
 

 Key: HBASE-4520
 URL: https://issues.apache.org/jira/browse/HBASE-4520
 Project: HBase
  Issue Type: Improvement
Reporter: Mikhail Bautin
Assignee: Mikhail Bautin
Priority: Minor
 Fix For: 0.94.0

 Attachments: 
 HBASE-4520_Better_handling_of_Bloom_filt-20111002010126-16a0d7ba.patch


 Modify StoreFile to make it clear where Bloom filter type settings come from. 
 We have two sources of truth: (1) HFile; and (2) CF configuration. (1) takes 
 precedence in the reader, and (2) takes precedence in the writer.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HBASE-4520) Better handling of Bloom filter type discrepancy between HFile and CF config

2011-10-04 Thread stack (Resolved) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-4520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-4520.
--

   Resolution: Fixed
Fix Version/s: (was: 0.94.0)
   0.92.0
 Hadoop Flags: Reviewed

Committed to 0.92 branch and trunk.  Thanks for the patch Mikhail.

 Better handling of Bloom filter type discrepancy between HFile and CF config
 

 Key: HBASE-4520
 URL: https://issues.apache.org/jira/browse/HBASE-4520
 Project: HBase
  Issue Type: Improvement
Reporter: Mikhail Bautin
Assignee: Mikhail Bautin
Priority: Minor
 Fix For: 0.92.0

 Attachments: 
 HBASE-4520_Better_handling_of_Bloom_filt-20111002010126-16a0d7ba.patch


 Modify StoreFile to make it clear where Bloom filter type settings come from. 
 We have two sources of truth: (1) HFile; and (2) CF configuration. (1) takes 
 precedence in the reader, and (2) takes precedence in the writer.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4531) hbase-4454 failsafe broke mvn site; back it out or fix

2011-10-04 Thread Nicolas Spiegelberg (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120368#comment-13120368
 ] 

stack commented on HBASE-4531:
--

Committed to 0.92 too.

 hbase-4454 failsafe broke mvn site; back it out or fix
 --

 Key: HBASE-4531
 URL: https://issues.apache.org/jira/browse/HBASE-4531
 Project: HBase
  Issue Type: Bug
Reporter: stack
Assignee: Akash Ashok
Priority: Blocker
 Fix For: 0.92.0

 Attachments: HBase-4531-v2.patch, HBase-4531-v2.patch, 
 HBase-4531-v3.patch, HBase-4531.patch


 mvn site is broke in head of trunk.  If I back out the last pom change it 
 works again:
 {code}
 
 r1177168 | stack | 2011-09-29 05:42:13 + (Thu, 29 Sep 2011) | 1 line
 HBASE-4454 Add failsafe plugin to build and rename integration tests
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4489) Better key splitting in RegionSplitter

[
https://issues.apache.org/jira/browse/HBASE-4489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120382#comment-13120382
]

Nicolas Spiegelberg commented on HBASE-4489:

@Dave:

Some more clarity on MD5StringSplit:

1. In the original RegionSplitter use case, the key is MD5(username) +
username. In the general case, users probably should use an MD5 algorithm to
partition their data if there is not access locality between 2 adjacent keys.
This provides proper random dispersion and helps avoid hot row issues.
2. It was kept as ASCII for readability on WebUI and logs. The original
application was IO bound the data size was ~1000 bytes/entry, so key size
saving was not a huge issue. Additionally, key compression (HBASE-4218) will
be available shortly and make byte code optimization pretty negligible compared
to readability benefits.
3. The end range is not a bug. Java doesn't natively support uint32, so 7FFF is
Max Int for an MD5 hash unless you use int64 in your calculations. I'm
guessing that you're using the Thrift API, but you probably should code for the
native interface.
4. The uneven key space can (and probably should) be fixed, but it is not a
significant issue with a large number of regions. For N regions across a range
of K, the common region size is floor(K/N) the skewed region is K - (N-1) *
floor(K/N) == floor(K/N) + K % N 2N. So the worse case is that one region
has double load. For 10 regions/server, 10% variance.

Better key splitting in RegionSplitter
--

[jira] [Commented] (HBASE-4534) A new unit test for lazy seek and StoreScanner in general

2011-10-04 Thread Mikhail Bautin (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120392#comment-13120392
 ] 

Mikhail Bautin commented on HBASE-4534:
---

This is ready to be committed (all unit tests pass). The diff is in reviewboard 
(https://reviews.apache.org/r/2155/diff/raw/).

 A new unit test for lazy seek and StoreScanner in general
 -

 Key: HBASE-4534
 URL: https://issues.apache.org/jira/browse/HBASE-4534
 Project: HBase
  Issue Type: Test
Affects Versions: 0.94.0
Reporter: Mikhail Bautin
Assignee: Mikhail Bautin

 A randomized unit test for Gets/Scans (all-row, single-row, multi-row, 
 all-column, single-column, and multi-column). Also all combinations of Bloom 
 filters and compression (NONE vs GZIP) are tested. The unit test flushes 
 multiple StoreFiles with disjoint timestamp ranges and runs various types of 
 queries against them. Currently we are not testing overlapping timestamp 
 ranges.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4534) A new unit test for lazy seek and StoreScanner in general

2011-10-04 Thread Jonathan Gray (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120400#comment-13120400
 ] 

Jonathan Gray commented on HBASE-4534:
--

Are we thinking these read optimizations are going to only go into 0.94?  
(Seems reasonable to me, but I will be pulling them into our internal 92 branch)

 A new unit test for lazy seek and StoreScanner in general
 -

 Key: HBASE-4534
 URL: https://issues.apache.org/jira/browse/HBASE-4534
 Project: HBase
  Issue Type: Test
Affects Versions: 0.94.0
Reporter: Mikhail Bautin
Assignee: Mikhail Bautin

 A randomized unit test for Gets/Scans (all-row, single-row, multi-row, 
 all-column, single-column, and multi-column). Also all combinations of Bloom 
 filters and compression (NONE vs GZIP) are tested. The unit test flushes 
 multiple StoreFiles with disjoint timestamp ranges and runs various types of 
 queries against them. Currently we are not testing overlapping timestamp 
 ranges.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HBASE-4534) A new unit test for lazy seek and StoreScanner in general

2011-10-04 Thread Jonathan Gray (Resolved) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-4534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Gray resolved HBASE-4534.
--

   Resolution: Fixed
Fix Version/s: 0.94.0

Committed to trunk.  Nice work Mikhail!

 A new unit test for lazy seek and StoreScanner in general
 -

 Key: HBASE-4534
 URL: https://issues.apache.org/jira/browse/HBASE-4534
 Project: HBase
  Issue Type: Test
Affects Versions: 0.94.0
Reporter: Mikhail Bautin
Assignee: Mikhail Bautin
 Fix For: 0.94.0


 A randomized unit test for Gets/Scans (all-row, single-row, multi-row, 
 all-column, single-column, and multi-column). Also all combinations of Bloom 
 filters and compression (NONE vs GZIP) are tested. The unit test flushes 
 multiple StoreFiles with disjoint timestamp ranges and runs various types of 
 queries against them. Currently we are not testing overlapping timestamp 
 ranges.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4508) Backport HBASE-3777 to 0.90 branch

2011-10-04 Thread Bright Fulton (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-4508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bright Fulton updated HBASE-4508:
-

Attachment: HBASE-4508.v2.patch

Renamed hbase.client.per.config.inst to hbase.connection.per.config based on 
[feedback|http://mail-archives.apache.org/mod_mbox/hbase-dev/201109.mbox/%3CCALte62wkkH_juDaMCrYq=vp6+sjrinfo7ed-j14psfpepco...@mail.gmail.com%3E].

Cumulative diff is now against rev 1178940.

 Backport HBASE-3777 to 0.90 branch
 --

 Key: HBASE-4508
 URL: https://issues.apache.org/jira/browse/HBASE-4508
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
 Attachments: HBASE-4508.v1.patch, HBASE-4508.v2.patch


 See discussion here: 
 http://search-hadoop.com/m/MJBId1aazTR1/backporting+HBASE-3777+to+0.90subj=backporting+HBASE+3777+to+0+90
 Rocketfuel has been running 0.90.3 with HBASE-3777 since its resolution.
 They have 10 RS nodes , 1 Master and 1 Zookeeper
 Live writes and reads but super heavy on reads. Cache hit is pretty high.
 The qps on one of their data centers is 50K.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4536) Allow CF to retain deleted rows

2011-10-04 Thread jirapos...@reviews.apache.org (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-4536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120469#comment-13120469
]

Lars Hofhansl commented on HBASE-4536:
--

Turns out that currently it is not at all possible to query (get or scan) any
rows behind a delete marker, even setting a timerange on a Get that is before
the delete marker will not return any rows.

Allow CF to retain deleted rows
---

[jira] [Commented] (HBASE-4536) Allow CF to retain deleted rows

[
https://issues.apache.org/jira/browse/HBASE-4536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120487#comment-13120487
]

jirapos...@reviews.apache.org commented on HBASE-4536:
--

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2178/
---

Review request for hbase.

Summary
---

In order to use a replicated cluster as backup and to make true use of HBase's
timestamps it has to be possible to get/scan rows that are hidden by a delete
marker. If a marker was placed at time T, it should be possible to retrieve
rows with get/scan timerange of [0-T).

This changes the following:
1. MinVersions now also means: Keep this number of version even when the row
was deleted.
2. Allow gets/scans to retrieve rows hidden by a delete marker.
3. Do not unconditionally collect all deleted rows during a compaction.

The change is pretty small, but the logic is intricate, so please review
carefully.

This addresses bug HBASE-4536.
https://issues.apache.org/jira/browse/HBASE-4536

Diffs
-

http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java
1178891

http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ColumnTracker.java
1178891

http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ExplicitColumnTracker.java
1178891

http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
1178891

http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanWildcardColumnTracker.java
1178891

http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
1178891

http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestMinVersions.java
1178891

Diff: https://reviews.apache.org/r/2178/diff

Testing
---

TestMinVersions (with a new test for this) passed. Have not run full suite,
yet. This is for initial feedback.

Thanks,

Lars

Allow CF to retain deleted rows
---

[jira] [Commented] (HBASE-4422) Move block cache parameters and references into single CacheConf class

2011-10-04 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120490#comment-13120490
 ] 

jirapos...@reviews.apache.org commented on HBASE-4422:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2089/#review2318
---



/src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileReader.java
https://reviews.apache.org/r/2089/#comment5323

is .toString() necessary?



/src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileWriter.java
https://reviews.apache.org/r/2089/#comment5324

This simplifies things a lot.



/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java
https://reviews.apache.org/r/2089/#comment5325

Why not make this private?



/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java
https://reviews.apache.org/r/2089/#comment5326

Should this be suffixed with _KEY for consistency with other conf key 
constants in this class?



/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java
https://reviews.apache.org/r/2089/#comment5327

Is this constructor ever used outside of CacheConfig itself? It looks like 
a potential source of errors, because the caller has to get the order of 7 
boolean arguments right.



/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java
https://reviews.apache.org/r/2089/#comment5328

Why is this not private?



/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java
https://reviews.apache.org/r/2089/#comment5329

nit: space between if and the parenthesis :)



/src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java
https://reviews.apache.org/r/2089/#comment5330

This is really simplifying a lot of code :)



/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV1.java
https://reviews.apache.org/r/2089/#comment5331

It might be worth it to rename isInMemory() to useInMemoryCacheBucket() or 
isInMemoryCF(). The current name is too general.



/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV1.java
https://reviews.apache.org/r/2089/#comment5332

Will this always return false if there is no block cache?



/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java
https://reviews.apache.org/r/2089/#comment5334

cacheBlock should probably be set to false if 
!cacheBlock.isBlockCacheEnabled()



/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java
https://reviews.apache.org/r/2089/#comment5333

cacheBlock is not set to false if !cacheConf.isBlockCacheEnabled(). Could 
this result in an NPE?



/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java
https://reviews.apache.org/r/2089/#comment5335

It would be good for readability and maybe for performance to assign 
cacheConf.shouldCacheIndexesOnWrite() to a local variable.


- Mikhail


On 2011-10-04 02:50:58, Jonathan Gray wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/2089/
bq.  ---
bq.  
bq.  (Updated 2011-10-04 02:50:58)
bq.  
bq.  
bq.  Review request for hbase, Dhruba Borthakur, Michael Stack, and Li Pi.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  Creates a new CacheConfig class and moves almost everything block cache 
related into this single class.  Adding new configuration params and booleans 
and such should be much better.
bq.  
bq.  All tests are NOT passing yet, still working on it, but wanted to have 
something up today.  Basically code complete but broken :)
bq.  
bq.  
bq.  This addresses bug HBASE-4422.
bq.  https://issues.apache.org/jira/browse/HBASE-4422
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq./src/main/java/org/apache/hadoop/hbase/io/HalfStoreFileReader.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileReader.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileWriter.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java 
PRE-CREATION 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java 1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFilePrettyPrinter.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV1.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV1.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java 
1178676 
bq.

[jira] [Commented] (HBASE-4531) hbase-4454 failsafe broke mvn site; back it out or fix


[ 
https://issues.apache.org/jira/browse/HBASE-4531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120496#comment-13120496
 ] 

Hudson commented on HBASE-4531:
---

Integrated in HBase-0.92 #42 (See 
[https://builds.apache.org/job/HBase-0.92/42/])
HBASE-4531 hbase-4454 failsafe broke mvn site; back it out or fix

stack : 
Files : 
* /hbase/branches/0.92/pom.xml


 hbase-4454 failsafe broke mvn site; back it out or fix
 --

 Key: HBASE-4531
 URL: https://issues.apache.org/jira/browse/HBASE-4531
 Project: HBase
  Issue Type: Bug
Reporter: stack
Assignee: Akash Ashok
Priority: Blocker
 Fix For: 0.92.0

 Attachments: HBase-4531-v2.patch, HBase-4531-v2.patch, 
 HBase-4531-v3.patch, HBase-4531.patch


 mvn site is broke in head of trunk.  If I back out the last pom change it 
 works again:
 {code}
 
 r1177168 | stack | 2011-09-29 05:42:13 + (Thu, 29 Sep 2011) | 1 line
 HBASE-4454 Add failsafe plugin to build and rename integration tests
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4520) Better handling of Bloom filter type discrepancy between HFile and CF config


[ 
https://issues.apache.org/jira/browse/HBASE-4520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120498#comment-13120498
 ] 

Hudson commented on HBASE-4520:
---

Integrated in HBase-0.92 #42 (See 
[https://builds.apache.org/job/HBase-0.92/42/])
HBASE-4520 Better handling of Bloom filter type discrepancy between HFile 
and CF config

stack : 
Files : 
* /hbase/branches/0.92/CHANGES.txt
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java


 Better handling of Bloom filter type discrepancy between HFile and CF config
 

 Key: HBASE-4520
 URL: https://issues.apache.org/jira/browse/HBASE-4520
 Project: HBase
  Issue Type: Improvement
Reporter: Mikhail Bautin
Assignee: Mikhail Bautin
Priority: Minor
 Fix For: 0.92.0

 Attachments: 
 HBASE-4520_Better_handling_of_Bloom_filt-20111002010126-16a0d7ba.patch


 Modify StoreFile to make it clear where Bloom filter type settings come from. 
 We have two sources of truth: (1) HFile; and (2) CF configuration. (1) takes 
 precedence in the reader, and (2) takes precedence in the writer.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4454) Add failsafe plugin to build and rename integration tests

2011-10-04 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120497#comment-13120497
 ] 

Hudson commented on HBASE-4454:
---

Integrated in HBase-0.92 #42 (See 
[https://builds.apache.org/job/HBase-0.92/42/])
HBASE-4531 hbase-4454 failsafe broke mvn site; back it out or fix


 Add failsafe plugin to build and rename integration tests
 -

 Key: HBASE-4454
 URL: https://issues.apache.org/jira/browse/HBASE-4454
 Project: HBase
  Issue Type: Improvement
Reporter: Jesse Yates
 Fix For: 0.92.0

 Attachments: mvn_HBASE-4454.patch


 Add the maven-failsafe-plugin to the build process so we can run integration 
 tests with mvn verify. This will also involve a renaming of integration 
 tests to conform to a new integration test regex.
 This is a stopgap measure while we until break them out into their own module.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4422) Move block cache parameters and references into single CacheConf class


[ 
https://issues.apache.org/jira/browse/HBASE-4422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120501#comment-13120501
 ] 

jirapos...@reviews.apache.org commented on HBASE-4422:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2089/#review2319
---


Sending the second part of comments. Still have not reviewed StoreFile changes. 
I guess my biggest concern is the initialization of a default CacheConfig in a 
lot of places (hopefully mostly tests) while it could be initialized based on 
the available Configuration.


/src/main/java/org/apache/hadoop/hbase/util/CompressionTest.java
https://reviews.apache.org/r/2089/#comment5336

Should not there be a way to create a default CacheConf() from a 
Configuration()? Currently it seems that you are creating a default CacheConfig 
to pass to getWriteFactory even though you have a conf available, potentially 
with some cache-related settings. Maybe there should be an overload of 
HFile.getWriterFactory that takes a conf and creates a default CacheConfig out 
of it.



/src/test/java/org/apache/hadoop/hbase/io/TestHalfStoreFileReader.java
https://reviews.apache.org/r/2089/#comment5337

The same concern about default CacheConfig as above.



/src/test/java/org/apache/hadoop/hbase/io/hfile/RandomSeek.java
https://reviews.apache.org/r/2089/#comment5338

nit: trailing whitespace (we probably should remove all of it in one patch 
someday :)



/src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java
https://reviews.apache.org/r/2089/#comment5340

Do you want to keep this debug code around?



/src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java
https://reviews.apache.org/r/2089/#comment5339

Is this debug code?



/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFile.java
https://reviews.apache.org/r/2089/#comment5341

Should the CacheConfig be initialized based on the configuration? (the same 
as other places)



/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileWriterV2.java
https://reviews.apache.org/r/2089/#comment5343

The same thing with CacheConfig and conf.



/src/test/java/org/apache/hadoop/hbase/io/hfile/TestReseekTo.java
https://reviews.apache.org/r/2089/#comment5344

The same thing with CacheConfig and conf.



/src/test/java/org/apache/hadoop/hbase/io/hfile/TestSeekTo.java
https://reviews.apache.org/r/2089/#comment5345

The same thing with CacheConfig and conf (OK, I won't mention this anymore).



/src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksRead.java
https://reviews.apache.org/r/2089/#comment5348

Should not we move the cast to initialization and change blockCache's type 
to LruBlockCache?

Is this method only intended for testing, by the way? If so, should this be 
indicated in the method name?


- Mikhail


On 2011-10-04 02:50:58, Jonathan Gray wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/2089/
bq.  ---
bq.  
bq.  (Updated 2011-10-04 02:50:58)
bq.  
bq.  
bq.  Review request for hbase, Dhruba Borthakur, Michael Stack, and Li Pi.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  Creates a new CacheConfig class and moves almost everything block cache 
related into this single class.  Adding new configuration params and booleans 
and such should be much better.
bq.  
bq.  All tests are NOT passing yet, still working on it, but wanted to have 
something up today.  Basically code complete but broken :)
bq.  
bq.  
bq.  This addresses bug HBASE-4422.
bq.  https://issues.apache.org/jira/browse/HBASE-4422
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq./src/main/java/org/apache/hadoop/hbase/io/HalfStoreFileReader.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileReader.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileWriter.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java 
PRE-CREATION 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java 1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFilePrettyPrinter.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV1.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV1.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java 
1178676 
bq.

[jira] [Commented] (HBASE-4422) Move block cache parameters and references into single CacheConf class

2011-10-04 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120505#comment-13120505
 ] 

jirapos...@reviews.apache.org commented on HBASE-4422:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2089/#review2320
---


Looks good.  Nice cleanup.  Just one issue w/ modelling where we ask 
CacheConfig for Cache instance -- seems 'off'.


/src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileWriter.java
https://reviews.apache.org/r/2089/#comment5342

CacheConfig has a Configuration?



/src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileWriter.java
https://reviews.apache.org/r/2089/#comment5346

Don't we still need this?



/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java
https://reviews.apache.org/r/2089/#comment5347

Its cache and a cache config?  Seems a bit odd that you come here to get 
the block cache instance too.  I'd think that'd be a CacheFactory that took a 
CacheConfig.



/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java
https://reviews.apache.org/r/2089/#comment5349

This is good defining these keys in here instead of up in HConstants or 
somewhere



/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java
https://reviews.apache.org/r/2089/#comment5350

Future feature?



/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV1.java
https://reviews.apache.org/r/2089/#comment5351

Yeah, seems odd asking CacheConfig for the Cache.



/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV1.java
https://reviews.apache.org/r/2089/#comment5352

ditto



/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
https://reviews.apache.org/r/2089/#comment5354

Is this right?  Its a static or something?  Can't pass Configuration and/or 
family?  Just defaults?


- Michael


On 2011-10-04 02:50:58, Jonathan Gray wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/2089/
bq.  ---
bq.  
bq.  (Updated 2011-10-04 02:50:58)
bq.  
bq.  
bq.  Review request for hbase, Dhruba Borthakur, Michael Stack, and Li Pi.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  Creates a new CacheConfig class and moves almost everything block cache 
related into this single class.  Adding new configuration params and booleans 
and such should be much better.
bq.  
bq.  All tests are NOT passing yet, still working on it, but wanted to have 
something up today.  Basically code complete but broken :)
bq.  
bq.  
bq.  This addresses bug HBASE-4422.
bq.  https://issues.apache.org/jira/browse/HBASE-4422
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq./src/main/java/org/apache/hadoop/hbase/io/HalfStoreFileReader.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileReader.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileWriter.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java 
PRE-CREATION 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java 1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFilePrettyPrinter.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV1.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV1.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java 
1178676 
bq.
/src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1178676 
bq./src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/regionserver/Store.java 1178676 
bq./src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/util/BloomFilterFactory.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/util/CompressionTest.java 1178676 
bq./src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java 
1178676 
bq./src/test/java/org/apache/hadoop/hbase/io/TestHalfStoreFileReader.java 
1178676 
bq./src/test/java/org/apache/hadoop/hbase/io/hfile/RandomSeek.java 1178676 
bq./src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java 
1178676 
bq./src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFile.java 1178676 
bq.

[jira] [Commented] (HBASE-4465) Lazy-seek optimization for StoreFile scanners

2011-10-04 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120520#comment-13120520
 ] 

jirapos...@reviews.apache.org commented on HBASE-4465:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2180/
---

Review request for hbase.


Summary
---

Previously, if we had several StoreFiles for a column family in a region, we 
would seek in each of them and only then merge the results, even though the 
row/column we are looking for might only be in the most recent (and the 
smallest) file. Now we prioritize our reads from those files so that we check 
the most recent file first. This is done by doing a lazy seek which pretends 
that the next value in the StoreFile is (seekRow, seekColumn, 
lastTimestampInStoreFile), which is earlier in the KV order than anything that 
might actually occur in the file. So if we don't find the result in earlier 
files, that fake KV will bubble up to the top of the KV heap and a real seek 
will be done. This is expected to significantly reduce the amount of disk IO 
(as of 09/22/2011 we are doing dark launch testing and measurement).

This is joint work with Liyin Tang – huge thanks to him for many helpful 
discussions on this and the idea of putting fake KVs with the highest timestamp 
of the StoreFile in the scanner priority queue.


This addresses bug HBASE-4465.
https://issues.apache.org/jira/browse/HBASE-4465


Diffs
-

  src/main/java/org/apache/hadoop/hbase/KeyValue.java aa34006 
  
src/main/java/org/apache/hadoop/hbase/regionserver/AbstractKeyValueScanner.java 
94ddce7 
  src/main/java/org/apache/hadoop/hbase/regionserver/ColumnCount.java 1be0280 
  src/main/java/org/apache/hadoop/hbase/regionserver/KeyValueHeap.java b8d33e8 
  src/main/java/org/apache/hadoop/hbase/regionserver/KeyValueScanner.java 
fbcd276 
  src/main/java/org/apache/hadoop/hbase/regionserver/MemStore.java 035f765 
  
src/main/java/org/apache/hadoop/hbase/regionserver/NonLazyKeyValueScanner.java 
PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java 
dad278a 
  src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java abb5931 
  src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java 
31bfea7 
  src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java 64a6e3e 
  src/main/java/org/apache/hadoop/hbase/util/CollectionBackedScanner.java 
8ad5aab 
  src/test/java/org/apache/hadoop/hbase/regionserver/TestMemStore.java 9d2b2a7 

Diff: https://reviews.apache.org/r/2180/diff


Testing
---

Running unit tests -- please do not commit yet.


Thanks,

Mikhail



 Lazy-seek optimization for StoreFile scanners
 -

 Key: HBASE-4465
 URL: https://issues.apache.org/jira/browse/HBASE-4465
 Project: HBase
  Issue Type: Improvement
Reporter: Mikhail Bautin
Assignee: Mikhail Bautin
  Labels: optimization, seek
 Fix For: 0.89.20100924, 0.94.0


 Previously, if we had several StoreFiles for a column family in a region, we 
 would seek in each of them and only then merge the results, even though the 
 row/column we are looking for might only be in the most recent (and the 
 smallest) file. Now we prioritize our reads from those files so that we check 
 the most recent file first. This is done by doing a lazy seek which 
 pretends that the next value in the StoreFile is (seekRow, seekColumn, 
 lastTimestampInStoreFile), which is earlier in the KV order than anything 
 that might actually occur in the file. So if we don't find the result in 
 earlier files, that fake KV will bubble up to the top of the KV heap and a 
 real seek will be done. This is expected to significantly reduce the amount 
 of disk IO (as of 09/22/2011 we are doing dark launch testing and 
 measurement).
 This is joint work with Liyin Tang -- huge thanks to him for many helpful 
 discussions on this and the idea of putting fake KVs with the highest 
 timestamp of the StoreFile in the scanner priority queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4522) Make hbase-site-custom.xml override the hbase-site.xml

2011-10-04 Thread Mikhail Bautin (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120522#comment-13120522
 ] 

Mikhail Bautin commented on HBASE-4522:
---

OK, discarding this JIRA. Implemented as an internal patch instead.

 Make hbase-site-custom.xml override the hbase-site.xml
 --

 Key: HBASE-4522
 URL: https://issues.apache.org/jira/browse/HBASE-4522
 Project: HBase
  Issue Type: Improvement
Reporter: Mikhail Bautin
Assignee: Liyin Tang
Priority: Minor
 Fix For: 0.94.0


 The motivation for diff is that we want to override some config change for 
 any specific cluster easily by just adding the config entries in the 
 hbase-site-custom.xml for that cluster. This change adds the 
 hbase-site-custom.xml configuration file into HBaseConfiguration.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HBASE-4522) Make hbase-site-custom.xml override the hbase-site.xml

2011-10-04 Thread Mikhail Bautin (Resolved) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-4522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Bautin resolved HBASE-4522.
---

Resolution: Not A Problem

 Make hbase-site-custom.xml override the hbase-site.xml
 --

 Key: HBASE-4522
 URL: https://issues.apache.org/jira/browse/HBASE-4522
 Project: HBase
  Issue Type: Improvement
Reporter: Mikhail Bautin
Assignee: Liyin Tang
Priority: Minor
 Fix For: 0.94.0


 The motivation for diff is that we want to override some config change for 
 any specific cluster easily by just adding the config entries in the 
 hbase-site-custom.xml for that cluster. This change adds the 
 hbase-site-custom.xml configuration file into HBaseConfiguration.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4536) Allow CF to retain deleted rows

2011-10-04 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120526#comment-13120526
 ] 

jirapos...@reviews.apache.org commented on HBASE-4536:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2178/#review2323
---


Nice feature.
Should a new config parameter be introduced ?

- Ted


On 2011-10-04 21:27:20, Lars Hofhansl wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/2178/
bq.  ---
bq.  
bq.  (Updated 2011-10-04 21:27:20)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  In order to use a replicated cluster as backup and to make true use of 
HBase's timestamps it has to be possible to get/scan rows that are hidden by a 
delete marker. If a marker was placed at time T, it should be possible to 
retrieve rows with get/scan timerange of [0-T).
bq.  
bq.  This changes the following:
bq.  1. MinVersions now also means: Keep this number of version even when the 
row was deleted.
bq.  2. Allow gets/scans to retrieve rows hidden by a delete marker.
bq.  3. Do not unconditionally collect all deleted rows during a compaction.
bq.  
bq.  The change is pretty small, but the logic is intricate, so please review 
carefully.
bq.  
bq.  
bq.  This addresses bug HBASE-4536.
bq.  https://issues.apache.org/jira/browse/HBASE-4536
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java
 1178891 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ColumnTracker.java
 1178891 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ExplicitColumnTracker.java
 1178891 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
 1178891 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanWildcardColumnTracker.java
 1178891 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
 1178891 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestMinVersions.java
 1178891 
bq.  
bq.  Diff: https://reviews.apache.org/r/2178/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  TestMinVersions (with a new test for this) passed. Have not run full 
suite, yet. This is for initial feedback.
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Lars
bq.  
bq.



 Allow CF to retain deleted rows
 ---

 Key: HBASE-4536
 URL: https://issues.apache.org/jira/browse/HBASE-4536
 Project: HBase
  Issue Type: Sub-task
  Components: regionserver
Affects Versions: 0.92.0
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.92.0, 0.94.0


 Parent allows for a cluster to retain rows for a TTL or keep a minimum number 
 of versions.
 However, if a client deletes a row all version older than the delete tomb 
 stone will be remove at the next major compaction (and even at memstore flush 
 - see HBASE-4241).
 There should be a way to retain those version to guard against software error.
 I see two options here:
 1. Add a new flag HColumnDescriptor. Something like RETAIN_DELETED.
 2. Folds this into the parent change. I.e. keep minimum-number-of-versions of 
 versions even past the delete marker.
 #1 would allow for more flexibility. #2 comes somewhat naturally with parent 
 (from a user viewpoint)
 Comments? Any other options?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4282) Potential data loss in retries of WAL close introduced in HBASE-4222


[ 
https://issues.apache.org/jira/browse/HBASE-4282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120528#comment-13120528
 ] 

stack commented on HBASE-4282:
--

You going to apply Gary?

 Potential data loss in retries of WAL close introduced in HBASE-4222
 

 Key: HBASE-4282
 URL: https://issues.apache.org/jira/browse/HBASE-4282
 Project: HBase
  Issue Type: Bug
Reporter: Gary Helmling
Assignee: Gary Helmling
Priority: Blocker
 Fix For: 0.92.0, 0.90.5

 Attachments: HBASE-4282_0.90_2.patch, HBASE-4282_trunk_2.patch, 
 HBASE-4282_trunk_prelim.patch


 The ability to ride over WAL close errors on log rolling added in HBASE-4222 
 could lead to missing HLog entries if:
 * A table has DEFERRED_LOG_FLUSH=true
 * There are unflushed WALEdit entries for that table in the current 
 SequenceFile writer buffer
 Since the writes were already acknowledged to the client, just ignoring the 
 close error to allow for another log roll doesn't seem like the right thing 
 to do here.
 We could easily flag this state and only ride over the close error if there 
 aren't unflushed entries.  This would bring the above condition back to the 
 previous behavior of aborting the region server.  However, aborting the 
 region server in this state is still guaranteeing data loss.  Is there 
 anything we can do better in this case?  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4482) Race Condition Concerning Eviction in SlabCache


 [ 
https://issues.apache.org/jira/browse/HBASE-4482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-4482:
-

Priority: Blocker  (was: Critical)

Making this a blocker because its a blocker on a blocker.

I suppose we can back out this feature if not done by the time we cut a 
0.92 shouldn't hold up the release (I suppose that makes it not a blocker)

 Race Condition Concerning Eviction in SlabCache
 ---

 Key: HBASE-4482
 URL: https://issues.apache.org/jira/browse/HBASE-4482
 Project: HBase
  Issue Type: Sub-task
Reporter: Li Pi
Assignee: Li Pi
Priority: Blocker
 Fix For: 0.92.0

 Attachments: hbase-4482v1.txt, hbase-4482v2.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4402) Retaining locality after restart broken


[ 
https://issues.apache.org/jira/browse/HBASE-4402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120530#comment-13120530
 ] 

Ted Yu commented on HBASE-4402:
---

Sure.

 Retaining locality after restart broken
 ---

 Key: HBASE-4402
 URL: https://issues.apache.org/jira/browse/HBASE-4402
 Project: HBase
  Issue Type: Bug
  Components: master
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Assignee: Todd Lipcon
Priority: Blocker
 Fix For: 0.92.0

 Attachments: hbase-4402.txt, hbase-4402.txt


 In DefaultLoadBalancer, we implement the retain assignment function like so:
 {code}
   if (sn != null  servers.contains(sn)) {
 assignments.get(sn).add(region.getKey());
 {code}
 but this will never work since after a cluster restart, all servers have a 
 new ServerName with a new startcode.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4410) FilterList.filterKeyValue can return suboptimal ReturnCodes


[ 
https://issues.apache.org/jira/browse/HBASE-4410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120538#comment-13120538
 ] 

stack commented on HBASE-4410:
--

Any comeback on the above feedback?

 FilterList.filterKeyValue can return suboptimal ReturnCodes
 ---

 Key: HBASE-4410
 URL: https://issues.apache.org/jira/browse/HBASE-4410
 Project: HBase
  Issue Type: Improvement
  Components: filters
Reporter: Jonathan Gray
Assignee: Jonathan Gray
Priority: Minor
 Fix For: 0.92.0

 Attachments: HBASE-4410-v1.patch


 FilterList.filterKeyValue does not always return the most optimal ReturnCode 
 in both the AND and OR conditions.
 For example, if you have F1 AND F2, F1 returns SKIP.  It immediately returns 
 the SKIP.  However, if F2 would have returned NEXT_COL or NEXT_ROW or 
 SEEK_NEXT_USING_HINT, we would actually be able to return the more optimal 
 ReturnCode from F2.
 For AND conditions, we can always pick the *most restrictive* return code.
 For OR conditions, we must always pick the *least restrictive* return code.
 This JIRA is to review the FilterList.filterKeyValue() method to try and make 
 it more optimal and to add a new unit test which verifies the correct 
 behavior.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4334) HRegion.get never validates row


 [ 
https://issues.apache.org/jira/browse/HBASE-4334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-4334:
-

Resolution: Fixed
Status: Resolved  (was: Patch Available)

Resolving.  Was applied to 0.92 branch and to trunk.  Thanks Lars.

 HRegion.get never validates row
 ---

 Key: HBASE-4334
 URL: https://issues.apache.org/jira/browse/HBASE-4334
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.90.4
Reporter: Joe Pallas
Assignee: Lars Hofhansl
 Fix For: 0.92.0

 Attachments: 4334-v2.txt, 4334-v3.txt, 4334.txt


 If a client gets confused (possibly by a hole in .META., see HBASE-4333), it 
 may send a request to the wrong region.  Paths through put, delete, 
 incrementColumnValue, and checkAndMutate all call checkRow either directly or 
 indirectly (through getLock).  But get apparently does not.  This can result 
 in returning an incorrect empty result instead of a WrongRegionException.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4282) Potential data loss in retries of WAL close introduced in HBASE-4222

2011-10-04 Thread Gary Helmling (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120539#comment-13120539
 ] 

Gary Helmling commented on HBASE-4282:
--

Let me update for HBASE-4487 changes.  They touch a lot of the same code and we 
should be able to use the same counter for both.

 Potential data loss in retries of WAL close introduced in HBASE-4222
 

 Key: HBASE-4282
 URL: https://issues.apache.org/jira/browse/HBASE-4282
 Project: HBase
  Issue Type: Bug
Reporter: Gary Helmling
Assignee: Gary Helmling
Priority: Blocker
 Fix For: 0.92.0, 0.90.5

 Attachments: HBASE-4282_0.90_2.patch, HBASE-4282_trunk_2.patch, 
 HBASE-4282_trunk_prelim.patch


 The ability to ride over WAL close errors on log rolling added in HBASE-4222 
 could lead to missing HLog entries if:
 * A table has DEFERRED_LOG_FLUSH=true
 * There are unflushed WALEdit entries for that table in the current 
 SequenceFile writer buffer
 Since the writes were already acknowledged to the client, just ignoring the 
 close error to allow for another log roll doesn't seem like the right thing 
 to do here.
 We could easily flag this state and only ride over the close error if there 
 aren't unflushed entries.  This would bring the above condition back to the 
 previous behavior of aborting the region server.  However, aborting the 
 region server in this state is still guaranteeing data loss.  Is there 
 anything we can do better in this case?  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-3833) ability to support includes/excludes list in Hbase

2011-10-04 Thread Vishal Kathuria (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-3833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120541#comment-13120541
 ] 

Vishal Kathuria commented on HBASE-3833:


I fell sick last week and then got busy with a conference. I should be getting 
back to this soon.

 ability to support includes/excludes list in Hbase
 --

 Key: HBASE-3833
 URL: https://issues.apache.org/jira/browse/HBASE-3833
 Project: HBase
  Issue Type: Improvement
  Components: client, regionserver
Affects Versions: 0.90.2
Reporter: dhruba borthakur
Assignee: dhruba borthakur
 Attachments: excl-patch.txt, excl-patch.txt


 An HBase cluster currently does not have the ability to specify that the 
 master should accept regionservers only from a specified list. This helps 
 preventing administrative errors where the same machine could be included in 
 two clusters. It also allows the administrator to easily remove un-ssh-able 
 machines from the cluster.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4467) Handle inconsistencies in Hadoop libraries naming in hbase script

2011-10-04 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120542#comment-13120542
 ] 

stack commented on HBASE-4467:
--

@LarsG You going to commit?

 Handle inconsistencies in Hadoop libraries naming in hbase script
 -

 Key: HBASE-4467
 URL: https://issues.apache.org/jira/browse/HBASE-4467
 Project: HBase
  Issue Type: Bug
  Components: scripts
Affects Versions: 0.92.0, 0.94.0
Reporter: Lars George
Assignee: Lars George
Priority: Trivial
 Fix For: 0.92.0, 0.94.0

 Attachments: HBASE-4467.patch


 When using an Hadoop tarball that has a library naming of hadoop-x.y.z-core 
 as opposed to hadoop-core-x.y.z then the hbase script throws errors.
 {noformat}
 $ bin/start-hbase.sh 
 ls: /projects/opensource/hadoop-0.20.2-append/hadoop-core*.jar: No such file 
 or directory
 Exception in thread main java.lang.NoClassDefFoundError: 
 org/apache/hadoop/util/PlatformName
 Caused by: java.lang.ClassNotFoundException: 
 org.apache.hadoop.util.PlatformName
   at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
   at java.security.AccessController.doPrivileged(Native Method)
   at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
   at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
   at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
   at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
 ls: /projects/opensource/hadoop-0.20.2-append/hadoop-core*.jar: No such file 
 or directory
 Exception in thread main java.lang.NoClassDefFoundError: 
 org/apache/hadoop/util/PlatformName
 Caused by: java.lang.ClassNotFoundException: 
 org.apache.hadoop.util.PlatformName
   at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
   at java.security.AccessController.doPrivileged(Native Method)
   at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
   at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
   at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
   at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
 localhost: starting zookeeper, logging to 
 /projects/opensource/hbase-trunk-rw//logs/hbase-larsgeorge-zookeeper-de1-app-mbp-2.out
 localhost: /projects/opensource/hadoop-0.20.2-append
 localhost: ls: /projects/opensource/hadoop-0.20.2-append/hadoop-core*.jar: No 
 such file or directory
 localhost: Exception in thread main java.lang.NoClassDefFoundError: 
 org/apache/hadoop/util/PlatformName
 localhost: Caused by: java.lang.ClassNotFoundException: 
 org.apache.hadoop.util.PlatformName
 localhost:at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
 localhost:at java.security.AccessController.doPrivileged(Native Method)
 localhost:at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
 localhost:at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
 localhost:at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
 localhost:at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
 starting master, logging to 
 /projects/opensource/hbase-trunk-rw/bin/../logs/hbase-larsgeorge-master-de1-app-mbp-2.out
 /projects/opensource/hadoop-0.20.2-append
 ls: /projects/opensource/hadoop-0.20.2-append/hadoop-core*.jar: No such file 
 or directory
 Exception in thread main java.lang.NoClassDefFoundError: 
 org/apache/hadoop/util/PlatformName
 Caused by: java.lang.ClassNotFoundException: 
 org.apache.hadoop.util.PlatformName
   at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
   at java.security.AccessController.doPrivileged(Native Method)
   at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
   at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
   at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
   at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
 localhost: starting regionserver, logging to 
 /projects/opensource/hbase-trunk-rw//logs/hbase-larsgeorge-regionserver-de1-app-mbp-2.out
 localhost: /projects/opensource/hadoop-0.20.2-append
 localhost: ls: /projects/opensource/hadoop-0.20.2-append/hadoop-core*.jar: No 
 such file or directory
 localhost: Exception in thread main java.lang.NoClassDefFoundError: 
 org/apache/hadoop/util/PlatformName
 localhost: Caused by: java.lang.ClassNotFoundException: 
 org.apache.hadoop.util.PlatformName
 localhost:at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
 localhost:at java.security.AccessController.doPrivileged(Native Method)
 localhost:at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
 localhost:at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
 localhost:at

[jira] [Commented] (HBASE-4422) Move block cache parameters and references into single CacheConf class


[ 
https://issues.apache.org/jira/browse/HBASE-4422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120544#comment-13120544
 ] 

jirapos...@reviews.apache.org commented on HBASE-4422:
--



bq.  On 2011-10-04 21:41:43, Mikhail Bautin wrote:
bq.   Sending the second part of comments. Still have not reviewed StoreFile 
changes. I guess my biggest concern is the initialization of a default 
CacheConfig in a lot of places (hopefully mostly tests) while it could be 
initialized based on the available Configuration.

Yeah, it was a concern i had.  I can go through a remove all the empty 
constructors and require a Configuration and see if I can still get all the 
tests to pass.


bq.  On 2011-10-04 21:41:43, Mikhail Bautin wrote:
bq.   /src/main/java/org/apache/hadoop/hbase/util/CompressionTest.java, line 
113
bq.   https://reviews.apache.org/r/2089/diff/5/?file=47801#file47801line113
bq.  
bq.   Should not there be a way to create a default CacheConf() from a 
Configuration()? Currently it seems that you are creating a default CacheConfig 
to pass to getWriteFactory even though you have a conf available, potentially 
with some cache-related settings. Maybe there should be an overload of 
HFile.getWriterFactory that takes a conf and creates a default CacheConfig out 
of it.

I will go through and see if i can clean that up and not use an empty 
constructor of CacheConfig anywhere


bq.  On 2011-10-04 21:41:43, Mikhail Bautin wrote:
bq.   /src/test/java/org/apache/hadoop/hbase/io/hfile/RandomSeek.java, line 72
bq.   https://reviews.apache.org/r/2089/diff/5/?file=47804#file47804line72
bq.  
bq.   nit: trailing whitespace (we probably should remove all of it in one 
patch someday :)

nipped


bq.  On 2011-10-04 21:41:43, Mikhail Bautin wrote:
bq.   /src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksRead.java, 
line 185
bq.   https://reviews.apache.org/r/2089/diff/5/?file=47816#file47816line185
bq.  
bq.   Should not we move the cast to initialization and change 
blockCache's type to LruBlockCache?
bq.   
bq.   Is this method only intended for testing, by the way? If so, should 
this be indicated in the method name?

There was work done so we didn't need to have LruBlockCache and could use the 
BlockCache interface.  Unfortunately the BlockCache interface doesn't expose a 
method to get the count of elements.  I could just add it to the BlockCache 
interface?


bq.  On 2011-10-04 21:41:43, Mikhail Bautin wrote:
bq.   /src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java, 
line 118
bq.   https://reviews.apache.org/r/2089/diff/5/?file=47805#file47805line118
bq.  
bq.   Do you want to keep this debug code around?

in a test, doesn't hurt i think (makes sense in context of this being a 
parameterized test)


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2089/#review2319
---


On 2011-10-04 02:50:58, Jonathan Gray wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/2089/
bq.  ---
bq.  
bq.  (Updated 2011-10-04 02:50:58)
bq.  
bq.  
bq.  Review request for hbase, Dhruba Borthakur, Michael Stack, and Li Pi.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  Creates a new CacheConfig class and moves almost everything block cache 
related into this single class.  Adding new configuration params and booleans 
and such should be much better.
bq.  
bq.  All tests are NOT passing yet, still working on it, but wanted to have 
something up today.  Basically code complete but broken :)
bq.  
bq.  
bq.  This addresses bug HBASE-4422.
bq.  https://issues.apache.org/jira/browse/HBASE-4422
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq./src/main/java/org/apache/hadoop/hbase/io/HalfStoreFileReader.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileReader.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileWriter.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java 
PRE-CREATION 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java 1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFilePrettyPrinter.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV1.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV1.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java

[jira] [Commented] (HBASE-4422) Move block cache parameters and references into single CacheConf class

2011-10-04 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120543#comment-13120543
 ] 

jirapos...@reviews.apache.org commented on HBASE-4422:
--



bq.  On 2011-10-04 21:28:58, Mikhail Bautin wrote:
bq.   
/src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileReader.java, line 
170
bq.   https://reviews.apache.org/r/2089/diff/5/?file=47785#file47785line170
bq.  
bq.   is .toString() necessary?

removed


bq.  On 2011-10-04 21:28:58, Mikhail Bautin wrote:
bq.   /src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java, line 35
bq.   https://reviews.apache.org/r/2089/diff/5/?file=47787#file47787line35
bq.  
bq.   Why not make this private?

done


bq.  On 2011-10-04 21:28:58, Mikhail Bautin wrote:
bq.   /src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java, line 67
bq.   https://reviews.apache.org/r/2089/diff/5/?file=47787#file47787line67
bq.  
bq.   Should this be suffixed with _KEY for consistency with other conf 
key constants in this class?

done


bq.  On 2011-10-04 21:28:58, Mikhail Bautin wrote:
bq.   /src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java, line 
169
bq.   https://reviews.apache.org/r/2089/diff/5/?file=47787#file47787line169
bq.  
bq.   Is this constructor ever used outside of CacheConfig itself? It 
looks like a potential source of errors, because the caller has to get the 
order of 7 boolean arguments right.

it actually has to be used if you want to manually set stuff, so it does get 
used inside of some unit tests...

It's actually only used from within the package so I will make it pkg private


bq.  On 2011-10-04 21:28:58, Mikhail Bautin wrote:
bq.   /src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java, line 
268
bq.   https://reviews.apache.org/r/2089/diff/5/?file=47787#file47787line268
bq.  
bq.   Why is this not private?

There is a RandomSeek unit test that injects a block cache.  This is not a 
change in behavior (moved over from how it was in StoreFile previously)


bq.  On 2011-10-04 21:28:58, Mikhail Bautin wrote:
bq.   /src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java, line 
307
bq.   https://reviews.apache.org/r/2089/diff/5/?file=47787#file47787line307
bq.  
bq.   nit: space between if and the parenthesis :)

yeah i do that a lot :)  changed... thanks for the detailed look!


bq.  On 2011-10-04 21:28:58, Mikhail Bautin wrote:
bq.   /src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV1.java, line 
355
bq.   https://reviews.apache.org/r/2089/diff/5/?file=47790#file47790line355
bq.  
bq.   Will this always return false if there is no block cache?

yes.  all of the cacheConf calls do:

  public boolean shouldEvictOnClose() {
return isBlockCacheEnabled()  this.evictOnClose;
  }


bq.  On 2011-10-04 21:28:58, Mikhail Bautin wrote:
bq.   /src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java, line 
244
bq.   https://reviews.apache.org/r/2089/diff/5/?file=47791#file47791line244
bq.  
bq.   cacheBlock should probably be set to false if 
!cacheBlock.isBlockCacheEnabled()

same thing as above.  shouldCacheDataOnRead() is false if block cache is not 
enabled.


bq.  On 2011-10-04 21:28:58, Mikhail Bautin wrote:
bq.   /src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java, line 
265
bq.   https://reviews.apache.org/r/2089/diff/5/?file=47791#file47791line265
bq.  
bq.   cacheBlock is not set to false if !cacheConf.isBlockCacheEnabled(). 
Could this result in an NPE?

same as above.  no check from cacheConf ever returns true if block cache is 
globally disabled


bq.  On 2011-10-04 21:28:58, Mikhail Bautin wrote:
bq.   /src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java, line 
168
bq.   https://reviews.apache.org/r/2089/diff/5/?file=47793#file47793line168
bq.  
bq.   It would be good for readability and maybe for performance to assign 
cacheConf.shouldCacheIndexesOnWrite() to a local variable.

done


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2089/#review2318
---


On 2011-10-04 02:50:58, Jonathan Gray wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/2089/
bq.  ---
bq.  
bq.  (Updated 2011-10-04 02:50:58)
bq.  
bq.  
bq.  Review request for hbase, Dhruba Borthakur, Michael Stack, and Li Pi.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  Creates a new CacheConfig class and moves almost everything block cache 
related into this single class.  Adding new configuration params and booleans 
and such should be much better.
bq.  
bq.  All tests are

[jira] [Commented] (HBASE-4422) Move block cache parameters and references into single CacheConf class

2011-10-04 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120550#comment-13120550
 ] 

jirapos...@reviews.apache.org commented on HBASE-4422:
--



bq.  On 2011-10-04 21:41:43, Mikhail Bautin wrote:
bq.   /src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksRead.java, 
line 185
bq.   https://reviews.apache.org/r/2089/diff/5/?file=47816#file47816line185
bq.  
bq.   Should not we move the cast to initialization and change 
blockCache's type to LruBlockCache?
bq.   
bq.   Is this method only intended for testing, by the way? If so, should 
this be indicated in the method name?
bq.  
bq.  Jonathan Gray wrote:
bq.  There was work done so we didn't need to have LruBlockCache and could 
use the BlockCache interface.  Unfortunately the BlockCache interface doesn't 
expose a method to get the count of elements.  I could just add it to the 
BlockCache interface?

Okay I added getBlockCount() to BlockCache interface.


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2089/#review2319
---


On 2011-10-04 02:50:58, Jonathan Gray wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/2089/
bq.  ---
bq.  
bq.  (Updated 2011-10-04 02:50:58)
bq.  
bq.  
bq.  Review request for hbase, Dhruba Borthakur, Michael Stack, and Li Pi.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  Creates a new CacheConfig class and moves almost everything block cache 
related into this single class.  Adding new configuration params and booleans 
and such should be much better.
bq.  
bq.  All tests are NOT passing yet, still working on it, but wanted to have 
something up today.  Basically code complete but broken :)
bq.  
bq.  
bq.  This addresses bug HBASE-4422.
bq.  https://issues.apache.org/jira/browse/HBASE-4422
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq./src/main/java/org/apache/hadoop/hbase/io/HalfStoreFileReader.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileReader.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileWriter.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java 
PRE-CREATION 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java 1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFilePrettyPrinter.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV1.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV1.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java 
1178676 
bq.
/src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1178676 
bq./src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/regionserver/Store.java 1178676 
bq./src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/util/BloomFilterFactory.java 
1178676 
bq./src/main/java/org/apache/hadoop/hbase/util/CompressionTest.java 1178676 
bq./src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java 
1178676 
bq./src/test/java/org/apache/hadoop/hbase/io/TestHalfStoreFileReader.java 
1178676 
bq./src/test/java/org/apache/hadoop/hbase/io/hfile/RandomSeek.java 1178676 
bq./src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java 
1178676 
bq./src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFile.java 1178676 
bq./src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlockIndex.java 
1178676 
bq.
/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFilePerformance.java 
1178676 
bq./src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileReaderV1.java 
1178676 
bq./src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileSeek.java 
1178676 
bq./src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileWriterV2.java 
1178676 
bq./src/test/java/org/apache/hadoop/hbase/io/hfile/TestReseekTo.java 
1178676 
bq./src/test/java/org/apache/hadoop/hbase/io/hfile/TestSeekTo.java 1178676 
bq.
/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java 
1178676 
bq.

[jira] [Commented] (HBASE-4070) [Coprocessors] Improve region server metrics to report loaded coprocessors to master

2011-10-04 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120557#comment-13120557
 ] 

jirapos...@reviews.apache.org commented on HBASE-4070:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2029/#review2327
---



src/main/jamon/org/apache/hbase/tmpl/master/MasterStatusTmpl.jamon
https://reviews.apache.org/r/2029/#comment5374

Nice



src/main/java/org/apache/hadoop/hbase/HServerLoad.java
https://reviews.apache.org/r/2029/#comment5375

Is this comparator needed?  Without it I'd think we'd sort by the String's 
natural order which is probably what we want?



src/main/java/org/apache/hadoop/hbase/HServerLoad.java
https://reviews.apache.org/r/2029/#comment5376

Are we doing this twice here?  There is a getProcessors and its makes them 
up on the fly.  Do we need this data member?



src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
https://reviews.apache.org/r/2029/#comment5377

We don't use a Comparator here.



src/main/java/org/apache/hadoop/hbase/master/HMaster.java
https://reviews.apache.org/r/2029/#comment5379

Could you just do masterCoprocessors.toArray here?



src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
https://reviews.apache.org/r/2029/#comment5381

Or do

return hsl == null? null: hsl.getCoprocessors();



src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorReporting.java
https://reviews.apache.org/r/2029/#comment5386

Can these asserts be done inside another CP test rather than in a 
standalone test as here?  These cluster spinups take time.  No biggie if not 
possible.


- Michael


On 2011-10-04 01:21:46, Eugene Koontz wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/2029/
bq.  ---
bq.  
bq.  (Updated 2011-10-04 01:21:46)
bq.  
bq.  
bq.  Review request for hbase and Mingjie Lai.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  Proposed fix for HBASE-4070. 
bq.  
bq.  
bq.  This addresses bug HBASE-4070.
bq.  https://issues.apache.org/jira/browse/HBASE-4070
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.src/main/jamon/org/apache/hbase/tmpl/master/MasterStatusTmpl.jamon 
abeb850 
bq.src/main/jamon/org/apache/hbase/tmpl/regionserver/RSStatusTmpl.jamon 
be6fceb 
bq.src/main/java/org/apache/hadoop/hbase/ClusterStatus.java 01bc1dd 
bq.src/main/java/org/apache/hadoop/hbase/HServerLoad.java 0c680e4 
bq.src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java a55a4b1 
bq.src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java 
dbae4fd 
bq.src/main/java/org/apache/hadoop/hbase/master/HMaster.java f80d232 
bq.src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 
3840279 
bq.
src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorReporting.java 
PRE-CREATION 
bq.  
bq.  Diff: https://reviews.apache.org/r/2029/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  Two new tests : testRegionServerCoprocessorReported() and 
testMasterServerCoprocessorsReported() included in a new source file 
src/test/java/o.a.h.h/coprocessor/TestCoprocessorReporting.java.
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Eugene
bq.  
bq.



 [Coprocessors] Improve region server metrics to report loaded coprocessors to 
 master
 

 Key: HBASE-4070
 URL: https://issues.apache.org/jira/browse/HBASE-4070
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.90.3
Reporter: Mingjie Lai
Assignee: Eugene Koontz
 Attachments: HBASE-4070.patch, HBASE-4070.patch, HBASE-4070.patch, 
 master-web-ui.jpg, rs-status-web-ui.jpg


 HBASE-3512 is about listing loaded cp classes at shell. To make it more 
 generic, we need a way to report this piece of information from region to 
 master (or just at region server level). So later on, we can display the 
 loaded class names at shell as well as web console. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-4537) Bad imports in TestUser from HBASE-4515 change

2011-10-04 Thread Gary Helmling (Created) (JIRA)

Bad imports in TestUser from HBASE-4515 change
--

 Key: HBASE-4537
 URL: https://issues.apache.org/jira/browse/HBASE-4537
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0, 0.94.0, 0.90.5
Reporter: Gary Helmling


The HBASE-4515 patch mistakenly added unused imports to TestUser:
{code}
import org.apache.hadoop.security.UnixUserGroupInformation;
import org.apache.hadoop.security.UserGroupInformation;
{code}

Both should be removed.  UnixUserGroupInformation is specific to non-secure 
Hadoop and importing it will break compilation against secure Hadoop variants 
(0.20.205, 0.22, 0.23).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4377) [hbck] Offline rebuild .META. from fs data only.

2011-10-04 Thread Jonathan Hsieh (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-4377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120561#comment-13120561
]

Jonathan Hsieh commented on HBASE-4377:
---

Although I've gotten this to work with live systems, it seems like that there
are some problems with the testing on the backports. Different versions have
different expected values which does not seem to make sense. HBASE-3777
changed some of the semantics of the HBaseTestingUtility so I'll be
investigating more.

[hbck] Offline rebuild .META. from fs data only.

Key: HBASE-4377
URL: https://issues.apache.org/jira/browse/HBASE-4377
Project: HBase
Issue Type: New Feature
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Attachments:
0001-HBASE-4377-hbck-Offline-rebuild-.META.-from-fs-data-.patch,
hbase-4377-trunk.v2.patch

In a worst case situation, it may be helpful to have an offline .META.
rebuilder that just looks at the file system's .regioninfos and rebuilds meta
from scratch. Users could move bad regions out until there is a clean
rebuild.
It would likely fill in region split holes. Follow on work could given
options to merge or select regions that overlap, or do online rebuilds.

[jira] [Commented] (HBASE-4537) Bad imports in TestUser from HBASE-4515 change


[ 
https://issues.apache.org/jira/browse/HBASE-4537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120565#comment-13120565
 ] 

stack commented on HBASE-4537:
--

I removed these two imports and it builds against 0.20.205.x for me.

 Bad imports in TestUser from HBASE-4515 change
 --

 Key: HBASE-4537
 URL: https://issues.apache.org/jira/browse/HBASE-4537
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0, 0.94.0, 0.90.5
Reporter: Gary Helmling

 The HBASE-4515 patch mistakenly added unused imports to TestUser:
 {code}
 import org.apache.hadoop.security.UnixUserGroupInformation;
 import org.apache.hadoop.security.UserGroupInformation;
 {code}
 Both should be removed.  UnixUserGroupInformation is specific to non-secure 
 Hadoop and importing it will break compilation against secure Hadoop variants 
 (0.20.205, 0.22, 0.23).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4482) Race Condition Concerning Eviction in SlabCache


[ 
https://issues.apache.org/jira/browse/HBASE-4482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120566#comment-13120566
 ] 

stack commented on HBASE-4482:
--

bq. The new cache is already marked experimental, right? If so, I don't think 
we have to necessarily back it out - just so long as people are clear that when 
they enable it they're playing with some fire 

Makes sense.

 Race Condition Concerning Eviction in SlabCache
 ---

 Key: HBASE-4482
 URL: https://issues.apache.org/jira/browse/HBASE-4482
 Project: HBase
  Issue Type: Sub-task
Reporter: Li Pi
Assignee: Li Pi
Priority: Blocker
 Fix For: 0.92.0

 Attachments: hbase-4482v1.txt, hbase-4482v2.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4377) [hbck] Offline rebuild .META. from fs data only.


[ 
https://issues.apache.org/jira/browse/HBASE-4377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120570#comment-13120570
 ] 

Ted Yu commented on HBASE-4377:
---

HBASE-4508 would backport HBASE-3777 to 0.90
We should get consistent behavior from HBaseTestingUtility after HBASE-4508 
goes in.

 [hbck] Offline rebuild .META. from fs data only.
 

 Key: HBASE-4377
 URL: https://issues.apache.org/jira/browse/HBASE-4377
 Project: HBase
  Issue Type: New Feature
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
 Attachments: 
 0001-HBASE-4377-hbck-Offline-rebuild-.META.-from-fs-data-.patch, 
 hbase-4377-trunk.v2.patch


 In a worst case situation, it may be helpful to have an offline .META. 
 rebuilder that just looks at the file system's .regioninfos and rebuilds meta 
 from scratch.  Users could move bad regions out until there is a clean 
 rebuild.  
 It would likely fill in region split holes.  Follow on work could given 
 options to merge or select regions that overlap, or do online rebuilds.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4300) Start of new-version master fails if old master's znode is hanging around


[ 
https://issues.apache.org/jira/browse/HBASE-4300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120573#comment-13120573
 ] 

stack commented on HBASE-4300:
--

Was going to commit in next day or so unless objection.

 Start of new-version master fails if old master's znode is hanging around
 -

 Key: HBASE-4300
 URL: https://issues.apache.org/jira/browse/HBASE-4300
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Assignee: stack
Priority: Critical
 Fix For: 0.92.0

 Attachments: 4300-v2.txt, 4300.txt


 I shut down an 0.90 cluster, and had to do so uncleanly. I then started a 
 trunk (0.92) cluster before the old master znode had expired. This cased:
 java.lang.StringIndexOutOfBoundsException: String index out of range: -1
 at java.lang.String.substring(String.java:1937)
 at 
 org.apache.hadoop.hbase.ServerName.parseHostname(ServerName.java:81)
 at org.apache.hadoop.hbase.ServerName.init(ServerName.java:63)
 at 
 org.apache.hadoop.hbase.master.ActiveMasterManager.blockUntilBecomingActiveMaster(ActiveMasterManager.java:148)
 at 
 org.apache.hadoop.hbase.master.HMaster.becomeActiveMaster(HMaster.java:342)
 at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:297)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4537) Bad imports in TestUser from HBASE-4515 change

2011-10-04 Thread Gary Helmling (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-4537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gary Helmling updated HBASE-4537:
-

Attachment: HBASE-4537.patch

Patch applied to 0.90, 0.92 and trunk.

 Bad imports in TestUser from HBASE-4515 change
 --

 Key: HBASE-4537
 URL: https://issues.apache.org/jira/browse/HBASE-4537
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0, 0.94.0, 0.90.5
Reporter: Gary Helmling
 Attachments: HBASE-4537.patch


 The HBASE-4515 patch mistakenly added unused imports to TestUser:
 {code}
 import org.apache.hadoop.security.UnixUserGroupInformation;
 import org.apache.hadoop.security.UserGroupInformation;
 {code}
 Both should be removed.  UnixUserGroupInformation is specific to non-secure 
 Hadoop and importing it will break compilation against secure Hadoop variants 
 (0.20.205, 0.22, 0.23).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HBASE-4537) Bad imports in TestUser from HBASE-4515 change

2011-10-04 Thread Gary Helmling (Resolved) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-4537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gary Helmling resolved HBASE-4537.
--

   Resolution: Fixed
Fix Version/s: 0.90.5
 Assignee: Gary Helmling

Committed trivial fix to branches 0.90, 0.92 and trunk.

 Bad imports in TestUser from HBASE-4515 change
 --

 Key: HBASE-4537
 URL: https://issues.apache.org/jira/browse/HBASE-4537
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0, 0.94.0, 0.90.5
Reporter: Gary Helmling
Assignee: Gary Helmling
 Fix For: 0.90.5

 Attachments: HBASE-4537.patch


 The HBASE-4515 patch mistakenly added unused imports to TestUser:
 {code}
 import org.apache.hadoop.security.UnixUserGroupInformation;
 import org.apache.hadoop.security.UserGroupInformation;
 {code}
 Both should be removed.  UnixUserGroupInformation is specific to non-secure 
 Hadoop and importing it will break compilation against secure Hadoop variants 
 (0.20.205, 0.22, 0.23).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4501) [replication] Shutting down a stream leaves recovered sources running

2011-10-04 Thread Jean-Daniel Cryans (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-4501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jean-Daniel Cryans updated HBASE-4501:
--

Attachment: HBASE-4501.patch

Patch that adds the terminate just before we close the recovered queue. Tested 
it on a simple 2 clusters setup where I first killed the second, then killed 
the first, restart it then did a remove_peer.

 [replication] Shutting down a stream leaves recovered sources running
 -

 Key: HBASE-4501
 URL: https://issues.apache.org/jira/browse/HBASE-4501
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.4
Reporter: Jean-Daniel Cryans
 Fix For: 0.92.0

 Attachments: HBASE-4501.patch


 When removing a peer it will call ReplicationSourceManager.removePeer which 
 calls closeRecoveredQueue which does this:
 {code}
 LOG.info(Done with the recovered queue  + src.getPeerClusterZnode());
 this.oldsources.remove(src);
 this.zkHelper.deleteSource(src.getPeerClusterZnode(), false);
 {code}
 This works in the case where the recovered source is done and is calling this 
 method, but when removing a peer it never calls terminate on thus it leaving 
 it running.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4386) NPE in TaskMonitor

2011-10-04 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120580#comment-13120580
 ] 

stack commented on HBASE-4386:
--

This seems reasonable.  All other accesses on tasks are synchronized on this.  
+1.

 NPE in TaskMonitor
 --

 Key: HBASE-4386
 URL: https://issues.apache.org/jira/browse/HBASE-4386
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Assignee: Todd Lipcon
Priority: Critical
 Fix For: 0.92.0

 Attachments: hbase-4386.txt


 Saw the following hitting /rs-status
 preINTERNAL_SERVER_ERROR/pre/ph3Caused 
 by:/h3prejava.lang.NullPointerException
 at 
 org.apache.hadoop.hbase.monitoring.TaskMonitor.purgeExpiredTasks(TaskMonitor.java:97)
 at 
 org.apache.hadoop.hbase.monitoring.TaskMonitor.getTasks(TaskMonitor.java:127)
 at 
 org.apache.hbase.tmpl.common.TaskMonitorTmplImpl.renderNoFlush(TaskMonitorTmplImpl.java:50)
 at 
 org.apache.hbase.tmpl.common.TaskMonitorTmpl.renderNoFlush(TaskMonitorTmpl.java:170)
 at 
 org.apache.hbase.tmpl.regionserver.RSStatusTmplImpl.renderNoFlush(RSStatusTmplImpl.java:70)
 at 
 org.apache.hbase.tmpl.regionserver.RSStatusTmpl.renderNoFlush(RSStatusTmpl.java:176)
 at 
 org.apache.hbase.tmpl.regionserver.RSStatusTmpl.render(RSStatusTmpl.java:167)
 at 
 org.apache.hadoop.hbase.regionserver.RSStatusServlet.doGet(RSStatusServlet.java:48)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4422) Move block cache parameters and references into single CacheConf class


[ 
https://issues.apache.org/jira/browse/HBASE-4422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120582#comment-13120582
 ] 

jirapos...@reviews.apache.org commented on HBASE-4422:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2089/
---

(Updated 2011-10-04 23:27:07.734296)


Review request for hbase, Dhruba Borthakur, Michael Stack, and Li Pi.


Changes
---

Addresses Mikhail and Stack's review comments.

I've completely removed all external calls to the CacheConfig() constructor, 
always passing conf now.

Just kicked off a unit test run.


Summary
---

Creates a new CacheConfig class and moves almost everything block cache related 
into this single class.  Adding new configuration params and booleans and such 
should be much better.

All tests are NOT passing yet, still working on it, but wanted to have 
something up today.  Basically code complete but broken :)


This addresses bug HBASE-4422.
https://issues.apache.org/jira/browse/HBASE-4422


Diffs (updated)
-

  /src/main/java/org/apache/hadoop/hbase/io/HalfStoreFileReader.java 1179008 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileReader.java 
1179008 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileWriter.java 
1179008 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/BlockCache.java 1179008 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java PRE-CREATION 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/DoubleBlockCache.java 1179008 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java 1179008 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/HFilePrettyPrinter.java 
1179008 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV1.java 1179008 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java 1179008 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV1.java 1179008 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java 1179008 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java 1179008 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/SimpleBlockCache.java 1179008 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/slab/SingleSizeCache.java 
1179008 
  /src/main/java/org/apache/hadoop/hbase/io/hfile/slab/SlabCache.java 1179008 
  /src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java 
1179008 
  /src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 
1179008 
  /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1179008 
  /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 
1179008 
  /src/main/java/org/apache/hadoop/hbase/regionserver/Store.java 1179008 
  /src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java 1179008 
  /src/main/java/org/apache/hadoop/hbase/util/BloomFilterFactory.java 1179008 
  /src/main/java/org/apache/hadoop/hbase/util/CompressionTest.java 1179008 
  /src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java 
1179008 
  /src/test/java/org/apache/hadoop/hbase/io/TestHalfStoreFileReader.java 
1179008 
  /src/test/java/org/apache/hadoop/hbase/io/hfile/RandomSeek.java 1179008 
  /src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java 1179008 
  /src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFile.java 1179008 
  /src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlockIndex.java 
1179008 
  /src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFilePerformance.java 
1179008 
  /src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileReaderV1.java 
1179008 
  /src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileSeek.java 1179008 
  /src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileWriterV2.java 
1179008 
  /src/test/java/org/apache/hadoop/hbase/io/hfile/TestReseekTo.java 1179008 
  /src/test/java/org/apache/hadoop/hbase/io/hfile/TestSeekTo.java 1179008 
  /src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java 
1179008 
  
/src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFiles.java 
1179008 
  /src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksRead.java 
1179008 
  /src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactSelection.java 
1179008 
  
/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompoundBloomFilter.java
 1179008 
  /src/test/java/org/apache/hadoop/hbase/regionserver/TestFSErrorsExposed.java 
1179008 
  /src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java 
1179008 
  
/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFileBlockCacheSummary.java
 1179008 
  /src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALReplay.java 
1179008 

Diff: https://reviews.apache.org/r/2089/diff

[jira] [Updated] (HBASE-4494) AvroServer:: get fails with NPE on a non-existent row

2011-10-04 Thread Kay Kay (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-4494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kay Kay updated HBASE-4494:
---

Resolution: Fixed
Status: Resolved  (was: Patch Available)

Patch available and attached. 

 AvroServer:: get fails with NPE on a non-existent row
 -

 Key: HBASE-4494
 URL: https://issues.apache.org/jira/browse/HBASE-4494
 Project: HBase
  Issue Type: Bug
  Components: avro
Affects Versions: 0.90.4
Reporter: Kay Kay
Assignee: Kay Kay
 Fix For: 0.90.5

 Attachments: HBASE-4494.patch


 Try to submit a get request to the avro gateway. 
 If the row specified for a given table does not exist, the server request 
 fails with a NPE.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4508) Backport HBASE-3777 to 0.90 branch

2011-10-04 Thread Bright Fulton (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-4508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bright Fulton updated HBASE-4508:
-

Attachment: HBASE-4508.v3.patch

Attached cumulative v3 patch which includes a one line change to 
hbase.connection.per.config functionality to work with existing HConnectionKey 
hash and equality.  Fixes TestZooKeeper.testMultipleZK.

 Backport HBASE-3777 to 0.90 branch
 --

 Key: HBASE-4508
 URL: https://issues.apache.org/jira/browse/HBASE-4508
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
 Attachments: HBASE-4508.v1.patch, HBASE-4508.v2.patch, 
 HBASE-4508.v3.patch


 See discussion here: 
 http://search-hadoop.com/m/MJBId1aazTR1/backporting+HBASE-3777+to+0.90subj=backporting+HBASE+3777+to+0+90
 Rocketfuel has been running 0.90.3 with HBASE-3777 since its resolution.
 They have 10 RS nodes , 1 Master and 1 Zookeeper
 Live writes and reads but super heavy on reads. Cache hit is pretty high.
 The qps on one of their data centers is 50K.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4501) [replication] Shutting down a stream leaves recovered sources running


[ 
https://issues.apache.org/jira/browse/HBASE-4501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120593#comment-13120593
 ] 

stack commented on HBASE-4501:
--

+1 lgtm

 [replication] Shutting down a stream leaves recovered sources running
 -

 Key: HBASE-4501
 URL: https://issues.apache.org/jira/browse/HBASE-4501
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.4
Reporter: Jean-Daniel Cryans
 Fix For: 0.92.0

 Attachments: HBASE-4501.patch


 When removing a peer it will call ReplicationSourceManager.removePeer which 
 calls closeRecoveredQueue which does this:
 {code}
 LOG.info(Done with the recovered queue  + src.getPeerClusterZnode());
 this.oldsources.remove(src);
 this.zkHelper.deleteSource(src.getPeerClusterZnode(), false);
 {code}
 This works in the case where the recovered source is done and is calling this 
 method, but when removing a peer it never calls terminate on thus it leaving 
 it running.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4494) AvroServer:: get fails with NPE on a non-existent row


 [ 
https://issues.apache.org/jira/browse/HBASE-4494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-4494:
-

Fix Version/s: (was: 0.90.5)
   0.92.0

Marking fixed in 0.92.  I applied to 0.92 branch and to TRUNK.  Thanks for the 
patch Kay Kay.  In future, we resolve the issue after the patch has been 
applied.  You can mark its state as 'patch available' after you've uploaded 
patch.

 AvroServer:: get fails with NPE on a non-existent row
 -

 Key: HBASE-4494
 URL: https://issues.apache.org/jira/browse/HBASE-4494
 Project: HBase
  Issue Type: Bug
  Components: avro
Affects Versions: 0.90.4
Reporter: Kay Kay
Assignee: Kay Kay
 Fix For: 0.92.0

 Attachments: HBASE-4494.patch


 Try to submit a get request to the avro gateway. 
 If the row specified for a given table does not exist, the server request 
 fails with a NPE.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4536) Allow CF to retain deleted rows

[
https://issues.apache.org/jira/browse/HBASE-4536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120603#comment-13120603
]

Lars Hofhansl commented on HBASE-4536:
--

Re: New config param. Wasn't sure in the beginning, but now I think this
should have really been part of the HBASE-4071.
With config param you mean a new flag on HColumnDescriptor? That would provide
for more flexibility, the question is whether it is needed, or would just
confuse any users (the minversion feature is confusing as is :) ).
I'm not opposed to the separate option for this, though.

Allow CF to retain deleted rows
---

[jira] [Commented] (HBASE-4501) [replication] Shutting down a stream leaves recovered sources running


[ 
https://issues.apache.org/jira/browse/HBASE-4501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120621#comment-13120621
 ] 

Lars Hofhansl commented on HBASE-4501:
--

+1

 [replication] Shutting down a stream leaves recovered sources running
 -

 Key: HBASE-4501
 URL: https://issues.apache.org/jira/browse/HBASE-4501
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.4
Reporter: Jean-Daniel Cryans
 Fix For: 0.92.0

 Attachments: HBASE-4501.patch


 When removing a peer it will call ReplicationSourceManager.removePeer which 
 calls closeRecoveredQueue which does this:
 {code}
 LOG.info(Done with the recovered queue  + src.getPeerClusterZnode());
 this.oldsources.remove(src);
 this.zkHelper.deleteSource(src.getPeerClusterZnode(), false);
 {code}
 This works in the case where the recovered source is done and is calling this 
 method, but when removing a peer it never calls terminate on thus it leaving 
 it running.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4536) Allow CF to retain deleted rows

2011-10-04 Thread jirapos...@reviews.apache.org (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-4536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120626#comment-13120626
]

Ted Yu commented on HBASE-4536:
---

Or maybe this feature should latch onto minversion being 0 so that we don't
introduce additional parameter ?

Allow CF to retain deleted rows
---

[jira] [Commented] (HBASE-4536) Allow CF to retain deleted rows

[
https://issues.apache.org/jira/browse/HBASE-4536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120629#comment-13120629
]

jirapos...@reviews.apache.org commented on HBASE-4536:
--

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2178/
---

(Updated 2011-10-05 01:06:33.205905)

Review request for hbase.

Changes
---

Version that actually compiles.

Summary
---

The change is pretty small, but the logic is intricate, so please review
carefully.

This addresses bug HBASE-4536.
https://issues.apache.org/jira/browse/HBASE-4536

Diffs (updated)
-

http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java
1178891

http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ColumnTracker.java
1178891

http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ExplicitColumnTracker.java
1178891

http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
1178891

http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanWildcardColumnTracker.java
1178891

http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
1178891

http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestExplicitColumnTracker.java
1178891

http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestMinVersions.java
1178891

http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestScanWildcardColumnTracker.java
1178891

Diff: https://reviews.apache.org/r/2178/diff

Testing
---

TestMinVersions (with a new test for this) passed. Have not run full suite,
yet. This is for initial feedback.

Thanks,

Lars

Allow CF to retain deleted rows
---

[jira] [Commented] (HBASE-4537) Bad imports in TestUser from HBASE-4515 change


[ 
https://issues.apache.org/jira/browse/HBASE-4537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120631#comment-13120631
 ] 

Hudson commented on HBASE-4537:
---

Integrated in HBase-0.92 #43 (See 
[https://builds.apache.org/job/HBase-0.92/43/])
HBASE-4537  Unused TestUser imports break compilation against secure Hadoop

garyh : 
Files : 
* /hbase/branches/0.92/CHANGES.txt
* 
/hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/security/TestUser.java


 Bad imports in TestUser from HBASE-4515 change
 --

 Key: HBASE-4537
 URL: https://issues.apache.org/jira/browse/HBASE-4537
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0, 0.94.0, 0.90.5
Reporter: Gary Helmling
Assignee: Gary Helmling
 Fix For: 0.90.5

 Attachments: HBASE-4537.patch


 The HBASE-4515 patch mistakenly added unused imports to TestUser:
 {code}
 import org.apache.hadoop.security.UnixUserGroupInformation;
 import org.apache.hadoop.security.UserGroupInformation;
 {code}
 Both should be removed.  UnixUserGroupInformation is specific to non-secure 
 Hadoop and importing it will break compilation against secure Hadoop variants 
 (0.20.205, 0.22, 0.23).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4422) Move block cache parameters and references into single CacheConf class

2011-10-04 Thread Jonathan Gray (Updated) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-4422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jonathan Gray updated HBASE-4422:
-

Attachment: CacheConfig92-v8.patch

Most recent patch from RB. All tests are passing.

I will definitely be pulling this into our internal 92 branch and will be
building some other stuff on top of it (compressed cache, etc). It will
probably be easiest to integrate the DeltaEncoding stuff with this in place as
well.

For that reason, I'd be interested in getting it into 92 branch, it should be
low risk.

Move block cache parameters and references into single CacheConf class
--

Key: HBASE-4422
URL: https://issues.apache.org/jira/browse/HBASE-4422
Project: HBase
Issue Type: Improvement
Components: io
Reporter: Jonathan Gray
Assignee: Jonathan Gray
Fix For: 0.92.0

Attachments: CacheConfig92-v8.patch

From StoreFile down to HFile, we currently use a boolean argument for each of
the various block cache configuration parameters that exist. The number of
parameters is going to continue to increase as we look at compressed cache,
delta encoding, and more specific L1/L2 configuration. Every new config
currently requires changing many constructors because it introduces a new
boolean.
We should move everything into a single class so that modifications are much
less disruptive.

[jira] [Commented] (HBASE-4422) Move block cache parameters and references into single CacheConf class

2011-10-04 Thread Jonathan Gray (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120637#comment-13120637
 ] 

Jonathan Gray commented on HBASE-4422:
--

Yeah, if this goes to trunk but not 92 then begins the fun of rebasing patches 
for each because it changes so many constructors in/around HFile.

 Move block cache parameters and references into single CacheConf class
 --

 Key: HBASE-4422
 URL: https://issues.apache.org/jira/browse/HBASE-4422
 Project: HBase
  Issue Type: Improvement
  Components: io
Reporter: Jonathan Gray
Assignee: Jonathan Gray
 Fix For: 0.92.0

 Attachments: CacheConfig92-v8.patch


 From StoreFile down to HFile, we currently use a boolean argument for each of 
 the various block cache configuration parameters that exist.  The number of 
 parameters is going to continue to increase as we look at compressed cache, 
 delta encoding, and more specific L1/L2 configuration.  Every new config 
 currently requires changing many constructors because it introduces a new 
 boolean.
 We should move everything into a single class so that modifications are much 
 less disruptive.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4422) Move block cache parameters and references into single CacheConf class


[ 
https://issues.apache.org/jira/browse/HBASE-4422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120644#comment-13120644
 ] 

Ted Yu commented on HBASE-4422:
---

+1 on integrating to 0.92 branch.

 Move block cache parameters and references into single CacheConf class
 --

 Key: HBASE-4422
 URL: https://issues.apache.org/jira/browse/HBASE-4422
 Project: HBase
  Issue Type: Improvement
  Components: io
Reporter: Jonathan Gray
Assignee: Jonathan Gray
 Fix For: 0.92.0

 Attachments: CacheConfig92-v8.patch


 From StoreFile down to HFile, we currently use a boolean argument for each of 
 the various block cache configuration parameters that exist.  The number of 
 parameters is going to continue to increase as we look at compressed cache, 
 delta encoding, and more specific L1/L2 configuration.  Every new config 
 currently requires changing many constructors because it introduces a new 
 boolean.
 We should move everything into a single class so that modifications are much 
 less disruptive.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4536) Allow CF to retain deleted rows

[
https://issues.apache.org/jira/browse/HBASE-4536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120654#comment-13120654
]

Lars Hofhansl commented on HBASE-4536:
--

@Ted, yep... That's what the parent is :)

Allow CF to retain deleted rows
---

[jira] [Commented] (HBASE-4479) TestMasterFailover failure in Hbase-0.92#17

2011-10-04 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120665#comment-13120665
 ] 

Lars Hofhansl commented on HBASE-4479:
--

+1 looks good to me

 TestMasterFailover failure in Hbase-0.92#17
 ---

 Key: HBASE-4479
 URL: https://issues.apache.org/jira/browse/HBASE-4479
 Project: HBase
  Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Minor
 Attachments: HBASE-4479.patch


 When the master restarted it was not able to get any servers online and the 
 restart was a clean restart.
 Hence there were no regions to assign.
 Hence the retainAssignment tries to get one of the regions and uses 
 RANDOM.getInt(size).  Here size is 0.
 So ideally 0 is not accepted here.  Hence we have got an exception making the 
 master not to come up and the test case timeout.
 Though we need to see if really no regions was expected when the master came 
 up, but this JIRA's intent is to deal such scenario where the size can be 0.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4070) [Coprocessors] Improve region server metrics to report loaded coprocessors to master


[ 
https://issues.apache.org/jira/browse/HBASE-4070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120671#comment-13120671
 ] 

jirapos...@reviews.apache.org commented on HBASE-4070:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2029/#review2336
---



src/main/jamon/org/apache/hbase/tmpl/regionserver/RSStatusTmpl.jamon
https://reviews.apache.org/r/2029/#comment5398

missing /tr



src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorReporting.java
https://reviews.apache.org/r/2029/#comment5397

should be getCoprocessors()


- Eugene


On 2011-10-04 01:21:46, Eugene Koontz wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/2029/
bq.  ---
bq.  
bq.  (Updated 2011-10-04 01:21:46)
bq.  
bq.  
bq.  Review request for hbase and Mingjie Lai.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  Proposed fix for HBASE-4070. 
bq.  
bq.  
bq.  This addresses bug HBASE-4070.
bq.  https://issues.apache.org/jira/browse/HBASE-4070
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.src/main/jamon/org/apache/hbase/tmpl/master/MasterStatusTmpl.jamon 
abeb850 
bq.src/main/jamon/org/apache/hbase/tmpl/regionserver/RSStatusTmpl.jamon 
be6fceb 
bq.src/main/java/org/apache/hadoop/hbase/ClusterStatus.java 01bc1dd 
bq.src/main/java/org/apache/hadoop/hbase/HServerLoad.java 0c680e4 
bq.src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java a55a4b1 
bq.src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java 
dbae4fd 
bq.src/main/java/org/apache/hadoop/hbase/master/HMaster.java f80d232 
bq.src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 
3840279 
bq.
src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorReporting.java 
PRE-CREATION 
bq.  
bq.  Diff: https://reviews.apache.org/r/2029/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  Two new tests : testRegionServerCoprocessorReported() and 
testMasterServerCoprocessorsReported() included in a new source file 
src/test/java/o.a.h.h/coprocessor/TestCoprocessorReporting.java.
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Eugene
bq.  
bq.



 [Coprocessors] Improve region server metrics to report loaded coprocessors to 
 master
 

 Key: HBASE-4070
 URL: https://issues.apache.org/jira/browse/HBASE-4070
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.90.3
Reporter: Mingjie Lai
Assignee: Eugene Koontz
 Attachments: HBASE-4070.patch, HBASE-4070.patch, HBASE-4070.patch, 
 master-web-ui.jpg, rs-status-web-ui.jpg


 HBASE-3512 is about listing loaded cp classes at shell. To make it more 
 generic, we need a way to report this piece of information from region to 
 master (or just at region server level). So later on, we can display the 
 loaded class names at shell as well as web console. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4479) TestMasterFailover failure in Hbase-0.92#17

2011-10-04 Thread ramkrishna.s.vasudevan (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-4479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-4479:
--

Fix Version/s: 0.92.0
 Hadoop Flags: Reviewed
   Status: Patch Available  (was: Open)

Integrated to trunk and 0.92.

Thanks for the review Ted and Lars.

 TestMasterFailover failure in Hbase-0.92#17
 ---

 Key: HBASE-4479
 URL: https://issues.apache.org/jira/browse/HBASE-4479
 Project: HBase
  Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Minor
 Fix For: 0.92.0

 Attachments: HBASE-4479.patch


 When the master restarted it was not able to get any servers online and the 
 restart was a clean restart.
 Hence there were no regions to assign.
 Hence the retainAssignment tries to get one of the regions and uses 
 RANDOM.getInt(size).  Here size is 0.
 So ideally 0 is not accepted here.  Hence we have got an exception making the 
 master not to come up and the test case timeout.
 Though we need to see if really no regions was expected when the master came 
 up, but this JIRA's intent is to deal such scenario where the size can be 0.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4422) Move block cache parameters and references into single CacheConf class


[ 
https://issues.apache.org/jira/browse/HBASE-4422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120678#comment-13120678
 ] 

stack commented on HBASE-4422:
--

As RM, I will not let this patch into 0.92.  Not unless you review hbase-3446!  
Then I might consider it.

+1 on commit to 0.92.  Looks low risk cleanup to me.  I don't like getting the 
cache instance from the CacheConf so we should file a new issue for that too 
before it can go in.

Good stuff.

 Move block cache parameters and references into single CacheConf class
 --

 Key: HBASE-4422
 URL: https://issues.apache.org/jira/browse/HBASE-4422
 Project: HBase
  Issue Type: Improvement
  Components: io
Reporter: Jonathan Gray
Assignee: Jonathan Gray
 Fix For: 0.92.0

 Attachments: CacheConfig92-v8.patch


 From StoreFile down to HFile, we currently use a boolean argument for each of 
 the various block cache configuration parameters that exist.  The number of 
 parameters is going to continue to increase as we look at compressed cache, 
 delta encoding, and more specific L1/L2 configuration.  Every new config 
 currently requires changing many constructors because it introduces a new 
 boolean.
 We should move everything into a single class so that modifications are much 
 less disruptive.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4508) Backport HBASE-3777 to 0.90 branch

2011-10-04 Thread Bright Fulton (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-4508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bright Fulton updated HBASE-4508:
-

Attachment: HBASE-4508.v4.patch

Attached v4: changes to TestHCM so that new and newly enabled tests there unset 
hbase.connection.per.config to test shared-connection behavior.


 Backport HBASE-3777 to 0.90 branch
 --

 Key: HBASE-4508
 URL: https://issues.apache.org/jira/browse/HBASE-4508
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
 Attachments: HBASE-4508.v1.patch, HBASE-4508.v2.patch, 
 HBASE-4508.v3.patch, HBASE-4508.v4.patch


 See discussion here: 
 http://search-hadoop.com/m/MJBId1aazTR1/backporting+HBASE-3777+to+0.90subj=backporting+HBASE+3777+to+0+90
 Rocketfuel has been running 0.90.3 with HBASE-3777 since its resolution.
 They have 10 RS nodes , 1 Master and 1 Zookeeper
 Live writes and reads but super heavy on reads. Cache hit is pretty high.
 The qps on one of their data centers is 50K.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4402) Retaining locality after restart broken


[ 
https://issues.apache.org/jira/browse/HBASE-4402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120680#comment-13120680
 ] 

stack commented on HBASE-4402:
--

Ran test suite w/o on 0.92 and all tests passed and with it these failed:

{code}
Failed tests:   
testOnlineChangeTableSchema(org.apache.hadoop.hbase.client.TestAdmin)
  testForceSplit(org.apache.hadoop.hbase.client.TestAdmin): expected:2 but 
was:1
  testForceSplitMultiFamily(org.apache.hadoop.hbase.client.TestAdmin): 
expected:2 but was:1

Tests in error:
  testMergeTable(org.apache.hadoop.hbase.util.TestMergeTable): test timed out 
after 30 milliseconds
{code}

I'm taking a look.

 Retaining locality after restart broken
 ---

 Key: HBASE-4402
 URL: https://issues.apache.org/jira/browse/HBASE-4402
 Project: HBase
  Issue Type: Bug
  Components: master
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Assignee: Todd Lipcon
Priority: Blocker
 Fix For: 0.92.0

 Attachments: hbase-4402.txt, hbase-4402.txt


 In DefaultLoadBalancer, we implement the retain assignment function like so:
 {code}
   if (sn != null  servers.contains(sn)) {
 assignments.get(sn).add(region.getKey());
 {code}
 but this will never work since after a cluster restart, all servers have a 
 new ServerName with a new startcode.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-4538) NPE in AssignmentManager#updateTimers

2011-10-04 Thread stack (Created) (JIRA)

NPE in AssignmentManager#updateTimers
-

 Key: HBASE-4538
 URL: https://issues.apache.org/jira/browse/HBASE-4538
 Project: HBase
  Issue Type: Bug
Reporter: stack
 Fix For: 0.92.0


Saw this in a failed TestAdmin on 0.92

{code}
2011-10-05 01:18:58,890 ERROR 
[MASTER_OPEN_REGION-sv4r9s38,52146,131098450-2] executor.EventHandler(171): 
Caught throwable while processing event RS_ZK_REGION_OPENED
java.lang.NullPointerException
at 
org.apache.hadoop.hbase.master.AssignmentManager.updateTimers(AssignmentManager.java:1053)
at 
org.apache.hadoop.hbase.master.AssignmentManager.regionOnline(AssignmentManager.java:1027)
at 
org.apache.hadoop.hbase.master.handler.OpenedRegionHandler.process(OpenedRegionHandler.java:108)
at 
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:168)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
{code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4538) NPE in AssignmentManager#updateTimers


[ 
https://issues.apache.org/jira/browse/HBASE-4538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120685#comment-13120685
 ] 

stack commented on HBASE-4538:
--

Happened a bunch in this test.

 NPE in AssignmentManager#updateTimers
 -

 Key: HBASE-4538
 URL: https://issues.apache.org/jira/browse/HBASE-4538
 Project: HBase
  Issue Type: Bug
Reporter: stack
 Fix For: 0.92.0


 Saw this in a failed TestAdmin on 0.92
 {code}
 2011-10-05 01:18:58,890 ERROR 
 [MASTER_OPEN_REGION-sv4r9s38,52146,131098450-2] 
 executor.EventHandler(171): Caught throwable while processing event 
 RS_ZK_REGION_OPENED
 java.lang.NullPointerException
 at 
 org.apache.hadoop.hbase.master.AssignmentManager.updateTimers(AssignmentManager.java:1053)
 at 
 org.apache.hadoop.hbase.master.AssignmentManager.regionOnline(AssignmentManager.java:1027)
 at 
 org.apache.hadoop.hbase.master.handler.OpenedRegionHandler.process(OpenedRegionHandler.java:108)
 at 
 org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:168)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-3581) hbase rpc should send size of response