[jira] [Updated] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487

2012-04-17 Thread Todd Lipcon (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HBASE-5782:
---

Attachment: hbase-5782.txt

New rev - this one should even compile! :)

I tested a few of the hlog tests locally and they seem to pass. Also ran the 
standalone hlog tool with a bunch of threads and -verify. Throughput is 
identical, and errors are gone.

I'm nervous about error handling, though... seems like if the syncer thread had 
an error, another thread might wait forever.

I'd really like to rip this apart and make it all component-ized and 
unit-testable, but tried to keep this patch minimal.

 Edits can be appended out of seqid order since HBASE-4487
 -

 Key: HBASE-5782
 URL: https://issues.apache.org/jira/browse/HBASE-5782
 Project: HBase
  Issue Type: Bug
  Components: wal
Affects Versions: 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
Priority: Blocker
 Fix For: 0.94.0

 Attachments: 5782-sketch.txt, 5782.txt, 5782.unfinished-stack.txt, 
 HBASE-5782.patch, hbase-5782.txt


 Create a table with 1000 splits, after the region assignemnt, kill the 
 regionserver wich contains META table.
 Here few regions are missing after the log splitting and region assigment. 
 HBCK report shows multiple region holes are got created.
 Same scenario was verified mulitple times in 0.92.1, no issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487

2012-04-16 Thread Todd Lipcon (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HBASE-5782:
---

Attachment: 5782-sketch.txt

half-done patch of a proposed solution... got to run at the moment but will 
come back later with something more complete. The idea is to only have one 
seqnum in use, and make sure only a single thread pushes from the pending 
edits queue into a seqfile writer at a time.

 Edits can be appended out of seqid order since HBASE-4487
 -

 Key: HBASE-5782
 URL: https://issues.apache.org/jira/browse/HBASE-5782
 Project: HBase
  Issue Type: Bug
  Components: wal
Affects Versions: 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
Priority: Blocker
 Fix For: 0.94.0

 Attachments: 5782-sketch.txt, 5782.txt, HBASE-5782.patch


 Create a table with 1000 splits, after the region assignemnt, kill the 
 regionserver wich contains META table.
 Here few regions are missing after the log splitting and region assigment. 
 HBCK report shows multiple region holes are got created.
 Same scenario was verified mulitple times in 0.92.1, no issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5739) Upgrade guava to 11.0.2

2012-04-06 Thread Todd Lipcon (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HBASE-5739:
---

Attachment: hbase-5739.txt

Attached patch updates the version, and changes the construction of the map in 
the slab cache, since the old MapMaker has been removed from guava 11

 Upgrade guava to 11.0.2
 ---

 Key: HBASE-5739
 URL: https://issues.apache.org/jira/browse/HBASE-5739
 Project: HBase
  Issue Type: Improvement
  Components: build
Affects Versions: 0.96.0
Reporter: Todd Lipcon
Assignee: Todd Lipcon
 Fix For: 0.96.0

 Attachments: hbase-5739.txt


 Hadoop has upgraded to this new version of Guava. We should, too, so we don't 
 have compatibility issues running on Hadoop 2.0+

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-4608) HLog Compression

2012-03-14 Thread Todd Lipcon (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-4608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HBASE-4608:
---

Attachment: hbase-4608-v28.txt
hbase-4608-v28-delta.txt

I reviewed the latest patch and made some improvements:
- added a new true unit test for KeyValueCompression
- addressed one of my pieces of review feedback from earlier about the API for 
uncompressIntoArray
- renamed some methods for clarity
- removed some spurious whitespace changes
- added an enum for HLogKey version so that the comparisons are clearer
- renamed MAXSIZE to MAX_SIZE
- redid the linked list inside BidirectionalLRUMap, since the nomenclature was 
previously backwards and I found the code hard to read. (next is supposed to 
point towards the tail, not towards the head)
- changed getEntry() to throw an error if you pass an index larger than the 
current size

I ran the related unit tests and they passed, but did not try the full suite.

 HLog Compression
 

 Key: HBASE-4608
 URL: https://issues.apache.org/jira/browse/HBASE-4608
 Project: HBase
  Issue Type: New Feature
Reporter: Li Pi
Assignee: stack
 Fix For: 0.94.0

 Attachments: 4608-v19.txt, 4608-v20.txt, 4608-v22.txt, 4608v1.txt, 
 4608v13.txt, 4608v13.txt, 4608v14.txt, 4608v15.txt, 4608v16.txt, 4608v17.txt, 
 4608v18.txt, 4608v23.txt, 4608v24.txt, 4608v25.txt, 4608v27.txt, 4608v5.txt, 
 4608v6.txt, 4608v7.txt, 4608v8fixed.txt, hbase-4608-v28-delta.txt, 
 hbase-4608-v28.txt


 The current bottleneck to HBase write speed is replicating the WAL appends 
 across different datanodes. We can speed up this process by compressing the 
 HLog. Current plan involves using a dictionary to compress table name, region 
 id, cf name, and possibly other bits of repeated data. Also, HLog format may 
 be changed in other ways to produce a smaller HLog.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5552) Clean up our jmx view; its a bit of a mess

2012-03-09 Thread Todd Lipcon (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HBASE-5552:
---

Hadoop Flags: Incompatible change

 Clean up our jmx view; its a bit of a mess
 --

 Key: HBASE-5552
 URL: https://issues.apache.org/jira/browse/HBASE-5552
 Project: HBase
  Issue Type: Bug
Reporter: stack
Assignee: stack
Priority: Blocker
 Fix For: 0.92.1, 0.94.0

 Attachments: 0.92.0jmx.png, 5552.txt, currentjmxview.png, 
 patchedjmxview.png


 Fix before we release 0.92.1

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-3484) Replace memstore's ConcurrentSkipListMap with our own implementation

2012-02-22 Thread Todd Lipcon (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HBASE-3484:
---

Attachment: hierarchical-map.txt

Here's something I hacked together tonight which maps the memstore maps 
hierarchical. It should save a bit of CPU especially when doing wide puts, but 
I haven't done any serious benchmarking. It probably has negative memory 
effects in its current incarnation. Seems to kind-of work.

 Replace memstore's ConcurrentSkipListMap with our own implementation
 

 Key: HBASE-3484
 URL: https://issues.apache.org/jira/browse/HBASE-3484
 Project: HBase
  Issue Type: Improvement
  Components: performance
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Priority: Critical
 Attachments: hierarchical-map.txt


 By copy-pasting ConcurrentSkipListMap into HBase we can make two improvements 
 to it for our use case in MemStore:
 - add an iterator.replace() method which should allow us to do upsert much 
 more cheaply
 - implement a Set directly without having to do MapKeyValue,KeyValue to 
 save one reference per entry
 It turns out CSLM is in public domain from its development as part of JSR 
 166, so we should be OK with licenses.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5311) Allow inmemory Memstore compactions

2012-02-07 Thread Todd Lipcon (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HBASE-5311:
---

Attachment: InternallyLayeredMap.java

here's what I hacked up - the only guarantee I make about this code is that it 
has on syntax errors :)

 Allow inmemory Memstore compactions
 ---

 Key: HBASE-5311
 URL: https://issues.apache.org/jira/browse/HBASE-5311
 Project: HBase
  Issue Type: Improvement
Reporter: Lars Hofhansl
 Attachments: InternallyLayeredMap.java


 Just like we periodically compact the StoreFiles we should also periodically 
 compact the MemStore.
 During these compactions we eliminate deleted cells, expired cells, cells to 
 removed because of version count, etc, before we even do a memstore flush.
 Besides the optimization that we could get from this, it should also allow us 
 to remove the special handling of ICV, Increment, and Append (all of which 
 use upsert logic to avoid accumulating excessive cells in the Memstore).
 Not targeting this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-4403) Adopt interface stability/audience classifications from Hadoop

2012-02-07 Thread Todd Lipcon (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-4403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HBASE-4403:
---

Attachment: hbase-4403-nowhere-near-done.txt

here's an entirely unfinished patch where I started this last fall

 Adopt interface stability/audience classifications from Hadoop
 --

 Key: HBASE-4403
 URL: https://issues.apache.org/jira/browse/HBASE-4403
 Project: HBase
  Issue Type: Task
Affects Versions: 0.90.5, 0.92.0
Reporter: Todd Lipcon
Assignee: Todd Lipcon
 Attachments: hbase-4403-nowhere-near-done.txt


 As HBase gets more widely used, we need to be more explicit about which APIs 
 are stable and not expected to break between versions, which APIs are still 
 evolving, etc. We also have many public classes that are really internal to 
 the RS or Master and not meant to be used by users. Hadoop has adopted a 
 classification scheme for audience (public, private, or limited-private) as 
 well as stability (stable, evolving, unstable). I think we should copy these 
 annotations to HBase and start to classify our public classes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5244) Add a test for a row with 1M (10M? 100M) columns and see how we do with it

2012-01-21 Thread Todd Lipcon (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HBASE-5244:
---

Summary: Add a test for a row with 1M (10M? 100M) columns and see how we do 
with it  (was: Add a test for a column with 1M (10M? 100M) items and see how we 
do with it)

 Add a test for a row with 1M (10M? 100M) columns and see how we do with it
 --

 Key: HBASE-5244
 URL: https://issues.apache.org/jira/browse/HBASE-5244
 Project: HBase
  Issue Type: Task
Reporter: stack



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5231) Backport HBASE-3373 (per-table load balancing) to 0.90

2012-01-19 Thread Todd Lipcon (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HBASE-5231:
---

Summary: Backport HBASE-3373 (per-table load balancing) to 0.90  (was: 
Backport HBASE-3373 to 0.90)

 Backport HBASE-3373 (per-table load balancing) to 0.90
 --

 Key: HBASE-5231
 URL: https://issues.apache.org/jira/browse/HBASE-5231
 Project: HBase
  Issue Type: Improvement
Reporter: Zhihong Yu

 This JIRA backports per-table load balancing to 0.90

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-3421) Very wide rows -- 30M plus -- cause us OOME

2012-01-18 Thread Todd Lipcon (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HBASE-3421:
---

Release Note: 
A new config parameter, hbase.hstore.compaction.kv.max, has been added to 
limit the number of rows processed in each iteration of the internal compaction 
code.
Default value is 10.

  was:
A new config parameter, hbase.hstore.compaction.kv.max, has been added to 
limit the number of rows scanner returns in next().
Default value is 10.


 Very wide rows -- 30M plus -- cause us OOME
 ---

 Key: HBASE-3421
 URL: https://issues.apache.org/jira/browse/HBASE-3421
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.0
Reporter: stack
Assignee: Nate Putnam
 Fix For: 0.90.5

 Attachments: 3421.addendum, HBASE-3421.patch, HBASE-34211-v2.patch, 
 HBASE-34211-v3.patch, HBASE-34211-v4.patch


 From the list, see 'jvm oom' in 
 http://mail-archives.apache.org/mod_mbox/hbase-user/201101.mbox/browser, it 
 looks like wide rows -- 30M or so -- causes OOME during compaction.  We 
 should check it out. Can the scanner used during compactions use the 'limit' 
 when nexting?  If so, this should save our OOME'ing (or, we need to add to 
 the next a max size rather than count of KVs).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5017) Bump the default hfile.block.cache.size because of HFileV2

2011-12-13 Thread Todd Lipcon (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HBASE-5017:
---

Release Note: The default percentage of heap space allocated to the block 
cache has been increased from 20% to 25%. However, index caching is now 
accounted for in the block cache, so total heap usage should be minimally 
affected.
Hadoop Flags: Incompatible change

 Bump the default hfile.block.cache.size because of HFileV2
 --

 Key: HBASE-5017
 URL: https://issues.apache.org/jira/browse/HBASE-5017
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.92.0
Reporter: Jean-Daniel Cryans
Assignee: Jean-Daniel Cryans
Priority: Blocker
 Fix For: 0.92.0, 0.94.0

 Attachments: HBASE-5017.patch


 Here's the email I sent to the mailing list describing the problem:
 {quote}
 A thought just stuck me while I was writing down a more detailed block
 caching documentation: with HFileV2, the indexes now live in the block
 cache which means that those who upgrade may all of a sudden get
 terrible cache hit ratios because of all that memory taken by the
 indexes. This is somewhat mitigated by the fact that people don't
 usually need to keep _all_ the index blocks in memory so in the end
 we're more efficient.
 Which brings me to a question: should we set hfile.block.cache.size
 higher since indexes are now kept in the block cache? Currently it's
 set to 20%.
 Looking over my own production machines I see that the
 storefileIndexSize is around 600-700MB so that's potentially how much
 more data I'd have to block cache (more likely it's half of that
 that's really being used actively).
 What would be a good new default? 25%? 30%? How do we handle those
 that will be pushed over the BC+memstore size limit because of that
 change?
 {quote}
 I'll bump this to 25% and put in the release note the fact that people should 
 verify their settings before upgrading to make sure memstore+block cache 
 isn't over 80% (meaning they'd haven't change the block cache size but would 
 have bumped the memstores from 40% to 60%).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-4966) Put/Delete values cannot be tested with MRUnit

2011-12-06 Thread Todd Lipcon (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-4966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HBASE-4966:
---

Hadoop Flags: Incompatible change

 Put/Delete values cannot be tested with MRUnit
 --

 Key: HBASE-4966
 URL: https://issues.apache.org/jira/browse/HBASE-4966
 Project: HBase
  Issue Type: Bug
  Components: client, mapreduce
Affects Versions: 0.90.4
Reporter: Nicholas Telford
Priority: Minor

 When using the IdentityTableReducer, which expects input values of either a 
 Put or Delete object, testing with MRUnit the Mapper with MRUnit is not 
 possible because neither Put nor Delete implement equals().
 We should implement equals() on both such that equality means:
 * Both objects are of the same class (in this case, Put or Delete)
 * Both objects are for the same key.
 * Both objects contain an equal set of KeyValues (applicable only to Put)
 KeyValue.equals() appears to already be implemented, but only checks for 
 equality of row key, column family and column qualifier - two KeyValues can 
 be considered equal if they contain different values. This won't work for 
 testing.
 Instead, the Put.equals() and Delete.equals() implementations should do a 
 deep equality check on their KeyValues, like this:
 {code:java}
 myKv.equals(theirKv)  Bytes.equals(myKv.getValue(), theirKv.getValue());
 {code}
 NOTE: This would impact any code that relies on the existing identity 
 implementation of Put.equals() and Delete.equals(), therefore cannot be 
 guaranteed to be backwards-compatible.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-4927) CatalogJanior:SplitParentFirstComparator doesn't sort as expected, for the last region when the endkey is empty

2011-12-06 Thread Todd Lipcon (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-4927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HBASE-4927:
---

Attachment: hbase-4927-fix-ws.txt

there were a few more hard tabs in the tests, and a couple places with 4 space 
indentation instead of 2. here's a fixed up version.

 CatalogJanior:SplitParentFirstComparator doesn't sort as expected, for the 
 last region when the endkey is empty
 ---

 Key: HBASE-4927
 URL: https://issues.apache.org/jira/browse/HBASE-4927
 Project: HBase
  Issue Type: Bug
  Components: master
Affects Versions: 0.92.0, 0.94.0
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor
 Fix For: 0.94.0

 Attachments: 
 0001-HBASE-4927-CatalogJanior-SplitParentFirstComparator-.patch, 
 0001-HBASE-4927-CatalogJanior-SplitParentFirstComparator_v2.patch, 
 hbase-4927-fix-ws.txt


 When reviewing HBASE-4238 backporting, Jon found this issue.
 What happens if the split points are  (empty end key is the last key, empty 
 start key is the first key)
 Parent [A,)
 L daughter [A,B), 
 R daughter [B,)
 When sorted, we gets to end key comparision which results in this incorrector 
 order:
 [A,B), [A,), [B,) 
 we wanted:
 [A,), [A,B), [B,)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-4927) CatalogJanior:SplitParentFirstComparator doesn't sort as expected, for the last region when the endkey is empty

2011-12-06 Thread Todd Lipcon (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-4927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HBASE-4927:
---

Status: Open  (was: Patch Available)

 CatalogJanior:SplitParentFirstComparator doesn't sort as expected, for the 
 last region when the endkey is empty
 ---

 Key: HBASE-4927
 URL: https://issues.apache.org/jira/browse/HBASE-4927
 Project: HBase
  Issue Type: Bug
  Components: master
Affects Versions: 0.92.0, 0.94.0
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor
 Fix For: 0.94.0

 Attachments: 
 0001-HBASE-4927-CatalogJanior-SplitParentFirstComparator-.patch, 
 0001-HBASE-4927-CatalogJanior-SplitParentFirstComparator_v2.patch, 
 hbase-4927-fix-ws.txt


 When reviewing HBASE-4238 backporting, Jon found this issue.
 What happens if the split points are  (empty end key is the last key, empty 
 start key is the first key)
 Parent [A,)
 L daughter [A,B), 
 R daughter [B,)
 When sorted, we gets to end key comparision which results in this incorrector 
 order:
 [A,B), [A,), [B,) 
 we wanted:
 [A,), [A,B), [B,)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-4927) CatalogJanior:SplitParentFirstComparator doesn't sort as expected, for the last region when the endkey is empty

2011-12-06 Thread Todd Lipcon (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-4927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HBASE-4927:
---

Status: Patch Available  (was: Open)

 CatalogJanior:SplitParentFirstComparator doesn't sort as expected, for the 
 last region when the endkey is empty
 ---

 Key: HBASE-4927
 URL: https://issues.apache.org/jira/browse/HBASE-4927
 Project: HBase
  Issue Type: Bug
  Components: master
Affects Versions: 0.92.0, 0.94.0
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor
 Fix For: 0.94.0

 Attachments: 
 0001-HBASE-4927-CatalogJanior-SplitParentFirstComparator-.patch, 
 0001-HBASE-4927-CatalogJanior-SplitParentFirstComparator_v2.patch, 
 hbase-4927-fix-ws.txt


 When reviewing HBASE-4238 backporting, Jon found this issue.
 What happens if the split points are  (empty end key is the last key, empty 
 start key is the first key)
 Parent [A,)
 L daughter [A,B), 
 R daughter [B,)
 When sorted, we gets to end key comparision which results in this incorrector 
 order:
 [A,B), [A,), [B,) 
 we wanted:
 [A,), [A,B), [B,)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-4957) Clean up some log messages, code in RecoverableZooKeeper

2011-12-05 Thread Todd Lipcon (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-4957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HBASE-4957:
---

Attachment: hbase-4957.txt

 Clean up some log messages, code in RecoverableZooKeeper
 

 Key: HBASE-4957
 URL: https://issues.apache.org/jira/browse/HBASE-4957
 Project: HBase
  Issue Type: Improvement
  Components: zookeeper
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Assignee: Todd Lipcon
Priority: Minor
 Fix For: 0.94.0

 Attachments: hbase-4957.txt


 In RecoverableZooKeeper, there are a number of log messages and comments 
 which don't really read correctly, and some other pieces of code that can be 
 cleaned up. Simple cleanup - shouldn't be any actual behavioral changes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-4957) Clean up some log messages, code in RecoverableZooKeeper

2011-12-05 Thread Todd Lipcon (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-4957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HBASE-4957:
---

Attachment: hbase-4957.txt

oops, uploaded only half the patch so it didn't apply. take 2!

 Clean up some log messages, code in RecoverableZooKeeper
 

 Key: HBASE-4957
 URL: https://issues.apache.org/jira/browse/HBASE-4957
 Project: HBase
  Issue Type: Improvement
  Components: zookeeper
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Assignee: Todd Lipcon
Priority: Minor
 Fix For: 0.94.0

 Attachments: hbase-4957.txt, hbase-4957.txt


 In RecoverableZooKeeper, there are a number of log messages and comments 
 which don't really read correctly, and some other pieces of code that can be 
 cleaned up. Simple cleanup - shouldn't be any actual behavioral changes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-4961) Lots of precommit builds hanging for days

2011-12-05 Thread Todd Lipcon (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-4961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HBASE-4961:
---

Attachment: hbase-hung-builds.tar.gz

 Lots of precommit builds hanging for days
 -

 Key: HBASE-4961
 URL: https://issues.apache.org/jira/browse/HBASE-4961
 Project: HBase
  Issue Type: Bug
  Components: build, test
Affects Versions: 0.92.0, 0.94.0
Reporter: Todd Lipcon
 Attachments: hbase-hung-builds.tar.gz


 I was logged into the ASF build machines and saw about 10-15 HBase precommit 
 builds that have been hung for weeks. I took a jstack of each, which I'll 
 attach here. I then kill -9ed them to free up the resources.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-4820) Distributed log splitting coding enhancement to make it easier to understand, no semantics change

2011-11-30 Thread Todd Lipcon (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-4820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HBASE-4820:
---

Hadoop Flags: Reviewed

 Distributed log splitting coding enhancement to make it easier to understand, 
 no semantics change
 -

 Key: HBASE-4820
 URL: https://issues.apache.org/jira/browse/HBASE-4820
 Project: HBase
  Issue Type: Improvement
  Components: wal
Affects Versions: 0.92.0, 0.94.0
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor
  Labels: newbie
 Fix For: 0.94.0

 Attachments: 
 0001-HBASE-4820-Distributed-log-splitting-coding-enhancement-to-makeit-easier-to-understand,-no-semantics-change..patch,
  
 0001-HBASE-4820-Distributed-log-splitting-coding-enhancement-to-makeit-easier-to-understand,-no-semantics-change..patch,
  0001-HBASE-4820-Minor-distributed-log-splitting-enhanceme.patch, 
 0001-HBASE-4820-Minor-distributed-log-splitting-enhanceme.patch, 
 0001-HBASE-4820-Minor-distributed-log-splitting-enhanceme_new.patch


 In reviewing distributed log splitting feature, we found some cosmetic 
 issues.  They make the code hard to understand.
 It will be great to fix them.  For this issue, there should be no semantic 
 change.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-4820) Distributed log splitting coding enhancement to make it easier to understand, no semantics change

2011-11-30 Thread Todd Lipcon (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-4820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HBASE-4820:
---

   Resolution: Fixed
Fix Version/s: (was: 0.92.0)
   Status: Resolved  (was: Patch Available)

Committed to trunk (94) only, since 92 is already in an rc phase. We can 
backport this to 92 easily later if need be.

 Distributed log splitting coding enhancement to make it easier to understand, 
 no semantics change
 -

 Key: HBASE-4820
 URL: https://issues.apache.org/jira/browse/HBASE-4820
 Project: HBase
  Issue Type: Improvement
  Components: wal
Affects Versions: 0.92.0, 0.94.0
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor
  Labels: newbie
 Fix For: 0.94.0

 Attachments: 
 0001-HBASE-4820-Distributed-log-splitting-coding-enhancement-to-makeit-easier-to-understand,-no-semantics-change..patch,
  
 0001-HBASE-4820-Distributed-log-splitting-coding-enhancement-to-makeit-easier-to-understand,-no-semantics-change..patch,
  0001-HBASE-4820-Minor-distributed-log-splitting-enhanceme.patch, 
 0001-HBASE-4820-Minor-distributed-log-splitting-enhanceme.patch, 
 0001-HBASE-4820-Minor-distributed-log-splitting-enhanceme_new.patch


 In reviewing distributed log splitting feature, we found some cosmetic 
 issues.  They make the code hard to understand.
 It will be great to fix them.  For this issue, there should be no semantic 
 change.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-3952) Guava snuck back in as a dependency via hbase-3777

2011-11-22 Thread Todd Lipcon (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-3952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HBASE-3952:
---

Fix Version/s: 0.90.5
   0.92.0

 Guava snuck back in as a dependency via hbase-3777
 --

 Key: HBASE-3952
 URL: https://issues.apache.org/jira/browse/HBASE-3952
 Project: HBase
  Issue Type: Task
Reporter: stack
 Fix For: 0.92.0, 0.90.5

 Attachments: hcm.txt


 Undo it as we did in HBASE-3264.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-4806) Fix logging message in HbaseObjectWritable

2011-11-17 Thread Todd Lipcon (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-4806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HBASE-4806:
---

   Resolution: Fixed
Fix Version/s: 0.90.5
   0.94.0
   0.92.0
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

Committed to 0.90.5, 0.92.0, 0.94.0. Thanks Jon.

 Fix logging message in HbaseObjectWritable
 --

 Key: HBASE-4806
 URL: https://issues.apache.org/jira/browse/HBASE-4806
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0, 0.94.0, 0.90.5
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Trivial
 Fix For: 0.92.0, 0.94.0, 0.90.5

 Attachments: hbase-4806.patch


 This is a trivial fix to HBASE-3316 to fix an error message.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-4643) Consider reverting HBASE-451 (change HRI to remove HTD) in 0.92

2011-10-21 Thread Todd Lipcon (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-4643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HBASE-4643:
---

Attachment: revert.txt

Here's a patch which does the revert. It passed all but three unit tests:

Results :

Failed tests:   testSizes(org.apache.hadoop.hbase.io.TestHeapSize): 
expected:296 but was:288

Tests in error: 
  
testMasterFailoverWithMockedRITOnDeadRS(org.apache.hadoop.hbase.master.TestMasterFailover):
 Server not running, aborting
  testTimestamps(org.apache.hadoop.hbase.TestMultiVersions): 
org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException: Column family 
colfamily1 does not exist in region 
testTimestamps,,1319176310979.39600bf02dc843a3dc6bf8b79567d8c7. in table {NAME 
= 'testTimestamps', FAMILIES = [{NAME = 'colfamily11', BLOOMFILTER = 
'NONE', REPLICATION_SCOPE = '0', COMPRESSION = 'NONE', VERSIONS = '3', TTL 
= '2147483647', MIN_VERSIONS = '0', BLOCKSIZE = '65536', IN_MEMORY = 
'false', BLOCKCACHE = 'true'}]}

It would need more work to be committed, but figured I'd post it just in case. 
It also reverted HBASE-3446 to make it simpler, so that woudl have to be 
re-incorporated.

 Consider reverting HBASE-451 (change HRI to remove HTD) in 0.92
 ---

 Key: HBASE-4643
 URL: https://issues.apache.org/jira/browse/HBASE-4643
 Project: HBase
  Issue Type: Brainstorming
Affects Versions: 0.92.0
Reporter: Todd Lipcon
 Attachments: revert.txt


 I've been chatting with some folks recently about this thought: it seems 
 like, if you enumerate the larger changes in 0.92, this is probably the one 
 that is the most destabilizing that hasn't been through a lot of baking 
 yet. You can see this evidenced by the very high number of followup commits 
 it generated: looks like somewhere around 15 of them, plus some bugs still 
 open.
 I've done a patch to revert this and the related followup changes on the 0.92 
 branch. Do we want to consider doing this?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-3929) Add option to HFile tool to produce basic stats

2011-10-21 Thread Todd Lipcon (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-3929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HBASE-3929:
---

  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Committed to trunk and 92. Thanks Matteo

 Add option to HFile tool to produce basic stats
 ---

 Key: HBASE-3929
 URL: https://issues.apache.org/jira/browse/HBASE-3929
 Project: HBase
  Issue Type: New Feature
  Components: io
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Assignee: Todd Lipcon
 Fix For: 0.94.0

 Attachments: HBASE-3929-v2.patch, HBASE-3929-v3.patch, 
 hbase-3929-draft.patch, hbase-3929-draft.txt


 In looking at HBASE-3421 I wrote a small tool to scan an HFile and produce 
 some basic statistics about it:
 - min/mean/max key size, value size (uncompressed)
 - min/mean/max number of columns per row (uncompressed)
 - min/mean/max number of bytes per row (uncompressed)
 - the key of the largest row

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-3929) Add option to HFile tool to produce basic stats

2011-10-21 Thread Todd Lipcon (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-3929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HBASE-3929:
---

Fix Version/s: (was: 0.94.0)
   0.92.0

 Add option to HFile tool to produce basic stats
 ---

 Key: HBASE-3929
 URL: https://issues.apache.org/jira/browse/HBASE-3929
 Project: HBase
  Issue Type: New Feature
  Components: io
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Assignee: Matteo Bertozzi
 Fix For: 0.92.0

 Attachments: HBASE-3929-v2.patch, HBASE-3929-v3.patch, 
 hbase-3929-draft.patch, hbase-3929-draft.txt


 In looking at HBASE-3421 I wrote a small tool to scan an HFile and produce 
 some basic statistics about it:
 - min/mean/max key size, value size (uncompressed)
 - min/mean/max number of columns per row (uncompressed)
 - min/mean/max number of bytes per row (uncompressed)
 - the key of the largest row

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-3680) Publish more metrics about mslab

2011-10-20 Thread Todd Lipcon (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-3680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HBASE-3680:
---

Fix Version/s: 0.92.0
   Status: Patch Available  (was: Open)

 Publish more metrics about mslab
 

 Key: HBASE-3680
 URL: https://issues.apache.org/jira/browse/HBASE-3680
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.90.1
Reporter: Jean-Daniel Cryans
Assignee: Todd Lipcon
 Fix For: 0.92.0

 Attachments: hbase-3680.txt


 We have been using mslab on all our clusters for a while now and it seems it 
 tends to OOME or send us into GC loops of death a lot more than it used to. 
 For example, one RS with mslab enabled and 7GB of heap died out of OOME this 
 afternoon; it had .55GB in the block cache and 2.03GB in the memstores which 
 doesn't account for much... but it could be that because of mslab a lot of 
 space was lost in those incomplete 2MB blocks and without metrics we can't 
 really tell. Compactions were running at the time of the OOME and I see block 
 cache activity. The average load on that cluster is 531.
 We should at least publish the total size of all those blocks and maybe even 
 take actions based on that (like force flushing).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-3680) Publish more metrics about mslab

2011-10-20 Thread Todd Lipcon (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-3680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HBASE-3680:
---

Attachment: hbase-3680.txt

Here's a patch which adds the metric. I tested by firing up a standalone 
instance and using jconsole to check that it showed the right value in JMX.

 Publish more metrics about mslab
 

 Key: HBASE-3680
 URL: https://issues.apache.org/jira/browse/HBASE-3680
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.90.1
Reporter: Jean-Daniel Cryans
Assignee: Todd Lipcon
 Fix For: 0.92.0

 Attachments: hbase-3680.txt


 We have been using mslab on all our clusters for a while now and it seems it 
 tends to OOME or send us into GC loops of death a lot more than it used to. 
 For example, one RS with mslab enabled and 7GB of heap died out of OOME this 
 afternoon; it had .55GB in the block cache and 2.03GB in the memstores which 
 doesn't account for much... but it could be that because of mslab a lot of 
 space was lost in those incomplete 2MB blocks and without metrics we can't 
 really tell. Compactions were running at the time of the OOME and I see block 
 cache activity. The average load on that cluster is 531.
 We should at least publish the total size of all those blocks and maybe even 
 take actions based on that (like force flushing).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-4570) Scan ACID problem with concurrent puts.

2011-10-14 Thread Todd Lipcon (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-4570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HBASE-4570:
---

Attachment: hbase-4570.txt

 Scan ACID problem with concurrent puts.
 ---

 Key: HBASE-4570
 URL: https://issues.apache.org/jira/browse/HBASE-4570
 Project: HBase
  Issue Type: Bug
  Components: client, regionserver
Affects Versions: 0.90.1, 0.90.3
Reporter: Jonathan Hsieh
 Attachments: 4570-instrumentation.tgz, hbase-4570.tgz, hbase-4570.txt


 When scanning a table sometimes rows that have multiple column families get 
 split into two rows if there are concurrent writes.  In this particular case 
 we are overwriting the contents of a Get directly back onto itself as a Put.
 For example, this is a two cf row (with f1, f2, .. f9 cfs).  It is 
 actually returned as two rows (#55 and #56). Interestingly if the two were 
 merged we would have a single proper row.
 Row row024461 had time stamps: [55: 
 keyvalues={row024461/f0:data/1318200440867/Put/vlen=1000, 
 row024461/f0:qual/1318200440867/Put/vlen=10, 
 row024461/f1:data/1318200440867/Put/vlen=1000, 
 row024461/f1:qual/1318200440867/Put/vlen=10, 
 row024461/f2:data/1318200440867/Put/vlen=1000, 
 row024461/f2:qual/1318200440867/Put/vlen=10, 
 row024461/f3:data/1318200440867/Put/vlen=1000, 
 row024461/f3:qual/1318200440867/Put/vlen=10, 
 row024461/f4:data/1318200440867/Put/vlen=1000, 
 row024461/f4:qual/1318200440867/Put/vlen=10}, 
 56: keyvalues={row024461/f5:data/1318200440867/Put/vlen=1000, 
 row024461/f5:qual/1318200440867/Put/vlen=10, 
 row024461/f6:data/1318200440867/Put/vlen=1000, 
 row024461/f6:qual/1318200440867/Put/vlen=10, 
 row024461/f7:data/1318200440867/Put/vlen=1000, 
 row024461/f7:qual/1318200440867/Put/vlen=10, 
 row024461/f8:data/1318200440867/Put/vlen=1000, 
 row024461/f8:qual/1318200440867/Put/vlen=10, 
 row024461/f9:data/1318200440867/Put/vlen=1000, 
 row024461/f9:qual/1318200440867/Put/vlen=10}]
 I've only tested this on 0.90.1+patches and 0.90.3+patches, but it is 
 consistent and duplicatable.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-4551) Small fixes to compile against 0.23-SNAPSHOT

2011-10-13 Thread Todd Lipcon (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-4551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HBASE-4551:
---

Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed to 92 and trunk

 Small fixes to compile against 0.23-SNAPSHOT
 

 Key: HBASE-4551
 URL: https://issues.apache.org/jira/browse/HBASE-4551
 Project: HBase
  Issue Type: Bug
  Components: build
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Assignee: Todd Lipcon
 Fix For: 0.92.0

 Attachments: hbase-4551.txt, hbase-4551.txt


 - fix pom.xml to properly pull the test artifacts
 - fix TestHLog to not use the private cluster.getNameNode() API

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-4569) Simpler implementation for slab cache

2011-10-10 Thread Todd Lipcon (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-4569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HBASE-4569:
---

Attachment: hbase-4569.txt

here's a draft patch. doesn't pass the heapsize tests, and lacks some metrics 
stuff, but passes the actual cache tests.

Interested to see if people think this is the right direction.

 Simpler implementation for slab cache
 -

 Key: HBASE-4569
 URL: https://issues.apache.org/jira/browse/HBASE-4569
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Affects Versions: 0.92.0
Reporter: Todd Lipcon
 Attachments: hbase-4569.txt


 I spent a little bit of time last night hacking on the slab cache 
 implementation to make it a little simpler. The change is:
 - no longer has the composition of SlabCache containing a SingleSizeCache per 
 slab size. SlabCache holds its own slabs
 - no longer use guava's map implementations to handle a size-bounded cache. 
 Instead, manages its own LRU linked list
 - significantly less clever about synchronization. since this is an L2 cache, 
 it should be less contended than the L1 cache, and I think we can afford to 
 be dumb.
 - should have less memory usage since there's only one map entry per key 
 instead of several.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-4551) Small fixes to compile against 0.23-SNAPSHOT

2011-10-07 Thread Todd Lipcon (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-4551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HBASE-4551:
---

Attachment: hbase-4551.txt

new revision also removes the places where we set the lease timeouts in the NN 
to non-standard values. Now that we use the recoverLease API instead of the 
appendFile API, we don't need to do this. I ran the modified tests and they 
still pass on 0.20.

 Small fixes to compile against 0.23-SNAPSHOT
 

 Key: HBASE-4551
 URL: https://issues.apache.org/jira/browse/HBASE-4551
 Project: HBase
  Issue Type: Bug
  Components: build
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Assignee: Todd Lipcon
 Fix For: 0.92.0

 Attachments: hbase-4551.txt, hbase-4551.txt


 - fix pom.xml to properly pull the test artifacts
 - fix TestHLog to not use the private cluster.getNameNode() API

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-4551) Small fixes to compile against 0.23-SNAPSHOT

2011-10-06 Thread Todd Lipcon (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-4551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HBASE-4551:
---

Attachment: hbase-4551.txt

 Small fixes to compile against 0.23-SNAPSHOT
 

 Key: HBASE-4551
 URL: https://issues.apache.org/jira/browse/HBASE-4551
 Project: HBase
  Issue Type: Bug
  Components: build
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Assignee: Todd Lipcon
 Fix For: 0.92.0

 Attachments: hbase-4551.txt


 - fix pom.xml to properly pull the test artifacts
 - fix TestHLog to not use the private cluster.getNameNode() API

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-4386) NPE in TaskMonitor

2011-10-05 Thread Todd Lipcon (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-4386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HBASE-4386:
---

  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Committed for 92 and trunk

 NPE in TaskMonitor
 --

 Key: HBASE-4386
 URL: https://issues.apache.org/jira/browse/HBASE-4386
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Assignee: Todd Lipcon
Priority: Critical
 Fix For: 0.92.0

 Attachments: hbase-4386.txt


 Saw the following hitting /rs-status
 preINTERNAL_SERVER_ERROR/pre/ph3Caused 
 by:/h3prejava.lang.NullPointerException
 at 
 org.apache.hadoop.hbase.monitoring.TaskMonitor.purgeExpiredTasks(TaskMonitor.java:97)
 at 
 org.apache.hadoop.hbase.monitoring.TaskMonitor.getTasks(TaskMonitor.java:127)
 at 
 org.apache.hbase.tmpl.common.TaskMonitorTmplImpl.renderNoFlush(TaskMonitorTmplImpl.java:50)
 at 
 org.apache.hbase.tmpl.common.TaskMonitorTmpl.renderNoFlush(TaskMonitorTmpl.java:170)
 at 
 org.apache.hbase.tmpl.regionserver.RSStatusTmplImpl.renderNoFlush(RSStatusTmplImpl.java:70)
 at 
 org.apache.hbase.tmpl.regionserver.RSStatusTmpl.renderNoFlush(RSStatusTmpl.java:176)
 at 
 org.apache.hbase.tmpl.regionserver.RSStatusTmpl.render(RSStatusTmpl.java:167)
 at 
 org.apache.hadoop.hbase.regionserver.RSStatusServlet.doGet(RSStatusServlet.java:48)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-4386) NPE in TaskMonitor

2011-10-03 Thread Todd Lipcon (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-4386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HBASE-4386:
---

Attachment: hbase-4386.txt

 NPE in TaskMonitor
 --

 Key: HBASE-4386
 URL: https://issues.apache.org/jira/browse/HBASE-4386
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Assignee: Todd Lipcon
Priority: Critical
 Fix For: 0.92.0

 Attachments: hbase-4386.txt


 Saw the following hitting /rs-status
 preINTERNAL_SERVER_ERROR/pre/ph3Caused 
 by:/h3prejava.lang.NullPointerException
 at 
 org.apache.hadoop.hbase.monitoring.TaskMonitor.purgeExpiredTasks(TaskMonitor.java:97)
 at 
 org.apache.hadoop.hbase.monitoring.TaskMonitor.getTasks(TaskMonitor.java:127)
 at 
 org.apache.hbase.tmpl.common.TaskMonitorTmplImpl.renderNoFlush(TaskMonitorTmplImpl.java:50)
 at 
 org.apache.hbase.tmpl.common.TaskMonitorTmpl.renderNoFlush(TaskMonitorTmpl.java:170)
 at 
 org.apache.hbase.tmpl.regionserver.RSStatusTmplImpl.renderNoFlush(RSStatusTmplImpl.java:70)
 at 
 org.apache.hbase.tmpl.regionserver.RSStatusTmpl.renderNoFlush(RSStatusTmpl.java:176)
 at 
 org.apache.hbase.tmpl.regionserver.RSStatusTmpl.render(RSStatusTmpl.java:167)
 at 
 org.apache.hadoop.hbase.regionserver.RSStatusServlet.doGet(RSStatusServlet.java:48)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-4386) NPE in TaskMonitor

2011-10-03 Thread Todd Lipcon (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-4386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HBASE-4386:
---

Status: Patch Available  (was: Open)

 NPE in TaskMonitor
 --

 Key: HBASE-4386
 URL: https://issues.apache.org/jira/browse/HBASE-4386
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Assignee: Todd Lipcon
Priority: Critical
 Fix For: 0.92.0

 Attachments: hbase-4386.txt


 Saw the following hitting /rs-status
 preINTERNAL_SERVER_ERROR/pre/ph3Caused 
 by:/h3prejava.lang.NullPointerException
 at 
 org.apache.hadoop.hbase.monitoring.TaskMonitor.purgeExpiredTasks(TaskMonitor.java:97)
 at 
 org.apache.hadoop.hbase.monitoring.TaskMonitor.getTasks(TaskMonitor.java:127)
 at 
 org.apache.hbase.tmpl.common.TaskMonitorTmplImpl.renderNoFlush(TaskMonitorTmplImpl.java:50)
 at 
 org.apache.hbase.tmpl.common.TaskMonitorTmpl.renderNoFlush(TaskMonitorTmpl.java:170)
 at 
 org.apache.hbase.tmpl.regionserver.RSStatusTmplImpl.renderNoFlush(RSStatusTmplImpl.java:70)
 at 
 org.apache.hbase.tmpl.regionserver.RSStatusTmpl.renderNoFlush(RSStatusTmpl.java:176)
 at 
 org.apache.hbase.tmpl.regionserver.RSStatusTmpl.render(RSStatusTmpl.java:167)
 at 
 org.apache.hadoop.hbase.regionserver.RSStatusServlet.doGet(RSStatusServlet.java:48)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira