[jira] [Commented] (HBASE-3484) Replace memstore's ConcurrentSkipListMap with our own implementation

2016-01-03 Thread Lars Hofhansl (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15080698#comment-15080698
 ] 

Lars Hofhansl commented on HBASE-3484:
--

bq. CSLM optimizes for Comparable keys, so if you specify a custom comparator, 
then it has to wrap every key you insert with a wrapper object. Specializing 
CSLM for our purposes would easily save 64 bytes per entry on this.

At least with JDK 8 this is not the case. CSLM works without extra objects with 
or without a custom comparator.
(The Harmony class linked above has that problem, though.)


> Replace memstore's ConcurrentSkipListMap with our own implementation
> 
>
> Key: HBASE-3484
> URL: https://issues.apache.org/jira/browse/HBASE-3484
> Project: HBase
>  Issue Type: Improvement
>  Components: Performance
>Affects Versions: 0.92.0
>Reporter: Todd Lipcon
>Priority: Critical
> Attachments: WIP_HBASE-3484.patch, hierarchical-map.txt, 
> memstore_drag.png
>
>
> By copy-pasting ConcurrentSkipListMap into HBase we can make two improvements 
> to it for our use case in MemStore:
> - add an iterator.replace() method which should allow us to do upsert much 
> more cheaply
> - implement a Set directly without having to do Map to 
> save one reference per entry
> It turns out CSLM is in public domain from its development as part of JSR 
> 166, so we should be OK with licenses.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-3484) Replace memstore's ConcurrentSkipListMap with our own implementation

2014-03-10 Thread haosdent (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925805#comment-13925805
 ] 

haosdent commented on HBASE-3484:
-

Anybody know about Order Maintenance Tree(https://github.com/shuttler/omt), 
which is used in TokuDB? As is know to us, SkipList isn't cpu cache-efficiency 
which OMT is cpu cache-efficiency.

 Replace memstore's ConcurrentSkipListMap with our own implementation
 

 Key: HBASE-3484
 URL: https://issues.apache.org/jira/browse/HBASE-3484
 Project: HBase
  Issue Type: Improvement
  Components: Performance
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Priority: Critical
 Attachments: WIP_HBASE-3484.patch, hierarchical-map.txt, 
 memstore_drag.png


 By copy-pasting ConcurrentSkipListMap into HBase we can make two improvements 
 to it for our use case in MemStore:
 - add an iterator.replace() method which should allow us to do upsert much 
 more cheaply
 - implement a Set directly without having to do MapKeyValue,KeyValue to 
 save one reference per entry
 It turns out CSLM is in public domain from its development as part of JSR 
 166, so we should be OK with licenses.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HBASE-3484) Replace memstore's ConcurrentSkipListMap with our own implementation

2013-12-16 Thread Lars Hofhansl (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13850170#comment-13850170
 ] 

Lars Hofhansl commented on HBASE-3484:
--

From [~mcorgan]...
bq. I've been pondering how to better compact the data in the memstore. 
Sometimes we see a 100MB memstore flush that is really 10MB of KeyValues, which 
gzips to like 2MB, meaning there is a ton of pointer overhead.

This should better now. In various patches I removed:
* caching of the row key (HBASE-7279)
* caching of the timestamp (HBASE-7279)
* caching of the KV length (HBASE-9956)

That saves 12 bytes + sizeOf(rowKey) for each KeyValue in the memstore.
The KV in memory overhead now is: 56 bytes. (the memstoreTS is also stored in 
the HFiles).


 Replace memstore's ConcurrentSkipListMap with our own implementation
 

 Key: HBASE-3484
 URL: https://issues.apache.org/jira/browse/HBASE-3484
 Project: HBase
  Issue Type: Improvement
  Components: Performance
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Priority: Critical
 Attachments: WIP_HBASE-3484.patch, hierarchical-map.txt, 
 memstore_drag.png


 By copy-pasting ConcurrentSkipListMap into HBase we can make two improvements 
 to it for our use case in MemStore:
 - add an iterator.replace() method which should allow us to do upsert much 
 more cheaply
 - implement a Set directly without having to do MapKeyValue,KeyValue to 
 save one reference per entry
 It turns out CSLM is in public domain from its development as part of JSR 
 166, so we should be OK with licenses.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (HBASE-3484) Replace memstore's ConcurrentSkipListMap with our own implementation

2013-09-06 Thread Anoop Sam John (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760178#comment-13760178
 ] 

Anoop Sam John commented on HBASE-3484:
---

In current patch the max size of the memstore slice is specified in terms of 
heap size. But what matters is the #entries in the CSLM. I am thinking this can 
be #entries in the slice.

 Replace memstore's ConcurrentSkipListMap with our own implementation
 

 Key: HBASE-3484
 URL: https://issues.apache.org/jira/browse/HBASE-3484
 Project: HBase
  Issue Type: Improvement
  Components: Performance
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Priority: Critical
 Attachments: hierarchical-map.txt, memstore_drag.png, 
 WIP_HBASE-3484.patch


 By copy-pasting ConcurrentSkipListMap into HBase we can make two improvements 
 to it for our use case in MemStore:
 - add an iterator.replace() method which should allow us to do upsert much 
 more cheaply
 - implement a Set directly without having to do MapKeyValue,KeyValue to 
 save one reference per entry
 It turns out CSLM is in public domain from its development as part of JSR 
 166, so we should be OK with licenses.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-3484) Replace memstore's ConcurrentSkipListMap with our own implementation

2013-09-03 Thread Anoop Sam John (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13756557#comment-13756557
 ] 

Anoop Sam John commented on HBASE-3484:
---

Trying out some thing like how there can be multiple HFiles within a store. 
Within a memstore there can be more than one KeyValueSkipListSet object at a 
time (and so CSLM)
For each of the KeyValueSkipListSet slice there is a configurable max size . 
Initially there will be only one KeyValueSkipListSet in the Memstore. Once the 
size reaches the threshold, we will create another KeyValueSkipListSet (So a 
new CSLM) and new KVs are inserted into this.  The old datastructure wont get 
KVs again. So within *one KeyValueSkipListSet*  KVs will be sorted. This 
continues and finally all these KeyValueSkipListSets are taken in to Snapshots 
and written to HFile.  We will need changes in the MemstoreScanner so as to 
consider this as a heap and emit KVs in the correct order.
Once the flush is over again there will be only one KeyValueSkipListSet in a 
memstore and this continues.   Basically trying to avoid a single CSLM to grow 
to very big size with more #entries.

By default there is no max size for a slice so single CSLM becoming bigger as 
long as KVs are inserted into memstore before a flush.

Done a POC and tested also. The initial test with LoadTestTool shows that we 
can avoid the decrease in throughput with size of the memstore.  Will attach a 
patch with this change by this weekend.

 Replace memstore's ConcurrentSkipListMap with our own implementation
 

 Key: HBASE-3484
 URL: https://issues.apache.org/jira/browse/HBASE-3484
 Project: HBase
  Issue Type: Improvement
  Components: Performance
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Priority: Critical
 Attachments: hierarchical-map.txt, memstore_drag.png


 By copy-pasting ConcurrentSkipListMap into HBase we can make two improvements 
 to it for our use case in MemStore:
 - add an iterator.replace() method which should allow us to do upsert much 
 more cheaply
 - implement a Set directly without having to do MapKeyValue,KeyValue to 
 save one reference per entry
 It turns out CSLM is in public domain from its development as part of JSR 
 166, so we should be OK with licenses.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-3484) Replace memstore's ConcurrentSkipListMap with our own implementation

2013-09-03 Thread Anoop Sam John (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13756558#comment-13756558
 ] 

Anoop Sam John commented on HBASE-3484:
---

Also seeing optimization possibilities as such to CSLM so as to have our own 
CSLM. Todd already mentioned some points above. Will be working on that as well 
as some other things. 

 Replace memstore's ConcurrentSkipListMap with our own implementation
 

 Key: HBASE-3484
 URL: https://issues.apache.org/jira/browse/HBASE-3484
 Project: HBase
  Issue Type: Improvement
  Components: Performance
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Priority: Critical
 Attachments: hierarchical-map.txt, memstore_drag.png


 By copy-pasting ConcurrentSkipListMap into HBase we can make two improvements 
 to it for our use case in MemStore:
 - add an iterator.replace() method which should allow us to do upsert much 
 more cheaply
 - implement a Set directly without having to do MapKeyValue,KeyValue to 
 save one reference per entry
 It turns out CSLM is in public domain from its development as part of JSR 
 166, so we should be OK with licenses.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-3484) Replace memstore's ConcurrentSkipListMap with our own implementation

2012-07-10 Thread Jean-Daniel Cryans (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13410818#comment-13410818
 ] 

Jean-Daniel Cryans commented on HBASE-3484:
---

Something that's not log(n), so a straight line would be ideal :)

 Replace memstore's ConcurrentSkipListMap with our own implementation
 

 Key: HBASE-3484
 URL: https://issues.apache.org/jira/browse/HBASE-3484
 Project: HBase
  Issue Type: Improvement
  Components: performance
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Priority: Critical
 Attachments: hierarchical-map.txt, memstore_drag.png


 By copy-pasting ConcurrentSkipListMap into HBase we can make two improvements 
 to it for our use case in MemStore:
 - add an iterator.replace() method which should allow us to do upsert much 
 more cheaply
 - implement a Set directly without having to do MapKeyValue,KeyValue to 
 save one reference per entry
 It turns out CSLM is in public domain from its development as part of JSR 
 166, so we should be OK with licenses.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-3484) Replace memstore's ConcurrentSkipListMap with our own implementation

2012-07-10 Thread Matt Corgan (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13410934#comment-13410934
 ] 

Matt Corgan commented on HBASE-3484:


I've been pondering how to better compact the data in the memstore.  Sometimes 
we see a 100MB memstore flush that is really 10MB of KeyValues, which gzips to 
like 2MB, meaning there is a ton of pointer overhead.

One thing that came to mind was splitting each memstore into regions of 
consecutive cell ranges and fronting these regions with an index of some sort.  
Instead of SetKeyValue the memstore is SetSetKeyValue.  When an internal 
region crosses a certain size we split it in half.  With a good index structure 
in front of the memstore blocks, it might get closer to a linear 
performance/size curve.  It's comparable with hbase splitting a table into 
regions.

Then, to address the pointer overhead problem, you could use DataBlockEncoding 
to encode each memstore region individually.  A memstore region could 
accumulate several blocks that get compacted periodically.  Given a region size 
of ~64-256KB, the compaction could be very aggressive and could even be done by 
the thread writing the data.  Again, very similar to how hbase manages the 
internals of a single region.

This adds moving pieces and complexity but could be developed as a pluggable 
module that passes the same unit tests as the current memstore.

 Replace memstore's ConcurrentSkipListMap with our own implementation
 

 Key: HBASE-3484
 URL: https://issues.apache.org/jira/browse/HBASE-3484
 Project: HBase
  Issue Type: Improvement
  Components: performance
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Priority: Critical
 Attachments: hierarchical-map.txt, memstore_drag.png


 By copy-pasting ConcurrentSkipListMap into HBase we can make two improvements 
 to it for our use case in MemStore:
 - add an iterator.replace() method which should allow us to do upsert much 
 more cheaply
 - implement a Set directly without having to do MapKeyValue,KeyValue to 
 save one reference per entry
 It turns out CSLM is in public domain from its development as part of JSR 
 166, so we should be OK with licenses.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-3484) Replace memstore's ConcurrentSkipListMap with our own implementation

2012-07-10 Thread Zhihong Ted Yu (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13410944#comment-13410944
 ] 

Zhihong Ted Yu commented on HBASE-3484:
---

+1 on the above suggestion.
We can trade some complexity for better compression rate.

 Replace memstore's ConcurrentSkipListMap with our own implementation
 

 Key: HBASE-3484
 URL: https://issues.apache.org/jira/browse/HBASE-3484
 Project: HBase
  Issue Type: Improvement
  Components: performance
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Priority: Critical
 Attachments: hierarchical-map.txt, memstore_drag.png


 By copy-pasting ConcurrentSkipListMap into HBase we can make two improvements 
 to it for our use case in MemStore:
 - add an iterator.replace() method which should allow us to do upsert much 
 more cheaply
 - implement a Set directly without having to do MapKeyValue,KeyValue to 
 save one reference per entry
 It turns out CSLM is in public domain from its development as part of JSR 
 166, so we should be OK with licenses.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-3484) Replace memstore's ConcurrentSkipListMap with our own implementation

2012-07-09 Thread Otis Gospodnetic (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13409901#comment-13409901
 ] 

Otis Gospodnetic commented on HBASE-3484:
-

@JD - what would/should the ideal graph look like, roughly?


 Replace memstore's ConcurrentSkipListMap with our own implementation
 

 Key: HBASE-3484
 URL: https://issues.apache.org/jira/browse/HBASE-3484
 Project: HBase
  Issue Type: Improvement
  Components: performance
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Priority: Critical
 Attachments: hierarchical-map.txt, memstore_drag.png


 By copy-pasting ConcurrentSkipListMap into HBase we can make two improvements 
 to it for our use case in MemStore:
 - add an iterator.replace() method which should allow us to do upsert much 
 more cheaply
 - implement a Set directly without having to do MapKeyValue,KeyValue to 
 save one reference per entry
 It turns out CSLM is in public domain from its development as part of JSR 
 166, so we should be OK with licenses.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-3484) Replace memstore's ConcurrentSkipListMap with our own implementation

2012-02-23 Thread stack (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13215446#comment-13215446
 ] 

stack commented on HBASE-3484:
--

Great stuff Todd.

bq. ...copy-on-write sorted array lists.

Could we do this?  We'd allocate a new array everytime we did an insert?  An 
array would be cheaper space wise and more efficient scanning, etc., I'd 
think It'd just be the insert and sort that'd be 'expensive'.

Let me have a go at your suggested microbenchmark.

 Replace memstore's ConcurrentSkipListMap with our own implementation
 

 Key: HBASE-3484
 URL: https://issues.apache.org/jira/browse/HBASE-3484
 Project: HBase
  Issue Type: Improvement
  Components: performance
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Priority: Critical
 Attachments: hierarchical-map.txt


 By copy-pasting ConcurrentSkipListMap into HBase we can make two improvements 
 to it for our use case in MemStore:
 - add an iterator.replace() method which should allow us to do upsert much 
 more cheaply
 - implement a Set directly without having to do MapKeyValue,KeyValue to 
 save one reference per entry
 It turns out CSLM is in public domain from its development as part of JSR 
 166, so we should be OK with licenses.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-3484) Replace memstore's ConcurrentSkipListMap with our own implementation

2012-02-22 Thread stack (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13214132#comment-13214132
 ] 

stack commented on HBASE-3484:
--

bq. It probably has negative memory effects in its current incarnation.

How you think Todd?  Because of the tiering cost more or is it something to do 
w/ mslab allocations?

What would you like to see test-wise proving this direction better than what we 
currently have? I could work up some tests?

 Replace memstore's ConcurrentSkipListMap with our own implementation
 

 Key: HBASE-3484
 URL: https://issues.apache.org/jira/browse/HBASE-3484
 Project: HBase
  Issue Type: Improvement
  Components: performance
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Priority: Critical
 Attachments: hierarchical-map.txt


 By copy-pasting ConcurrentSkipListMap into HBase we can make two improvements 
 to it for our use case in MemStore:
 - add an iterator.replace() method which should allow us to do upsert much 
 more cheaply
 - implement a Set directly without having to do MapKeyValue,KeyValue to 
 save one reference per entry
 It turns out CSLM is in public domain from its development as part of JSR 
 166, so we should be OK with licenses.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-3484) Replace memstore's ConcurrentSkipListMap with our own implementation

2012-02-22 Thread Todd Lipcon (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13214179#comment-13214179
 ] 

Todd Lipcon commented on HBASE-3484:


bq. How you think Todd? Because of the tiering cost more or is it something to 
do w/ mslab allocations?

Extra container costs with the fact that we have extra CSLM objects for each 
row. I haven't measured but I bet there is some map-wide overhead that we're 
paying.

There are some other things I noticed that could be improved, though. In 
particular, CSLM optimizes for Comparable keys, so if you specify a custom 
comparator, then it has to wrap every key you insert with a wrapper object. 
Specializing CSLM for our purposes would easily save 64 bytes per entry on this.

Another thought I had was to do the following:
- have the actual entries in rowMap be Object-typed, rather than CSLMs.
- when the first insert happens, just insert the KeyValue itself (optimization 
for the case where each row has only one cell)
- when more inserts happen, swap it out for a proper container type

The proper container type's also interesting to consider here. We never have 
contention on update within a row, since the updates happen under a row lock, 
right? So, we can consider any map type that supports single-writer 
multiple-reader efficiently, which is a wider range of data structures than 
support multi-writer multi-reader. One possibility is snap trees or even 
copy-on-write sorted array lists.

bq. What would you like to see test-wise proving this direction better than 
what we currently have? I could work up some tests?
Would be great if we had a benchmark focused on memstore-only which allowed a 
mix of the following operations from different threads:
- full scans
- range scans
- updates to existing rows which just touch 1 or a few columns
- updates to existing rows which touch lots of columns
- inserts of new rows (few or lots of columns)

But it's a bit of work to do all that. So, a microbenchmark which just timed 
something like having 20 threads each do a bunch of inserts with multi-column 
rows would at least show whether there's promise here.

 Replace memstore's ConcurrentSkipListMap with our own implementation
 

 Key: HBASE-3484
 URL: https://issues.apache.org/jira/browse/HBASE-3484
 Project: HBase
  Issue Type: Improvement
  Components: performance
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Priority: Critical
 Attachments: hierarchical-map.txt


 By copy-pasting ConcurrentSkipListMap into HBase we can make two improvements 
 to it for our use case in MemStore:
 - add an iterator.replace() method which should allow us to do upsert much 
 more cheaply
 - implement a Set directly without having to do MapKeyValue,KeyValue to 
 save one reference per entry
 It turns out CSLM is in public domain from its development as part of JSR 
 166, so we should be OK with licenses.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-3484) Replace memstore's ConcurrentSkipListMap with our own implementation

2011-06-10 Thread Joe Pallas (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13047342#comment-13047342
 ] 

Joe Pallas commented on HBASE-3484:
---

I think the performance issue I mentioned above may actually be HBASE-3855.

 Replace memstore's ConcurrentSkipListMap with our own implementation
 

 Key: HBASE-3484
 URL: https://issues.apache.org/jira/browse/HBASE-3484
 Project: HBase
  Issue Type: Improvement
  Components: performance
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Assignee: Todd Lipcon
Priority: Critical
 Fix For: 0.92.0


 By copy-pasting ConcurrentSkipListMap into HBase we can make two improvements 
 to it for our use case in MemStore:
 - add an iterator.replace() method which should allow us to do upsert much 
 more cheaply
 - implement a Set directly without having to do MapKeyValue,KeyValue to 
 save one reference per entry
 It turns out CSLM is in public domain from its development as part of JSR 
 166, so we should be OK with licenses.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-3484) Replace memstore's ConcurrentSkipListMap with our own implementation

2011-04-26 Thread Joe Pallas (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13025463#comment-13025463
 ] 

Joe Pallas commented on HBASE-3484:
---

This issue was cited by jdcryans as related to unfortunate performance seen in 
the following case:

A test program fills a single row of a family with tens of thousands of 
sequentially increasing qualifiers.  Then it performs random gets (or exists) 
of those qualifiers.  The response time seen is (on average) proportional to 
the ordinal position of the qualifier.  If the table is flushed before the 
random tests begin, then the average response time is basically constant, 
independent of the qualifier's ordinal position.

I'm not sure that either of the two points in the description actually covers 
this case, but I don't know enough to say.


 Replace memstore's ConcurrentSkipListMap with our own implementation
 

 Key: HBASE-3484
 URL: https://issues.apache.org/jira/browse/HBASE-3484
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Assignee: Todd Lipcon
 Fix For: 0.92.0


 By copy-pasting ConcurrentSkipListMap into HBase we can make two improvements 
 to it for our use case in MemStore:
 - add an iterator.replace() method which should allow us to do upsert much 
 more cheaply
 - implement a Set directly without having to do MapKeyValue,KeyValue to 
 save one reference per entry
 It turns out CSLM is in public domain from its development as part of JSR 
 166, so we should be OK with licenses.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (HBASE-3484) Replace memstore's ConcurrentSkipListMap with our own implementation

2011-01-27 Thread Todd Lipcon (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12987702#action_12987702
 ] 

Todd Lipcon commented on HBASE-3484:


Here's the link to the class in Apache Harmony:
https://svn.apache.org/repos/asf/harmony/enhanced/java/branches/java6/classlib/modules/concurrent/src/main/java/java/util/concurrent/ConcurrentSkipListMap.java

 Replace memstore's ConcurrentSkipListMap with our own implementation
 

 Key: HBASE-3484
 URL: https://issues.apache.org/jira/browse/HBASE-3484
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Assignee: Todd Lipcon
 Fix For: 0.92.0


 By copy-pasting ConcurrentSkipListMap into HBase we can make two improvements 
 to it for our use case in MemStore:
 - add an iterator.replace() method which should allow us to do upsert much 
 more cheaply
 - implement a Set directly without having to do MapKeyValue,KeyValue to 
 save one reference per entry
 It turns out CSLM is in public domain from its development as part of JSR 
 166, so we should be OK with licenses.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.