[
https://issues.apache.org/jira/browse/HBASE-4027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087232#comment-13087232
]
[email protected] commented on HBASE-4027:
------------------------------------------------------
bq. On 2011-08-09 22:05:13, Todd Lipcon wrote:
bq. >
src/main/java/org/apache/hadoop/hbase/io/hfile/slab/SingleSizeCache.java, lines
118-119
bq. > <https://reviews.apache.org/r/1214/diff/8/?file=31771#file31771line118>
bq. >
bq. > I think there's a bug here if you have multiple users hammering the
same contentBlock -- two people can get to "rewind()" at the same time. You
probably need synchronized(contentBlock) around these two lines. See if you can
add a unit test which puts just one block in the cache and starts several
threads which hammer it - I bet you eventually one of the blocks comes back
returned as all 0x0000
bq.
bq. Li Pi wrote:
bq. changed it to returnBlock.put(contentBlock.duplicate()) instead.
bq.
bq. Todd Lipcon wrote:
bq. don't you need to duplicate it before you call rewind? also, did you
add a test that catches this bug, in case there are other similar bugs that I
didn't spot?
Since we duplicate it, we don't need to rewind after using it. The initial copy
is already rewound, so we just duplicate it, do our put, and then discard it.
TestHammerSingleKey has been written.
bq. On 2011-08-09 22:05:13, Todd Lipcon wrote:
bq. >
src/test/java/org/apache/hadoop/hbase/io/hfile/SingleSizeCacheTestUtils.java,
line 45
bq. > <https://reviews.apache.org/r/1214/diff/8/?file=31780#file31780line45>
bq. >
bq. > hrm, this is identical to the other method?
bq.
bq. Li Pi wrote:
bq. Ones for SingleSizeCache, which takes ByteBuffers, the other is for a
BlockCache, which (hypothetically) takes anything with HeapSize.
bq.
bq. Todd Lipcon wrote:
bq. but you should be able to extract a method, even if the method has to
take a type parameter. too much dup code
We have a Cacheable interface now. No more dup code.
- Li
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1214/#review1363
-----------------------------------------------------------
On 2011-08-18 08:31:59, Li Pi wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/1214/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-08-18 08:31:59)
bq.
bq.
bq. Review request for hbase, Todd Lipcon, Ted Yu, Michael Stack, Jonathan
Gray, and Li Pi.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Review request - I apparently can't edit tlipcon's earlier posting of my
diff, so creating a new one.
bq.
bq.
bq. This addresses bug HBase-4027.
bq. https://issues.apache.org/jira/browse/HBase-4027
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. conf/hbase-env.sh 2d55d27
bq. src/main/java/org/apache/hadoop/hbase/io/hfile/BlockCache.java 2d4002c
bq. src/main/java/org/apache/hadoop/hbase/io/hfile/CacheStats.java
PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/io/hfile/Cacheable.java
PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/io/hfile/CachedBlock.java 3b130d8
bq. src/main/java/org/apache/hadoop/hbase/io/hfile/DoubleBlockCache.java
PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java 097dc50
bq. src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
1338453
bq. src/main/java/org/apache/hadoop/hbase/io/hfile/SimpleBlockCache.java
886c31d
bq. src/main/java/org/apache/hadoop/hbase/io/hfile/slab/SingleSizeCache.java
PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/io/hfile/slab/Slab.java
PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/io/hfile/slab/SlabCache.java
PRE-CREATION
bq.
src/main/java/org/apache/hadoop/hbase/io/hfile/slab/SlabItemEvictionWatcher.java
PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
e2c6c93
bq. src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
7b7bf73
bq. src/main/java/org/apache/hadoop/hbase/util/DirectMemoryUtils.java
PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/io/hfile/CacheTestUtils.java
PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/io/hfile/TestCachedBlockQueue.java
1ad2ece
bq. src/test/java/org/apache/hadoop/hbase/io/hfile/TestLruBlockCache.java
f0a9832
bq.
src/test/java/org/apache/hadoop/hbase/io/hfile/slab/TestSingleSizeCache.java
PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/io/hfile/slab/TestSlab.java
PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/io/hfile/slab/TestSlabCache.java
PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/regionserver/TestMemStoreLAB.java
d7e43a0
bq. src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java
4387170
bq.
bq. Diff: https://reviews.apache.org/r/1214/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Ran benchmarks against it in HBase standalone mode. Wrote test cases for
all classes, multithreaded test cases exist for the cache.
bq.
bq.
bq. Thanks,
bq.
bq. Li
bq.
bq.
> Enable direct byte buffers LruBlockCache
> ----------------------------------------
>
> Key: HBASE-4027
> URL: https://issues.apache.org/jira/browse/HBASE-4027
> Project: HBase
> Issue Type: Improvement
> Reporter: Jason Rutherglen
> Assignee: Li Pi
> Priority: Minor
> Attachments: 4027-v5.diff, 4027v7.diff, HBase-4027 (1).pdf,
> HBase-4027.pdf, HBase4027v8.diff, HBase4027v9.diff, hbase-4027-v10.5.diff,
> hbase-4027-v10.diff, hbase-4027v10.6.diff, hbase-4027v6.diff,
> hbase4027v11.5.diff, hbase4027v11.6.diff, hbase4027v11.7.diff,
> hbase4027v11.diff, hbase4027v12.1.diff, hbase4027v12.diff,
> slabcachepatch.diff, slabcachepatchv2.diff, slabcachepatchv3.1.diff,
> slabcachepatchv3.2.diff, slabcachepatchv3.diff, slabcachepatchv4.5.diff,
> slabcachepatchv4.diff
>
>
> Java offers the creation of direct byte buffers which are allocated outside
> of the heap.
> They need to be manually free'd, which can be accomplished using an
> documented {{clean}} method.
> The feature will be optional. After implementing, we can benchmark for
> differences in speed and garbage collection observances.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
