Build failed in Jenkins: HBase-TRUNK #2014

2011-07-08 Thread Apache Jenkins Server
See https://builds.apache.org/job/HBase-TRUNK/2014/changes Changes: [tedyu] HBASE-4054 extending HTable because HTableInterface does not include getRegionInfos and setAutoFlush -- [...truncated 1095 lines...] A bin/replication A

Re: Converting byte[] to ByteBuffer

2011-07-08 Thread Jason Rutherglen
Also, it's for a good cause, moving the blocks out of main heap using direct byte buffers or some other more native-like facility (if DBB's don't work). On Fri, Jul 8, 2011 at 5:34 PM, Ryan Rawson ryano...@gmail.com wrote: Where? Everywhere? An array is 24 bytes, bb is 56 bytes. Also the API

Re: Converting byte[] to ByteBuffer

2011-07-08 Thread Ryan Rawson
The overhead in a byte buffer is the extra integers to keep track of the mark, position, limit. I am not sure that putting the block cache in to heap is the way to go. Getting faster local dfs reads is important, and if you run hbase on top of Mapr, these things are taken care of for you. On Jul

Re: Converting byte[] to ByteBuffer

2011-07-08 Thread Li Pi
if you do that, you'll have to do a bit of reference counting. i'm working on a slab allocated solution. On Fri, Jul 8, 2011 at 6:20 PM, Jason Rutherglen jason.rutherg...@gmail.com wrote: Also, it's for a good cause, moving the blocks out of main heap using direct byte buffers or some other

Re: Converting byte[] to ByteBuffer

2011-07-08 Thread Jason Rutherglen
Reference counting is doable. Can you describe what the advantages are of the slab allocated solution? On Fri, Jul 8, 2011 at 6:30 PM, Li Pi l...@cloudera.com wrote: if you do that, you'll have to do a bit of reference counting. i'm working on a slab allocated solution. On Fri, Jul 8, 2011

Re: Converting byte[] to ByteBuffer

2011-07-08 Thread Ryan Rawson
Hey, When running on top of Mapr, hbase has fast cached access to locally stored files, the Mapr client ensures that. Likewise, hdfs should also ensure that local reads are fast and come out of cache as necessary. Eg: the kernel block cache. I wouldn't support mmap, it would require 2 different

Re: Converting byte[] to ByteBuffer

2011-07-08 Thread Ryan Rawson
On Jul 8, 2011 7:19 PM, Jason Rutherglen jason.rutherg...@gmail.com wrote: When running on top of Mapr, hbase has fast cached access to locally stored files, the Mapr client ensures that. Likewise, hdfs should also ensure that local reads are fast and come out of cache as necessary. Eg: the

Re: Converting byte[] to ByteBuffer

2011-07-08 Thread Jason Rutherglen
Especially when a perf solution is already here. Use Mapr or hdfs-347/local reads. Right. It goes back to avoiding GC and performing memory deallocation manually (like C). I think this makes sense given the number of issues people have with HBase and GC (more so than Lucene for example).

Re: Converting byte[] to ByteBuffer

2011-07-08 Thread Li Pi
I have a slab allocated cache coded up, testing in YCSB right now :). On Fri, Jul 8, 2011 at 7:52 PM, Jason Rutherglen jason.rutherg...@gmail.com wrote: Especially when a perf solution is already here. Use Mapr or hdfs-347/local reads. Right. It goes back to avoiding GC and performing

Build failed in Jenkins: HBase-TRUNK #2015

2011-07-08 Thread Apache Jenkins Server
See https://builds.apache.org/job/HBase-TRUNK/2015/changes Changes: [tedyu] HBASE-4079 HTableUtil - helper class for loading data (Doug Meil via Ted Yu) -- [...truncated 1283 lines...] [INFO] --- maven-surefire-plugin:2.4.3:test (default-test) @ hbase

Build failed in Jenkins: HBase-TRUNK #2016

2011-07-08 Thread Apache Jenkins Server
See https://builds.apache.org/job/HBase-TRUNK/2016/changes Changes: [stack] HBASE-4017 BlockCache interface should be truly modular -- [...truncated 1077 lines...] A bin/rename_table.rb AUbin/hbase A bin/copy_table.rb A