RE: HBase Push Video or Image File

2011-06-06 Thread Doug Meil
Hi there- I'd recommend reading this... http://hbase.apache.org/book.html#datamodel ... and focusing on what is available in the HBase client API before worrying about hbase-util, which are wrappers on the same HBase client API. -Original Message- From: sgtpepper [mailto:rajat@bt.

RE: HBase Push Video or Image File

2011-06-06 Thread sgtpepper
Hey Doug, Thanks a lot for the reply. But is there any sample/example code available that does the same? I found this code in code.google.com, but it looks more like it is putting the avi file in HDFS instead of HBase. http://code.google.com/p/hbase-jdo/ But I cant seem to be able to understand th

Re: running unit tests

2011-06-06 Thread Stack
On TRUNK, usually all tests pass but intermittently some are broken. The best gauge for the current state of trunk can be found here, https://builds.apache.org//job/HBase-TRUNK/, on the Apache CI server. As J-D says, there is nothing special that you need to do to run the tests other than 'mvn cle

Re: Pluggable block index

2011-06-06 Thread Jason Rutherglen
> But, assuming that the block size is set appropriately, copying one key per > 100 or so values into the block index does not really bloat the hfile Right, this is what Lucene does and should work fine for HBase. Eg, the FST will enable a denser index of keys at the same storage costs.

Re: Pluggable block index

2011-06-06 Thread Jason Rutherglen
> Are you sure that is a good idea? Surely the disk seeks would destroy > you on index load? I'm not sure what you mean there. We can create a total HFile block key index that has pointers to within blocks, eg, they'll store a block index int, and a position (within the block) int. Where would

Re: Pluggable block index

2011-06-06 Thread Ryan Rawson
When I thought about it, I didn't think cross-block compression would be a good idea - this is because you want to be able to decompress each block independently of each other. Perhaps a master HFile dictionary or something. -ryan On Mon, Jun 6, 2011 at 12:06 AM, M. C. Srivas wrote: > On Sun, J

Re: Pluggable block index

2011-06-06 Thread M. C. Srivas
On Sun, Jun 5, 2011 at 11:37 PM, Ryan Rawson wrote: > On Sun, Jun 5, 2011 at 11:33 PM, Jason Rutherglen > wrote: > > Ok, the block index is only storing the first key of each block? > > Hmm... I think we can store a pointer to an exact position in the > > block, or at least allow that (for the F