memory mapping. My experience with memory mapped ByteBuffer in Java is
> > that
> > > it is very fast and scalable. By fast, I mean I have clocked reads in
> the
> > > microseconds using nanotime. So I was just wondering why you wouldn't
> at
> > > least make a
One other issue we haven't talked about is JBOD systems - some people
run over a dozen disks per machine. I run 4 disks/node. HDFS does a
valuable service for HBase in balancing IO across multiple JBODs on a
single node. Handling disk issues is something we can completely
ignore in HBase right now
On Mon, Apr 26, 2010 at 3:36 PM, Geoff Hendrey wrote:
> My thought with memory mapping was, as you noted, *not* to try to map files
> that are inside of HDFS but rather to copy as many blocks as possible out of
> HDFS, onto region server filesystems, and memory map the file on the region
> serv
; that
> > it is very fast and scalable. By fast, I mean I have clocked reads in the
> > microseconds using nanotime. So I was just wondering why you wouldn't at
> > least make a 2nd level cache with memory mapping.
> >
> > -geoff
> >
> > -Original
microseconds using nanotime. So I was just wondering why you wouldn't at
>> least make a 2nd level cache with memory mapping.
>>
>> -geoff
>>
>> -Original Message-
>> From: Ryan Rawson [mailto:ryano...@gmail.com]
>> Sent: Monday, April 26, 2010
apping.
>
> -geoff
>
> -Original Message-
> From: Ryan Rawson [mailto:ryano...@gmail.com]
> Sent: Monday, April 26, 2010 1:24 PM
> To: hbase-user@hadoop.apache.org
> Subject: Re: optimizing for random access
>
> HFile uses in memory indexes to only need 1 seek to access data
level cache with memory mapping.
-geoff
-Original Message-
From: Ryan Rawson [mailto:ryano...@gmail.com]
Sent: Monday, April 26, 2010 1:24 PM
To: hbase-user@hadoop.apache.org
Subject: Re: optimizing for random access
HFile uses in memory indexes to only need 1 seek to access data. How
HFile uses in memory indexes to only need 1 seek to access data. How
is this only "decent" ?
As for memory mapped files, given that HDFS files are not local, we
can't mmap() them. However HBase does block caching in memory to
reduce the trips to HDFS.
-ryan
On Mon, Apr 26, 2010 at 11:33 AM,
