Rack assignment isn't a problem. The problem with removing the transient rack
is really an HDFS problem - how to coax it into copying at least 1 replica of
all blocks onto the permanent rack so that the transient rack can be removed
form the cluster without getting corrupt blocks. ... and to do it "quickly" so
that machine instances can be returned to the cloud...
The use model is three fold: (I would venture to guess I'm not the only one
trying to do this sort of thing)
1) to have HBase answer queries in real-time from clients (i.e., analysts
and web servers)
2) write data (i.e., from a data stream, or from the web servers)
3) run processing jobs on that data (i.e., M/R job), that pull some data
out, play with it, and then send some updates (puts) back
I was hoping to accomplish 1 and 2 with "permanent" machine instances, and
accomplish 3 with "transient" machine instances that are only part of the
cluster for the time the processing jobs are running.
I'm learning that 3 should probably be a completely separate "transient"
cluster. It looks like the fastest way to get the data out of the permanent
cluster is to run a job on the transient cluster, using a TableInputFormat.
This way, the transient cluster doesn't need to run an HMaster or HRegion, it
just needs the HBase libraries and would be a client only.
Is this what you were suggesting?
-Matthew
On Aug 25, 2010, at 10:29 AM, Stack wrote:
> On Wed, Aug 25, 2010 at 8:05 AM, Matthew LeMieux <[email protected]> wrote:
>> For those who are curious, using rack awareness to speed up the process of
>> adding and removing nodes did not work in my experiment.
>>
>> Once the extra rack was no longer needed, HDFS was using up time and
>> bandwidth to duplicate the data on the primary rack over to the transient
>> rack rather than replicating the data on transient rack over to the primary
>> rack. With the alternative method of putting all machines in one rack, the
>> data on the permanent machines does not need to be copied over to the
>> transient machines, making the process of replication much faster.
>>
>
> Would adding a custom script for rack awareness help here?
>
>
>> But, once I've copied the files over to the M/R cluster, is there a way to
>> read in the files, i.e., is there an HFileInputFormat equivalent?
>>
>
> No. We don't have such a beastie at the moment. It'd be a little
> tricky to write in that it would in essence need to keep the order and
> merge the content of all files in the same way as its done inside in
> the HRegion class that floats in a running HRegionServer (maybe whats
> needed is a HRegionInputFormat?).
>
> You'll need to bring up an hbase instance on the M/R cluster if you
> want to run M/R against the hbase content. Or, just M/R across racks?
>
> St.Ack
