Hi,
I just want to point out that you can "manually" handle writing
to HBase in maps without even involving TableInputFormat or
TableOutputFormat:
public class MyMap
extends TableMap<ImmutableBytesWritable,MapWritable> // or whatever
{
private HTable table;
public void configure(JobConf job) {
super.configure(job);
try {
HBaseConfiguration conf = new HBaseConfiguration(job);
table = new HTable(conf, "mytable");
} catch (Exception) {
// can't do anything about this now
}
}
public void map(ImmutableBytesWritable key, RowResult value,
OutputCollector<ImmutableBytesWritable,MapWritable> output,
Reporter reporter) throws IOException
{
// now we can report an exception opening the table
if (table == null)
throw new IOException("could not open mytable");
// ...
// commit the result
BatchUpdate update = new BatchUpdate();
// ...
table.commit(update);
}
}
This assumes that you do this when setting up your job:
JobConf conf = new JobConf(new HBaseConfiguration());
Or maybe something like this:
JobConf conf = new JobConf(new Configuration());
conf.set("hbase.master", myMaster);
Hope this helps,
- Andy
> From: William Clay Moody <[EMAIL PROTECTED]>
> Subject: Re: HRegionservers aborting after rolling logs
> To: [email protected]
> Date: Thursday, July 24, 2008, 1:25 PM
>
> I am using TableOutputFormat and HBase is the data sink.
> Our data source is a set of TextInputFile in HDFS. From my
> reading of the discussion, its seems you can only write to
> the HBase from the map if you are using TableInputFormat.
