Changing Zookeeper address programmatically for reduces

[Stuart Scott]

Hi,

 

I have a map/reduce job that reads from a Hbase Table and writes to
another.

Does anyone know how to programmatically set the Zoo Keeper address for
a Reducer?

I can create a job as below, set the IP address using .set... it works
fine for the Maper. The reducer defaults to localhost. We have a couple
of clusters and wanted to be able to connect to them at different times.

 

        HBaseConfiguration conf = new HBaseConfiguration();

        conf.set("hbase.master", "44.128.161.86:60000");

        conf.set("hbase.zookeeper.quorum","44.128.161.86");

 

...

 

        Job job = new Job(conf, "MyJob_Job");

        job.setJarByClass(MyClass.class);

        Scan scan = new Scan();

        String columns = "A:B";


        scan.addColumns(columns);

        scan.setStartRow(Bytes.toBytes(GetPaddedLongKey(1)));

        scan.setStopRow(Bytes.toBytes(GetPaddedLongKey(100+1)));

        TableMapReduceUtil.initTableMapperJob("MPI", scan,
Mapper2.class, ImmutableBytesWritable.class, IntWritable.class, job);

 

        TableMapReduceUtil.initTableReducerJob("MyClass",
Reducer1.class, job);

 

        job.waitForCompletion(true);

 

Runs the mapper without problem, then fails..

 

11/03/30 16:19:50 INFO mapred.LocalJobRunner: 

11/03/30 16:19:50 INFO zookeeper.ZooKeeperWrapper: Reconnecting to
zookeeper

11/03/30 16:19:50 INFO zookeeper.ZooKeeper: Initiating client
connection, connectString=localhost:21810 sessionTimeout=60000
watcher=org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper@ae97c4

11/03/30 16:19:50 INFO zookeeper.ClientCnxn: Opening socket connection
to server localhost/127.0.0.1:21810

 

Closing socket connection and attempting reconnect

java.net.ConnectException: Connection refused

        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)

        at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)

        at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1078)

2011-03-30 16:30:14,637 INFO org.apache.zookeeper.ClientCnxn: Opening
socket connection to server localhost/127.0.0.1:21810

2011-03-30 16:30:14,638 WARN org.apache.zookeeper.ClientCnxn: Session
0x0 for server null, unexpected error, closing socket connection and
attempting reconnect

java.net.ConnectException: Connection refused

        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)

 

 

 

 

Any help would be really appreciated.

 

Stuart