Ah ok, most of the time we were using the default Hadoop configuration object and not the HBase one.
I guess that's a change between 0.20 and 0.90? Would it not make sense for the TableMapReduceUtil class to do that for you? As you'll need it in every HBase map reduce job. Anyway, I guess we should just change our job configs to include the HBase configuration too then it would work fine. Thanks, On 23 Feb 2011, at 19:44, Jean-Daniel Cryans <[email protected]> wrote: > How do you create the configuration object Dan? Are you doing: > > Configuration conf = HBaseConfiguration.create(); > Job job = new Job(conf, "somename"); > > or are you just creating a normal Configuration? > > BTW the code I wrote is what I expect people do and what I'm doing myself. > > J-D > > On Wed, Feb 23, 2011 at 7:19 AM, Dan <[email protected]> wrote: >> Or the other way would be adding the HBase configs to the Hadoop config, >> which I think maybe what is intended. >> >> If I do it whilst I'm setting up the job >> with HBaseConfiguration.addHbaseResources() it works fine, should >> the TableMapReduceUtil.initTableMapperJob do this for you? >> >> I think this was the change that remove the old HBaseConfiguration from >> being used https://issues.apache.org/jira/browse/HBASE-2036, but no other >> way to add the HBase configs for a TableInputFormat seems to have been >> added? >> >> Sorry if I'm completely missing something here! >> >> On Wed, Feb 23, 2011 at 1:57 PM, Dan <[email protected]> wrote: >> >>> Hey, >>> >>> I'm just testing our code to move over to 0.90 and I'm finding some issues >>> with the map/reduce jobs we've written using TableInputFormat. >>> >>> We setup the jobs using TableMapReduceUtil.initTableMapperJob(..); which >>> worked fine in 0.20.6 but now throws the following errors when I try to run >>> them :- >>> >>> 11/02/23 12:38:44 ERROR zookeeper.ZKConfig: no clientPort found in zoo.cfg >>> 11/02/23 12:38:44 ERROR mapreduce.TableInputFormat: >>> org.apache.hadoop.hbase.ZooKeeperConnectionException: java.io.IOException: >>> Unable to determine ZooKeeper ensemble >>> ... >>> Exception in thread "main" java.io.IOException: No table was provided. >>> at >>> org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:130) >>> ... >>> >>> Which I think basically means it can't find the quorum/port for zookeeper >>> in the hbase configuration object from the hbase-*.xml files. >>> Looking into this a bit further it seems the TableInputFormat's setConf() >>> method creates the HTable using the Hadoop configuration object which >>> doesn't contain any of the HBase configuration... >>> >>> So am I setting the job up incorrectly? If not it doesn't seem to make >>> sense to me that the HTable should be constructed with the hadoop >>> configuration so I guess there might be a bug in the TableInputFormat. >>> Modifying it to not take the hadoop configuration object allows the job to >>> work correctly and picks up the hbase configuration I've put on the class >>> path, https://gist.github.com/840403. If that's the case I'll file a bug >>> report and add the patch. >>> >>> Thanks, >>> >>
