Sure, I've put this up on Jira here :- https://issues.apache.org/jira/browse/HBASE-3578
On Wed, Feb 23, 2011 at 11:36 PM, Jean-Daniel Cryans <[email protected]>wrote: > Yeah it should, also I'm pretty sure you're right to say that this > regression comes from HBASE-2036... would you mind opening a jira? > > Thanks for the report and the digging Dan! > > J-D > > On Wed, Feb 23, 2011 at 3:30 PM, Dan Harvey <[email protected]> wrote: > > Ah ok, most of the time we were using the default Hadoop configuration > object and not the HBase one. > > > > I guess that's a change between 0.20 and 0.90? Would it not make sense > for the TableMapReduceUtil class to do that for you? As you'll need it in > every HBase map reduce job. > > > > Anyway, I guess we should just change our job configs to include the > HBase configuration too then it would work fine. > > > > Thanks, > > > > On 23 Feb 2011, at 19:44, Jean-Daniel Cryans <[email protected]> > wrote: > > > >> How do you create the configuration object Dan? Are you doing: > >> > >> Configuration conf = HBaseConfiguration.create(); > >> Job job = new Job(conf, "somename"); > >> > >> or are you just creating a normal Configuration? > >> > >> BTW the code I wrote is what I expect people do and what I'm doing > myself. > >> > >> J-D > >> > >> On Wed, Feb 23, 2011 at 7:19 AM, Dan <[email protected]> wrote: > >>> Or the other way would be adding the HBase configs to the Hadoop > config, > >>> which I think maybe what is intended. > >>> > >>> If I do it whilst I'm setting up the job > >>> with HBaseConfiguration.addHbaseResources() it works fine, should > >>> the TableMapReduceUtil.initTableMapperJob do this for you? > >>> > >>> I think this was the change that remove the old HBaseConfiguration from > >>> being used https://issues.apache.org/jira/browse/HBASE-2036, but no > other > >>> way to add the HBase configs for a TableInputFormat seems to have been > >>> added? > >>> > >>> Sorry if I'm completely missing something here! > >>> > >>> On Wed, Feb 23, 2011 at 1:57 PM, Dan <[email protected]> wrote: > >>> > >>>> Hey, > >>>> > >>>> I'm just testing our code to move over to 0.90 and I'm finding some > issues > >>>> with the map/reduce jobs we've written using TableInputFormat. > >>>> > >>>> We setup the jobs using TableMapReduceUtil.initTableMapperJob(..); > which > >>>> worked fine in 0.20.6 but now throws the following errors when I try > to run > >>>> them :- > >>>> > >>>> 11/02/23 12:38:44 ERROR zookeeper.ZKConfig: no clientPort found in > zoo.cfg > >>>> 11/02/23 12:38:44 ERROR mapreduce.TableInputFormat: > >>>> org.apache.hadoop.hbase.ZooKeeperConnectionException: > java.io.IOException: > >>>> Unable to determine ZooKeeper ensemble > >>>> ... > >>>> Exception in thread "main" java.io.IOException: No table was provided. > >>>> at > >>>> > org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:130) > >>>> ... > >>>> > >>>> Which I think basically means it can't find the quorum/port for > zookeeper > >>>> in the hbase configuration object from the hbase-*.xml files. > >>>> Looking into this a bit further it seems the TableInputFormat's > setConf() > >>>> method creates the HTable using the Hadoop configuration object which > >>>> doesn't contain any of the HBase configuration... > >>>> > >>>> So am I setting the job up incorrectly? If not it doesn't seem to make > >>>> sense to me that the HTable should be constructed with the hadoop > >>>> configuration so I guess there might be a bug in the TableInputFormat. > >>>> Modifying it to not take the hadoop configuration object allows the > job to > >>>> work correctly and picks up the hbase configuration I've put on the > class > >>>> path, https://gist.github.com/840403. If that's the case I'll file a > bug > >>>> report and add the patch. > >>>> > >>>> Thanks, > >>>> > >>> > > >
