Thank you, Dmitriy, indeed, it was so gnarly that I used an MR job instead :)
Mark On Thu, Mar 8, 2012 at 8:26 PM, Dmitriy Ryaboy <[email protected]> wrote: > Yeah... check out what we do in HBaseStorage to pass the config > around. It's a bit gnarly. > > D > > On Wed, Mar 7, 2012 at 9:49 AM, Mark Kerzner <[email protected]> > wrote: > > Raghu, > > > > it almost works. It just gives me ZooKeeper exception, probably because > Pig > > environment does not know enough about HBase, so that > > > > Configuration hConf = HBaseConfiguration.create(); > > > > does not have all that it needs for the HBase connection. Here is my code > > snippet > > > > Configuration hConf = HBaseConfiguration.create(); > > hConf.set(HBASE_CONFIGURATION_ZOOKEEPER_QUORUM, zookeeperUrl); > > hConf.set(HBASE_CONFIGURATION_ZOOKEEPER_CLIENTPORT, > zookeeperPort); > > HTable hTable = new HTable(hConf, tableName); > > > > Thank you, > > Mark > > > > On Wed, Mar 7, 2012 at 2:27 AM, Raghu Angadi <[email protected]> wrote: > > > >> On Tue, Mar 6, 2012 at 5:02 PM, Mark Kerzner <[email protected] > >> >wrote: > >> > >> > Hi, > >> > > >> > I need to initialize the HBase connection, which I normally do in > >> > configure() in the Mapper, and then my mapper uses it. How do I do it > in > >> > Pig? > >> > > >> > I am ready to define a UDF that will return a handle, but is it a best > >> > practice? > >> > > >> > >> yes. you can initialize inside the first call to UDF.exec(). The same > UDF > >> object is used for the entire mapper. > >> > >> Don't initialize inside the constructor for UDF. AFIK there is no way to > >> tell how many times and when the constructor is called (though it is no > >> more than a handful of times on the front end). > >> > >> Raghu. > >> > >> > Thank you, > >> > Mark > >> > > >> >
