Thank you guys for your replies. TableMapReduceUtil.initCredentials() needs a Job object as argument not a Configuration object returned by UDFContext.getUDFContext().getJobConf.
I didn't dive into the hbase credentials very deeply, so I don't know how to create a credential file and cache it. Only I know is to use some utilities provided by hadoop and hbase to create credentials for Job like TableMapReduceUtil.initCredentials(). But to use the utilities, I need get the Job object when it's being constructed in backend like the setLocation() of LoadFunc. Then the credentials can take effect when job run. I guess I can't do this in EvalFunc. On Wed, Sep 26, 2012 at 11:26 PM, Ashutosh Chauhan <[email protected]>wrote: > I will suggest to use HBase api to manage your credentials, instead > handling them of your own. > > TableMapReduceUtil.initCredentials(UDFContext.getUDFContext().getJobConf); > > Ashutosh > On Tue, Sep 25, 2012 at 10:38 PM, Rohini Palaniswamy < > [email protected]> wrote: > > > That should work too. Just a note that UDFContext is serialized in > JobConf > > and would be visible to all (though base64 encoded) through Jobtracker UI > > unless acls for viewing are set. If you are not that paranoid about > > security as in Yahoo! then it should be fine. > > > > -Rohini > > > > On Tue, Sep 25, 2012 at 5:09 PM, Alan Gates <[email protected]> > wrote: > > > > > You can use the UDFContext to pass information for the UDF in the > JobConf > > > without writing files. > > > > > > Alan. > > > > > > On Sep 25, 2012, at 10:48 AM, Rohini Palaniswamy wrote: > > > > > > > Ray, > > > > Looking at the EvalFunc interface, I can not see a way or loophole > to > > > do > > > > it. EvalFunc does not have a reference to Job or JobConf object to > add > > > > credentials to it. It has getCacheFiles() to add files to > > > DistributedCache, > > > > but no method to add credentials. We should probably add one. The not > > so > > > > nice workaround I can think of is to add the token as a file > > > > to DistributedCache using getCacheFiles() and read the file yourself > in > > > > EvalFunc and use it in communication with HBase. > > > > > > > > Regards, > > > > Rohini > > > > > > > > On Tue, Sep 25, 2012 at 1:51 AM, Ray <[email protected]> wrote: > > > > > > > >> Hi, > > > >> > > > >> I have a requirement to access HBase in UDF. But the HBase is > > > configured to > > > >> be secure, which needs a credential when being connected in a > > mapreduce > > > >> job. > > > >> I see you have added support of secure HBase in HBaseStorage > > > >> > > > > https://issues.apache.org/jira/browse/PIG-2821?attachmentSortBy=dateTime > > > >> But my UDF is an EvalFunc not Loader. > > > >> > > > >> Could you tell me how I can achieve this? Or is there any way I can > > add > > > the > > > >> credential in Job when the job is setup in backend? > > > >> > > > >> Thanks, > > > >> Ray > > > >> > > > > > > > > >
