Ah. Thanks Scott, I think I should've looked at PigStorage much more closely earlier. It does seem to cover everything.
Stored using the signature and UDFContext, and retrieved using the same where I wanted to. Thank you :) On Mon, Jun 28, 2010 at 8:24 PM, Scott Carey <[email protected]> wrote: > UdfContext is what you need to push data to the back end. It would be nice > if something internal to pig like the schema could be made available without > custom sterilization though. > > ----- Reply message ----- > From: "Harsh J" <[email protected]> > Date: Mon, Jun 28, 2010 3:08 am > Subject: Schema access while writing > To: "[email protected]" <[email protected]> > > Hello, > > I'm implementing a custom Store UDF using StoreFuncInterface. > > I need access to the ResourceSchema object each time I do a putNext > operation, but am unable to do this since checkSchema() [which carries > what I require] is only called once and that's during job init or so. > If I try to store a reference/copy of that object, it does not work > since the mapper-side instances of my UDF don't get the checkSchema() > call. > > What I've tried is to process and store required parts of the > ResourceSchema into the Job's Configuration using the fact that > setStoreLocation(String, Job) is called in the init AFTER > checkSchema(), and tried retrieving that to no avail. It looks like > the changes I make to the Job object given to me goes futile, as I get > a null at the map/reduce side for the configuration name I've stored > it as. > > What do I do to access either the ResourceSchema or even a > Job-Configuration variable that I wish to set post processing the > ResourceSchema? > > -- > Harsh J > www.harshj.com<http://www.harshj.com> > -- Harsh J www.harshj.com
