Hi Alan,
when you use a loader:
A = load 'stuff' using my.pig.Loader('foo', 'bar');
the loader gets constructed with 'foo', 'bar'', then it gets set up
(with the various setSignature, prepareToRead, etc, calls), and its
getNext() gets called repeatedly until there is nothing left to read.
when you use a udf:
B = foreach A generate my.pig.UDF($0);
pig iterates through relation A and invokes the UDF's exec method on a
tuple composed of the fields specified in your script -- in this case,
the first field in each row of A. If you want to have a non-default
constructor used to create the UDF instance that will be exec'd on all
these tuples, you can do this through a "define" call as I described
earlier.
Loaders (and Storers) are very different from UDFs in how they are
used and invoked, and they implement totally different interfaces.
-Dmitriy
On Mon, Apr 9, 2012 at 5:34 AM, Walker, Alan <[email protected]> wrote:
> Dmitriy,
>
> I have also tried that pattern for a Loader and it doesn't find the String
> constructor, it only works with the void constructor.
>
> grunt> define myreader com.sabre.pigshop.ShoppingReader('all');
> grunt> A = LOAD '/user/alanw/*.xml' USING myreader() AS (x);
> 2012-04-09 07:33:58,502 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR
> 2999: Unexpected internal error. could not instantiate
> 'com.sabre.pigshop.ShoppingReader' with arguments '[all]'
>
>
> This works:
>
> grunt> define myreader com.sabre.pigshop.ShoppingReader();
> grunt> A = LOAD '/user/alanw/*.xml' USING myreader AS (x);
>
>
> I haven't dug into the Pig source yet, perhaps the Loader functions are
> treated differently than another UDF? Seems unlikely.
>
> Thanks,
> Alan
>
>
> -----Original Message-----
> From: Dmitriy Ryaboy [mailto:[email protected]]
> Sent: Friday, April 06, 2012 6:21 PM
> To: [email protected]; Walker, Alan
> Subject: Re: Sending parameters to a customer load function
>
> Hi Alan,
> You can use "define" to supply an argument to a UDF constructor.
>
> You can see an example here:
> http://ofps.oreilly.com/titles/9781449302641/intro_pig_latin.html#udf_define
>
> I did just find to my surprise that this isn't in our documentation..
> we should add that.
>
> D
>
> On Fri, Apr 6, 2012 at 1:38 PM, Walker, Alan <[email protected]> wrote:
>> Hi,
>>
>> I'm having some challenges with a load function. It only seems to work
>> with a void constructor. The Java code has a void constructor and a String
>> constructor, much like the SimpleTextLoader example. Any thoughts on what
>> might be going wrong?
>>
>> public ShoppingReader() {
>> parms = "";
>> }
>>
>> public ShoppingReader(String tmp) {
>> parms = tmp;
>> }
>>
>> grunt> A = LOAD '/user/alanw/*.xml' USING
>> com.sabre.pigshop.ShoppingReader('all') AS (x);
>> 2012-04-06 16:04:08,593 [main] ERROR org.apache.pig.tools.grunt.Grunt -
>> ERROR 2999: Unexpected internal error. could not instantiate
>> com.sabre.pigshop.ShoppingReader' with arguments '[all]'
>>
>> Thanks,
>> Alan.
>>
