I understand you solution. I want to reuse exisitng code. This json with schema to Tuple conversion utility would took to much time to implement.
2013/8/30 Ruslan Al-Fakikh <[email protected]> > If you don't want to run the whole Pig script, then you probably shouldn't > use the AvroStorage, because it loads data to the Pig as per my > understanding. But as far as I understand you want to avoid the Pig > execution environment and just execute the UDF in isolation. > Probably you need to use the core Avro Java library, not the Pig+Avro > integration. In that case you would still need to write some conversion to > Pig tuples:( > > Hope that helps, > Ruslan > > > On Fri, Aug 30, 2013 at 11:22 AM, Serega Sheypak > <[email protected]>wrote: > > > 1. I want to test logic of my UDF without running the whole pig. I can > set > > breakpoints, e.t.c. > > 2. UDF accepts a bag of tuples. I have source data which I want to read > > using AvroStorage to tuples and feed to UDF. Avro storage would preserve > > schema. I don;t have to wrie cusotm converters/cast'ers e,t,c, > > > > >>Or you can extract the > > >>logic to a java method outside of the UDF and test it within normal > junit > > Who will convert sample avro data for me to Tuples and feed them to java > > method? I don't want to invent the wheel and repeat AvroStorage > > functionality. > > > > > > 2013/8/30 Ruslan Al-Fakikh <[email protected]> > > > > > Hi, > > > > > > What exactly do you want to test? The logic inside UDFs? In that case I > > > would recommend not bothering about input format of the whole Pig > script. > > > You can use plain text files as input for the test. Or you can extract > > the > > > logic to a java method outside of the UDF and test it within normal > junit > > > tests. Also these ideas come to my mind: > > > 1) You can take a look at PigUnit unility > > > 2) Avro has a human-readable (json text, non-binary) form for debugging > > > purposes > > > > > > Best Regards, > > > Ruslan > > > > > > > > > On Thu, Aug 29, 2013 at 2:56 PM, Serega Sheypak < > > [email protected] > > > >wrote: > > > > > > > Hi, we have itegration test java utility which helps to test pig > > scripts. > > > > We often develop different UDFs in java and I would like to create > unit > > > > tests for them. Right now they are tested with pig scripts during > > > > integration tests. > > > > > > > > I have a pack of prepared avro files with etalon input for my java > > UDFs. > > > > I would like to use some utility which would help me to convert avro > to > > > > tuple list. > > > > I'm trying to use org.apache.pig.piggybank.storage.avro.AvroStorage > > with > > > no > > > > luck. I don't understand how it works. > > > > > > > > > >
