Using pig 0.9. My data is very dynamic so I use a custom LoadFunc to parse it.
The problem is that I cant figure out how to access the schema that is defined
in the load statement. I am forced to do something like this:
A = LOAD '/test/loadfiles/*' USING
com.custom.pig.LogStorage('(site:chararray,zone:chararray,pos:chararray)') AS
(site:chararray,zone:chararray,pos:chararray);
I have to define my schema twice, once for my custom loader and once for pig. I
can see that there is a LoadCastor interface, but its not clear to me how to
use it in LoadFunc. All I need to do is get access to the schema inside of my
LogStorage class. Whats the proper way to load custom (non uniform) data into a
schema?
thanks
