They derive from the same classes as far as lzo handling goes, so I suspect something's up with your environment or inputs if you get LzoTokenizedLoader to work, but LzoJsonStorage does not.
Note that LzoTokenizedLoader is deprecated -- just use LzoPigStorage. JsonLoader wouldn't work for you because it expects the complete input line to be json, not part of it. You want to load with LzoPigStorage, and then apply the JsonStringToMap udf to the third field. -D On Fri, Sep 9, 2011 at 3:49 PM, Eli Finkelshteyn <[email protected]> wrote: > Hi, > I'm currently working on trying to load lzos that contain some JSON > elements. This is of the form: > > item1 item2 {'thing1':'1','thing2':'2'} > item3 item4 {'thing3':'1','thing27':'2'} > item5 item6 {'thing5':'1','thing19':'2'} > > I was thinking I could use LzoJsonLoader for this, but it keeps throwing me > errors like: > ERROR com.hadoop.compression.lzo.**LzoCodec - Cannot load native-lzo > without native-hadoop > > This is despite the fact that I can load normal lzos just fine using > LzoTokenizedLoader('\\t'). So, now I'm at a bit of a standstill. What should > I do to go about loading these files? Does anyone have any ideas? > > Cheers, > Eli >
