Re: using MultiStorage

2014-07-11 Thread Jimmy
I apologize, the code got cut off REGISTER /lib/pig/piggybank.jar; A = LOAD '/raw/*' USING PigStorage('\n') AS (mytext:chararray); B = FOREACH A GENERATE SUBSTRING(mytext,5,7), mytext; STORE B INTO '/output' USING org.apache.pig.piggybank.storage.MultiStorage('/output', '0', 'none', ' ') ;

using MultiStorage

2014-07-11 Thread Jimmy
have a directory with files with somewhat mailformatted logs (NEWLINE delimited). I would like to select specific position in each row and use it as a directory/file name, then store the original content as-is in the files. Basically re-partition files based on the content. code below works just