Pig has MultiStorage in piggybank. https://github.com/apache/pig/blob/trunk/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/storage/MultiStorage.java
I think it has some limitation. You can check the javadoc/jiras for it. Thanks, Aniket On Mon, Jan 9, 2012 at 10:21 PM, IGZ Nick <[email protected]> wrote: > I am able to group the tuples by date. But the problem I am facing is how > do I ensure that when I finally STORE it, it is stored in separate folders? > > On Tue, Jan 10, 2012 at 11:27 AM, Daniel Dai <[email protected]> > wrote: > > > You can use custom partitioner. Check > > http://pig.apache.org/docs/r0.9.1/basic.html#partitionby. > > > > Daniel > > > > On Mon, Jan 9, 2012 at 9:39 PM, IGZ Nick <[email protected]> wrote: > > > > > Hi, > > > > > > What I would like to do is to store outputs to different directories > > based > > > on record value. Essentially I want to read the date from a field and > > store > > > the output in yyyy/mm/dd directory structure. How should I go about > > this? I > > > want to use AvroStorage for storing the stuff. I want to specify STORE > > xyz > > > INTO '$location' USING MyStorage(); where $location would be the base > > > output directory. MyStorage() would be the modified version of > > AvroStorage > > > which stores the values in $location/yyyy/mm/dd/part-abc files, reading > > the > > > yyyymmdd from a particular field in the input records. > > > > > > What is the way to achieve this with minimal changes? > > > > > > Nick > > > > > > -- "...:::Aniket:::... Quetzalco@tl"
