You can use a shell script to invoke your Pig scripts. Then you can do anything you want in that shell script. We are following this design. It gives you power to conditionally run, modify Pig scripts and its parameters too.
Of course the assumption here is that, you want to delete the folders and file before or after the Pig script ends and not in between. Regards, Shahab On Wed, Sep 11, 2013 at 10:08 AM, Dip Kharod <[email protected]> wrote: > Hi, > My Pig script stores data in an HDFS folder and I want to delete > those folders (like in MapReduce with File object) in the script, every > time I run it - don't want to delete manually. > My attempt with “hadoop fs –rmr” or “rmf” does not work – meaning Pig > script execution fails saying that destination folder exists (I need to > manually delete the folder prior to running the script) It looks > like the first thing that the interpreter does is check for the output > folder > check. > Any pointers is appreciated. > Thanks,Dip
