Benjamin Reed
Mon, 07 Apr 2008 08:15:49 -0700
PIG-55, which may be the patch you are talking about, was committed last week. It allows a LoadFunc to validate the filename, and therefore use the filename as a DB spec or ignore it completely. Does PIG-55 satisfy your requirements? ben On Sunday 06 April 2008 18:46:02 Spencer Proffit wrote: > Pig comes very close to meeting my needs, however I need to be able to > load from DB. I noticed in Jira there was a patch to remove > dependency on file names from the loader. I'd also like it to scale > down so I can test and dev without hadoop. Is that possible? I'm > also thinking about using it with Amazon s3 (disclosure: I work for > Amazon). If these changes are within the scope of where pig wants to > go (it eats everything, right), I'd be happy to work on the changes > myself. > > Spencer