Dmitriy Ryaboy
Mon, 15 Mar 2010 14:59:26 -0700
Alan, this would be quite useful, as essentially this would allow developers to create functions by writing them into separate pig scripts and combining them as necessary.
For example we have code that auto-generates load statements with fairly complex schemas based on protocol buffers (see http://www.slideshare.net/hadoopusergroup/twitter-protobufs-and-hadoop-hug-021709). It would be very handy to be able to say something like #include common_jars.pig #include load_tweets.pig #include load_users.pig #include filter_nonenglish_tweets.pig #include geomap_users.pig .. etc .. -D On Mon, Mar 15, 2010 at 2:23 PM, Alan Gates <ga...@yahoo-inc.com> wrote: > > On Mar 12, 2010, at 10:36 AM, hc busy wrote: > > >> >> Is there any work towards something like C languages '#include' in Pig? My >> large pig script is actually developed separately in several smaller pig >> files. Individually the pig files do not run because they depend on >> previous >> scripts, but logically they are separate because each step does something >> different. >> >> Currently the only thing existing along these lines is the exec command > in grunt. I don't think we're opposed to a #include functionality, we just > haven't done it. However, given that Pig doesn't have function calls, and > presumably each Pig Latin script is self contained, it isn't clear to me how > useful it will be. > > Alan. >