Grant, Globbing is supported by Pig (for pig.additional.jars) only for LocalFileSystem.
The <file>'s pre-# component can be any arbitrary HDFS path though, but not the argument to pig.additional.jars (these are picked up from resources such as uber-jars or local file systems only). On Thu, Nov 8, 2012 at 1:12 AM, Grant Ingersoll <[email protected]> wrote: > > On Nov 7, 2012, at 12:51 PM, Harsh J wrote: > >> Hi Grant, >> >> You can leverage the <argument> feature of the Pig action, in tandem >> with the distributed-cache-using <file> element to do this I think >> (over pig action schema 0.2). >> >> If you add after your <script>, the following: >> >> <argument>-Dpig.additional.jars=jar1.jar:jar2.jar</argument> >> >> And then in the outer section, add: >> >> <file>lib/jar1.jar#jar1.jar</file> >> <file>lib/jar2.jar#jar2.jar</file> >> >> (Assuming your WF has a lib/ directory with jar1.jar and jar2.jar in it) >> >> Then Oozie will load these jars onto distributed cache, and symlink >> them (during runtime) to the task working directory (sorta like a pwd >> for the task). Hence, your Pig will "see" these files locally and >> utilize them properly for the "pig.additional.jars" feature. >> >> Does this work for you? > > I'll give it a try. > > Is an HDFS path and glob OK? > > >> >> On Wed, Nov 7, 2012 at 10:54 PM, Grant Ingersoll <[email protected]> wrote: >>> Hi, >>> >>> I was wondering how Oozie deals with additional JARs one needs for Pig >>> files. Currently, I have a REGISTER statement in Pig that points at the >>> location of the libs, but I'd like to get away from that and use Pig's >>> additional.jars mechanism, but I don't see support for that in the Oozie >>> spec for the Pig action. >>> >>> Is this possible? I'm on 3.2-SNAPSHOT. >>> >>> Thanks, >>> Grant >>> >>> -------------------------------------------- >>> Grant Ingersoll >>> http://www.lucidworks.com >>> >>> >>> >>> >> >> >> >> -- >> Harsh J > > -------------------------------------------- > Grant Ingersoll > http://www.lucidworks.com > > > > -- Harsh J
