have had some success creating and registering one single jar with
dependencies. downside - the jar can be quite big. If you use maven to build
your jar files just the following to the plugin section of your pom.xml
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>attached</goal>
</goals>
</execution>
</executions>
</plugin>
On Oct 21, 2010, at 6:08 PM, Yong-gang Cao wrote:
> Hi,
> I met a headache about using UDFs with many dependence, adding them using
> register command is very painful and not extensible. I can make
> self-contained jar for hadoop job using maven (a jar with a lib directory
> which contains all jars will be used for class look up), but it seems
> doesn't work for pig. pig just treat that jar as a regular jar and try to
> find classes directly inside it instead of inside those embedded jars.
> Is there a way to make pig do the hadoop way of looking into the
> self-contained big jar for class loading?
> Thanks!
>
> --
> Regards,
>
> Yong-gang Cao
> Seattle,WA,98104
