Re: Pattern for pig/hive setup files

2015-01-27 Thread Edward Capriolo
Newer versions of hive allow permanent and groovy defined functions.  I
know that does not help your pig case.

On Tue, Jan 27, 2015 at 5:08 PM, Pradeep Gollakota 
wrote:

> Hi All,
>
> I'm trying to establish a good pattern and practice with Oozie for sharing
> a setup file for pig/hive. For example, I have several scripts that use a
> set of UDFs that are built in-house. In order to use the UDF, I need to add
> the jar file, and then register the UDF. Rather than repeating this process
> multiple times, I can simply put this in the .hiverc/.pigbootup file of the
> user which will be running these scripts. However, this does not translate
> well into Oozieland. Another option is to put the common code into a
> "setup" file and add "source setup" to the beginning of every script. This
> is kind of ugly in that each file will have the same "source setup"
> command. There's another problem of the path'ing not necessarily being
> translatable across scrips/workflows.
>
> Has anyone come up with a good pattern for sharing hive/pig code across
> scripts in Oozieland?
>
> Thanks,
> Pradeep
>


Pattern for pig/hive setup files

2015-01-27 Thread Pradeep Gollakota
Hi All,

I'm trying to establish a good pattern and practice with Oozie for sharing
a setup file for pig/hive. For example, I have several scripts that use a
set of UDFs that are built in-house. In order to use the UDF, I need to add
the jar file, and then register the UDF. Rather than repeating this process
multiple times, I can simply put this in the .hiverc/.pigbootup file of the
user which will be running these scripts. However, this does not translate
well into Oozieland. Another option is to put the common code into a
"setup" file and add "source setup" to the beginning of every script. This
is kind of ugly in that each file will have the same "source setup"
command. There's another problem of the path'ing not necessarily being
translatable across scrips/workflows.

Has anyone come up with a good pattern for sharing hive/pig code across
scripts in Oozieland?

Thanks,
Pradeep