[
https://issues.apache.org/jira/browse/PIG-4233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Praveen Rachabattuni updated PIG-4233:
--------------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Package pig along with dependencies into a fat jar while job submission to
> Spark cluster
> ----------------------------------------------------------------------------------------
>
> Key: PIG-4233
> URL: https://issues.apache.org/jira/browse/PIG-4233
> Project: Pig
> Issue Type: Sub-task
> Components: spark
> Reporter: Praveen Rachabattuni
> Assignee: Praveen Rachabattuni
> Attachments: PIG-4233.patch
>
>
> Currently we have a fat jar created in legacy directory which contains pig
> along with dependencies.
> Would need to modify build.xml to add spark dependency jars to include in
> legacy fat jar.
> Running job on Spark cluster:
> 1. export SPARK_HOME=/path/to/spark
> 2. export
> SPARK_PIG_JAR=$PIG_HOME/legacy/pig-0.14.0-SNAPSHOT-withouthadoop-h1.jar
> 3. export SPARK_MASTER=spark://localhost:7077
> 4 export HADOOP_HOME=/path/to/hadoop
> 5. Launch the job using ./bin/pig -x spark
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
