[
https://issues.apache.org/jira/browse/PIG-4846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liyunzhang_intel updated PIG-4846:
----------------------------------
Attachment: PIG-4846_1.patch
We need change bin/pig in PIG-4846_1.patch. add
$PIG_HOME/pig-0.16.0-SNAPSHOT-core-h2.jar:$PIG_HOME/test/perf/pigmix/pigmix.jar
to SPARK_DIST_CLASSPATH. If we don't add these two jars.
Some errors like "java.lang.NoClassDefFoundError:
org/apache/pig/builtin/PigStorage","java.lang.NoClassDefFoundError:
org/apache/pig/test/pigmix/udf/PigPerformanceLoader" will be thrown:
{code}
git diff bin/pig
diff --git a/bin/pig b/bin/pig
index ac2bb9d..74de9eb 100755
--- a/bin/pig
+++ b/bin/pig
@@ -390,6 +390,7 @@ CLASSPATH=${CLASSPATH}:${SPARK_JARS}
export SPARK_YARN_DIST_FILES=`echo ${SPARK_YARN_DIST_FILES} | sed 's/^,//g'`
export SPARK_JARS=${SPARK_YARN_DIST_FILES}
+export
SPARK_DIST_CLASSPATH=$SPARK_DIST_CLASSPATH:$PIG_HOME/pig-0.16.0-SNAPSHOT-core-h2.jar:$PIG_HOME/test/perf/pigmix/pigmix.jar
export SPARK_DIST_CLASSPATH
################# ADDING SPARK DEPENDENCIES ##################
{code}
> Use pigmix to test the performance of pig on spark
> --------------------------------------------------
>
> Key: PIG-4846
> URL: https://issues.apache.org/jira/browse/PIG-4846
> Project: Pig
> Issue Type: Sub-task
> Components: spark
> Reporter: liyunzhang_intel
> Assignee: liyunzhang_intel
> Fix For: spark-branch
>
> Attachments: PIG-4846.patch, PIG-4846_1.patch
>
>
> We can compare the performance between mr and spark mode by pigmix.
> The introduction of pigmix is
> https://cwiki.apache.org/confluence/display/PIG/PigMix.
> PIG-4846.patch is to make pigmix run by specied exectype.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
