YoungWoo Kim created BIGTOP-2959:
------------------------------------
Summary: Add SPARK_DIST_CLASSPATH to include libraries for HDFS
and YARN
Key: BIGTOP-2959
URL: https://issues.apache.org/jira/browse/BIGTOP-2959
Project: Bigtop
Issue Type: Improvement
Components: build, spark
Affects Versions: 1.2.1
Reporter: YoungWoo Kim
Priority: Minor
It might be a better way to use SPARK_DIST_CLASSPATH for including the Hadoop
libs instead of CLASSPATH.
{code}
export SPARK_DIST_CLASSPATH=$(hadoop classpath)
{code}
Refs. Using Spark's "Hadoop Free" Build,
https://spark.apache.org/docs/2.1.1/hadoop-provided.html
Current spark-env.sh:
{code}
# Let's make sure that all needed hadoop libs are added properly
export
CLASSPATH="$CLASSPATH:$HADOOP_HOME/*:$HADOOP_HDFS_HOME/*:$HADOOP_YARN_HOME/*:$HADOOP_MAPRED_HOME/*"
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
