Re: Re: spark-1.1.0 with make-distribution.sh problem
Thank you very much. It is helpful for end users. Zhanfeng Huo From: Patrick Wendell Date: 2014-09-15 10:19 To: Zhanfeng Huo CC: user Subject: Re: spark-1.1.0 with make-distribution.sh problem Yeah that issue has been fixed by adding better docs, it just didn't make it in time for the release: https://github.com/apache/spark/blob/branch-1.1/make-distribution.sh#L54 On Thu, Sep 11, 2014 at 11:57 PM, Zhanfeng Huo wrote: resolved： ./make-distribution.sh --name spark-hadoop-2.3.0 --tgz --with-tachyon -Pyarn -Phadoop-2.3 -Dhadoop.version=2.3.0 -Phive -DskipTests This code is a bit misleading Zhanfeng Huo From: Zhanfeng Huo Date: 2014-09-12 14:13 To: user Subject: spark-1.1.0 with make-distribution.sh problem Hi, I compile spark with cmd "bash -x make-distribution.sh -Pyarn -Phive --skip-java-test --with-tachyon --tgz -Pyarn.version=2.3.0 -Phadoop.version=2.3.0", it errors. How to use it correct? message: + set -o pipefail + set -e +++ dirname make-distribution.sh ++ cd . ++ pwd + FWDIR=/home/syn/spark/spark-1.1.0 + DISTDIR=/home/syn/spark/spark-1.1.0/dist + SPARK_TACHYON=false + MAKE_TGZ=false + NAME=none + (( 7 )) + case $1 in + break + '[' -z /home/syn/usr/jdk1.7.0_55 ']' + '[' -z /home/syn/usr/jdk1.7.0_55 ']' + which git ++ git rev-parse --short HEAD + GITREV=5f6f219 + '[' '!' -z 5f6f219 ']' + GITREVSTRING=' (git revision 5f6f219)' + unset GITREV + which mvn ++ mvn help:evaluate -Dexpression=project.version ++ grep -v INFO ++ tail -n 1 + VERSION=1.1.0 ++ mvn help:evaluate -Dexpression=hadoop.version -Pyarn -Phive --skip-java-test --with-tachyon --tgz -Pyarn.version=2.3.0 -Phadoop.version=2.3.0 ++ grep -v INFO ++ tail -n 1 + SPARK_HADOOP_VERSION=' -X,--debug Produce execution debug output' Best Regards Zhanfeng Huo
Re: spark-1.1.0 with make-distribution.sh problem
Yeah that issue has been fixed by adding better docs, it just didn't make it in time for the release: https://github.com/apache/spark/blob/branch-1.1/make-distribution.sh#L54 On Thu, Sep 11, 2014 at 11:57 PM, Zhanfeng Huo wrote: > resolved： > > ./make-distribution.sh --name spark-hadoop-2.3.0 --tgz --with-tachyon > -Pyarn -Phadoop-2.3 -Dhadoop.version=2.3.0 -Phive -DskipTests > > This code is a bit misleading > > > -- > Zhanfeng Huo > > > *From:* Zhanfeng Huo > *Date:* 2014-09-12 14:13 > *To:* user > *Subject:* spark-1.1.0 with make-distribution.sh problem > Hi, > > I compile spark with cmd "bash -x make-distribution.sh -Pyarn -Phive > --skip-java-test --with-tachyon --tgz -Pyarn.version=2.3.0 > -Phadoop.version=2.3.0", it errors. > > How to use it correct? > >message: > + set -o pipefail > + set -e > +++ dirname make-distribution.sh > ++ cd . > ++ pwd > + FWDIR=/home/syn/spark/spark-1.1.0 > + DISTDIR=/home/syn/spark/spark-1.1.0/dist > + SPARK_TACHYON=false > + MAKE_TGZ=false > + NAME=none > + (( 7 )) > + case $1 in > + break > + '[' -z /home/syn/usr/jdk1.7.0_55 ']' > + '[' -z /home/syn/usr/jdk1.7.0_55 ']' > + which git > ++ git rev-parse --short HEAD > + GITREV=5f6f219 > + '[' '!' -z 5f6f219 ']' > + GITREVSTRING=' (git revision 5f6f219)' > + unset GITREV > + which mvn > ++ mvn help:evaluate -Dexpression=project.version > ++ grep -v INFO > ++ tail -n 1 > + VERSION=1.1.0 > ++ mvn help:evaluate -Dexpression=hadoop.version -Pyarn -Phive > --skip-java-test --with-tachyon --tgz -Pyarn.version=2.3.0 > -Phadoop.version=2.3.0 > ++ grep -v INFO > ++ tail -n 1 > + SPARK_HADOOP_VERSION=' -X,--debug Produce execution debug output' > > Best Regards > -- > Zhanfeng Huo > >
Re: spark-1.1.0 with make-distribution.sh problem
resolved： ./make-distribution.sh --name spark-hadoop-2.3.0 --tgz --with-tachyon -Pyarn -Phadoop-2.3 -Dhadoop.version=2.3.0 -Phive -DskipTests This code is a bit misleading Zhanfeng Huo From: Zhanfeng Huo Date: 2014-09-12 14:13 To: user Subject: spark-1.1.0 with make-distribution.sh problem Hi, I compile spark with cmd "bash -x make-distribution.sh -Pyarn -Phive --skip-java-test --with-tachyon --tgz -Pyarn.version=2.3.0 -Phadoop.version=2.3.0", it errors. How to use it correct? message: + set -o pipefail + set -e +++ dirname make-distribution.sh ++ cd . ++ pwd + FWDIR=/home/syn/spark/spark-1.1.0 + DISTDIR=/home/syn/spark/spark-1.1.0/dist + SPARK_TACHYON=false + MAKE_TGZ=false + NAME=none + (( 7 )) + case $1 in + break + '[' -z /home/syn/usr/jdk1.7.0_55 ']' + '[' -z /home/syn/usr/jdk1.7.0_55 ']' + which git ++ git rev-parse --short HEAD + GITREV=5f6f219 + '[' '!' -z 5f6f219 ']' + GITREVSTRING=' (git revision 5f6f219)' + unset GITREV + which mvn ++ mvn help:evaluate -Dexpression=project.version ++ grep -v INFO ++ tail -n 1 + VERSION=1.1.0 ++ mvn help:evaluate -Dexpression=hadoop.version -Pyarn -Phive --skip-java-test --with-tachyon --tgz -Pyarn.version=2.3.0 -Phadoop.version=2.3.0 ++ grep -v INFO ++ tail -n 1 + SPARK_HADOOP_VERSION=' -X,--debug Produce execution debug output' Best Regards Zhanfeng Huo
spark-1.1.0 with make-distribution.sh problem
Hi, I compile spark with cmd "bash -x make-distribution.sh -Pyarn -Phive --skip-java-test --with-tachyon --tgz -Pyarn.version=2.3.0 -Phadoop.version=2.3.0", it errors. How to use it correct? message: + set -o pipefail + set -e +++ dirname make-distribution.sh ++ cd . ++ pwd + FWDIR=/home/syn/spark/spark-1.1.0 + DISTDIR=/home/syn/spark/spark-1.1.0/dist + SPARK_TACHYON=false + MAKE_TGZ=false + NAME=none + (( 7 )) + case $1 in + break + '[' -z /home/syn/usr/jdk1.7.0_55 ']' + '[' -z /home/syn/usr/jdk1.7.0_55 ']' + which git ++ git rev-parse --short HEAD + GITREV=5f6f219 + '[' '!' -z 5f6f219 ']' + GITREVSTRING=' (git revision 5f6f219)' + unset GITREV + which mvn ++ mvn help:evaluate -Dexpression=project.version ++ grep -v INFO ++ tail -n 1 + VERSION=1.1.0 ++ mvn help:evaluate -Dexpression=hadoop.version -Pyarn -Phive --skip-java-test --with-tachyon --tgz -Pyarn.version=2.3.0 -Phadoop.version=2.3.0 ++ grep -v INFO ++ tail -n 1 + SPARK_HADOOP_VERSION=' -X,--debug Produce execution debug output' Best Regards Zhanfeng Huo
