Adding a hadoop-2.6 profile is not necessary. Use hadoop-2.4, which already exists and is intended for 2.4+. In fact this declaration is missing things that Hadoop 2 needs.
On Thu, Dec 18, 2014 at 3:46 AM, Kyle Lin <kylelin2...@gmail.com> wrote: > Hi there > > The following is my steps. And got the same exception with Daniel's. > Another question: how can I build a tgz file like the pre-build file I > download from official website? > > 1. download trunk from git. > > 2. add following lines in pom.xml > + <profile> > + <id>hadoop-2.6</id> > + <properties> > + <hadoop.version>2.6.0</hadoop.version> > + <protobuf.version>2.5.0</protobuf.version> > + <jets3t.version>0.9.0</jets3t.version> > + </properties> > + </profile> > > 3. run "mvn -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -DskipTests clean > package" > > 4. in $SPARK_HOME, run following command > ./bin/spark-submit --master yarn-cluster --class > org.apache.spark.examples.SparkPi lib/spark-examples*.jar 10 > > Kyle > > > 2014-12-18 2:24 GMT+08:00 Daniel Haviv <danielru...@gmail.com>: >> >> Thanks for your replies. >> I was building spark from trunk. >> >> Daniel >> >> On 17 בדצמ׳ 2014, at 19:49, Nicholas Chammas <nicholas.cham...@gmail.com> >> wrote: >> >> Thanks for the correction, Sean. Do the docs need to be updated on this >> point, or is it safer for now just to note 2.4 specifically? >> >> On Wed Dec 17 2014 at 5:54:53 AM Sean Owen <so...@cloudera.com> wrote: >>> >>> Spark works fine with 2.4 *and later*. The docs don't mean to imply >>> 2.4 is the last supported version. >>> >>> On Wed, Dec 17, 2014 at 10:19 AM, Nicholas Chammas >>> <nicholas.cham...@gmail.com> wrote: >>> > Spark 1.3 does not exist. Spark 1.2 hasn't been released just yet. >>> > Which >>> > version of Spark did you mean? >>> > >>> > Also, from what I can see in the docs, I believe the latest version of >>> > Hadoop that Spark supports is 2.4, not 2.6. >>> > >>> > Nick >>> > >>> > On Wed Dec 17 2014 at 2:09:56 AM Kyle Lin <kylelin2...@gmail.com> >>> > wrote: >>> >> >>> >> >>> >> I also got the same problem...... >>> >> >>> >> 2014-12-09 22:58 GMT+08:00 Daniel Haviv <danielru...@gmail.com>: >>> >>> >>> >>> Hi, >>> >>> I've built spark 1.3 with hadoop 2.6 but when I startup the >>> >>> spark-shell I >>> >>> get the following exception: >>> >>> >>> >>> 14/12/09 06:54:24 INFO server.AbstractConnector: Started >>> >>> SelectChannelConnector@0.0.0.0:4040 >>> >>> 14/12/09 06:54:24 INFO util.Utils: Successfully started service >>> >>> 'SparkUI' >>> >>> on port 4040. >>> >>> 14/12/09 06:54:24 INFO ui.SparkUI: Started SparkUI at >>> >>> http://hdname:4040 >>> >>> 14/12/09 06:54:25 INFO impl.TimelineClientImpl: Timeline service >>> >>> address: >>> >>> http://0.0.0.0:8188/ws/v1/timeline/ >>> >>> java.lang.NoClassDefFoundError: >>> >>> org/codehaus/jackson/map/deser/std/StdDeserializer >>> >>> at java.lang.ClassLoader.defineClass1(Native Method) >>> >>> at java.lang.ClassLoader.defineClass(ClassLoader.java:800) >>> >>> at >>> >>> >>> >>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) >>> >>> >>> >>> Any idea why ? >>> >>> >>> >>> Thanks, >>> >>> Daniel >>> >>> >>> >>> >>> > --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
