Re: Maven build failed (Spark master)

Thank you. But I'm getting same warnings and it's still preventing the archive from being generated. I've ran this on both OSX Lion and Ubuntu 12. Same error. No .gz file On Mon, Oct 26, 2015 at 9:10 PM, Ted Yu wrote: > Looks like '-Pyarn' was missing in your command. > >

Re: Maven build failed (Spark master)

Can you try the same command shown in the pull request ? Thanks > On Oct 27, 2015, at 12:40 AM, Kayode Odeyemi wrote: > > Thank you. > > But I'm getting same warnings and it's still preventing the archive from > being generated. > > I've ran this on both OSX Lion and

Re: Maven build failed (Spark master)

I issued the same basic command and it worked fine. RADTech-MBP:spark $ ./make-distribution.sh --name hadoop-2.6 --tgz -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -Phive -Phive-thriftserver -DskipTests Which created: spark-1.6.0-SNAPSHOT-bin-hadoop-2.6.tgz in the root directory of the project.

Re: Maven build failed (Spark master)

I used the following command: make-distribution.sh --name custom-spark --tgz -Phadoop-2.4 -Phive -Phive-thriftserver -Pyarn spark-1.6.0-SNAPSHOT-bin-custom-spark.tgz was generated (with patch from SPARK-11348) Can you try above command ? Thanks On Tue, Oct 27, 2015 at 7:03 AM, Kayode Odeyemi

Re: Maven build failed (Spark master)

Thanks gents. Removal of 'clean package -U' made the difference. On Tue, Oct 27, 2015 at 6:39 PM, Todd Nist wrote: > I issued the same basic command and it worked fine. > > RADTech-MBP:spark $ ./make-distribution.sh --name hadoop-2.6 --tgz -Pyarn > -Phadoop-2.6

Re: Maven build failed (Spark master)

Seems the build and directory structure in dist is similar to the .gz file downloaded from the downloads page. Can the dist directory be used as is? On Tue, Oct 27, 2015 at 4:03 PM, Kayode Odeyemi wrote: > Ted, I switched to this: > > ./make-distribution.sh --name

Re: Maven build failed (Spark master)

Ted, I switched to this: ./make-distribution.sh --name spark-latest --tgz -Dhadoop.version=2.6.0 -Phadoop-2.6 -Phive -Phive-thriftserver -Pyarn -DskipTests clean package -U Same error. No .gz file. Here's the bottom output log: + rm -rf /home/emperor/javaprojects/spark/dist + mkdir -p

Re: Maven build failed (Spark master)

I used this command which is synonymous to what you have: ./make-distribution.sh --name spark-latest --tgz --mvn mvn -Dhadoop.version=2.6.0 -Phadoop-2.6 -Phive -Phive-thriftserver -DskipTests clean package -U But I still see WARNINGS like this in the output and no .gz file created: cp:

Re: Maven build failed (Spark master)

Looks like '-Pyarn' was missing in your command. On Mon, Oct 26, 2015 at 12:06 PM, Kayode Odeyemi wrote: > I used this command which is synonymous to what you have: > > ./make-distribution.sh --name spark-latest --tgz --mvn mvn > -Dhadoop.version=2.6.0 -Phadoop-2.6 -Phive

Re: Maven build failed (Spark master)

I see a lot of stuffs like this after the a successful maven build: cp: /usr/local/spark-latest/spark-[WARNING] See http://docs.codehaus.org/display/MAVENUSER/Shade+Plugin-bin-spark-latest/python/test_support/sql/parquet_partitioned/year=2014/month=9/day=1/ part-r-8.gz.parquet: No such file

Re: Maven build failed (Spark master)

If you use the command shown in: https://github.com/apache/spark/pull/9281 You should have got the following: ./dist/python/test_support/sql/parquet_partitioned/year=2014/month=9/day=1/part-r-8.gz.parquet

Re: Maven build failed (Spark master)

Hi, The ./make_distribution task completed. However, I can't seem to locate the .tar.gz file. Where does Spark save this? or should I just work with the dist directory? On Fri, Oct 23, 2015 at 4:23 PM, Kayode Odeyemi wrote: > I saw this when I tested manually (without

Re: Maven build failed (Spark master)

In 1.4 ./make_distribution produces a .tgz file in the root directory (same directory that make_distribution is in) On Mon, Oct 26, 2015 at 8:46 AM, Kayode Odeyemi wrote: > Hi, > > The ./make_distribution task completed. However, I can't seem to locate the > .tar.gz file. >

Re: Maven build failed (Spark master)

This doesn't show the actual error output from Maven. I have a strong guess that you haven't set MAVEN_OPTS to increase the memory Maven can use. On Fri, Oct 23, 2015 at 6:14 AM, Kayode Odeyemi wrote: > Hi, > > I can't seem to get a successful maven build. Please see command

Maven build failed (Spark master)

Hi, I can't seem to get a successful maven build. Please see command output below: bash-3.2$ ./make-distribution.sh --name spark-latest --tgz --mvn mvn -Dhadoop.version=2.7.0 -Phadoop-2.7 -Phive -Phive-thriftserver -DskipTests clean package +++ dirname ./make-distribution.sh ++ cd . ++ pwd +

Re: Maven build failed (Spark master)

I saw this when I tested manually (without ./make-distribution) Detected Maven Version: 3.2.2 is not in the allowed range 3.3.3. So I simply upgraded maven to 3.3.3. Resolved. Thanks On Fri, Oct 23, 2015 at 3:17 PM, Sean Owen wrote: > This doesn't show the actual error