Hi all, I have a same error and it seems depending on Maven versions.
I tried building Spark using Maven with several versions on Jenkins. + Output of "/Users/tomohiko/.jenkins/tools/hudson.tasks.Maven_MavenInstallation/mvn-3.3.3/bin/mvn -version": Apache Maven 3.3.3 (7994120775791599e205a5524ec3e0dfe41d4a06; 2015-04-22T20:57:37+09:00) Maven home: /Users/tomohiko/.jenkins/tools/hudson.tasks.Maven_MavenInstallation/mvn-3.3.3 Java version: 1.8.0, vendor: Oracle Corporation Java home: /Library/Java/JavaVirtualMachines/jdk1.8.0.jdk/Contents/Home/jre Default locale: en_US, platform encoding: UTF-8 OS name: "mac os x", version: "10.10.3", arch: "x86_64", family: "mac" + Jenkins Configuration: Jenkins project type: Maven Project Goals and options: -Phadoop-2.6 -DskipTests clean package + Maven versions and results: 3.3.3 -> infinite loop 3.3.1 -> infinite loop 3.2.5 -> SUCCESS So do we prefer to build Spark with Maven 3.2.5? On 4 July 2015 at 12:28, Andrew Or <and...@databricks.com> wrote: > Thanks, I just tried it with 3.3.3 and I was able to reproduce it as well. > > 2015-07-03 18:51 GMT-07:00 Tarek Auel <tarek.a...@gmail.com>: > >> That's mine >> >> Apache Maven 3.3.3 (7994120775791599e205a5524ec3e0dfe41d4a06; >> 2015-04-22T04:57:37-07:00) >> >> Maven home: /usr/local/Cellar/maven/3.3.3/libexec >> >> Java version: 1.8.0_45, vendor: Oracle Corporation >> >> Java home: >> /Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre >> >> Default locale: en_US, platform encoding: UTF-8 >> >> OS name: "mac os x", version: "10.10.3", arch: "x86_64", family: "mac" >> >> On Fri, Jul 3, 2015 at 6:32 PM Ted Yu <yuzhih...@gmail.com> wrote: >> >>> Here is mine: >>> >>> Apache Maven 3.3.1 (cab6659f9874fa96462afef40fcf6bc033d58c1c; >>> 2015-03-13T13:10:27-07:00) >>> Maven home: /home/hbase/apache-maven-3.3.1 >>> Java version: 1.8.0_45, vendor: Oracle Corporation >>> Java home: /home/hbase/jdk1.8.0_45/jre >>> Default locale: en_US, platform encoding: UTF-8 >>> OS name: "linux", version: "2.6.32-504.el6.x86_64", arch: "amd64", >>> family: "unix" >>> >>> On Fri, Jul 3, 2015 at 6:05 PM, Andrew Or <and...@databricks.com> wrote: >>> >>>> @Tarek and Ted, what maven versions are you using? >>>> >>>> 2015-07-03 17:35 GMT-07:00 Krishna Sankar <ksanka...@gmail.com>: >>>> >>>>> Patrick, >>>>> I assume an RC3 will be out for folks like me to test the >>>>> distribution. As usual, I will run the tests when you have a new >>>>> distribution. >>>>> Cheers >>>>> <k/> >>>>> >>>>> On Fri, Jul 3, 2015 at 4:38 PM, Patrick Wendell <pwend...@gmail.com> >>>>> wrote: >>>>> >>>>>> Patch that added test-jar dependencies: >>>>>> https://github.com/apache/spark/commit/bfe74b34 >>>>>> >>>>>> Patch that originally disabled dependency reduced poms: >>>>>> >>>>>> https://github.com/apache/spark/commit/984ad60147c933f2d5a2040c87ae687c14eb1724 >>>>>> >>>>>> Patch that reverted the disabling of dependency reduced poms: >>>>>> >>>>>> https://github.com/apache/spark/commit/bc51bcaea734fe64a90d007559e76f5ceebfea9e >>>>>> >>>>>> On Fri, Jul 3, 2015 at 4:36 PM, Patrick Wendell <pwend...@gmail.com> >>>>>> wrote: >>>>>> > Okay I did some forensics with Sean Owen. Some things about this >>>>>> bug: >>>>>> > >>>>>> > 1. The underlying cause is that we added some code to make the tests >>>>>> > of sub modules depend on the core tests. For unknown reasons this >>>>>> > causes Spark to hit MSHADE-148 for *some* combinations of build >>>>>> > profiles. >>>>>> > >>>>>> > 2. MSHADE-148 can be worked around by disabling building of >>>>>> > "dependency reduced poms" because then the buggy code path is >>>>>> > circumvented. Andrew Or did this in a patch on the 1.4 branch. >>>>>> > However, that is not a tenable option for us because our *published* >>>>>> > pom files require dependency reduction to substitute in the scala >>>>>> > version correctly for the poms published to maven central. >>>>>> > >>>>>> > 3. As a result, Andrew Or reverted his patch recently, causing some >>>>>> > package builds to start failing again (but publishing works now). >>>>>> > >>>>>> > 4. The reason this is not detected in our test harness or release >>>>>> > build is that it is sensitive to the profiles enabled. The >>>>>> combination >>>>>> > of profiles we enable in the test harness and release builds do not >>>>>> > trigger this bug. >>>>>> > >>>>>> > The best path I see forward right now is to do the following: >>>>>> > >>>>>> > 1. Disable creation of dependency reduced poms by default (this >>>>>> > doesn't matter for people doing a package build) so typical users >>>>>> > won't have this bug. >>>>>> > >>>>>> > 2. Add a profile that re-enables that setting. >>>>>> > >>>>>> > 3. Use the above profile when publishing release artifacts to maven >>>>>> central. >>>>>> > >>>>>> > 4. Hope that we don't hit this bug for publishing. >>>>>> > >>>>>> > - Patrick >>>>>> > >>>>>> > On Fri, Jul 3, 2015 at 3:51 PM, Tarek Auel <tarek.a...@gmail.com> >>>>>> wrote: >>>>>> >> Doesn't change anything for me. >>>>>> >> >>>>>> >> On Fri, Jul 3, 2015 at 3:45 PM Patrick Wendell <pwend...@gmail.com> >>>>>> wrote: >>>>>> >>> >>>>>> >>> Can you try using the built in maven "build/mvn..."? All of our >>>>>> builds >>>>>> >>> are passing on Jenkins so I wonder if it's a maven version issue: >>>>>> >>> >>>>>> >>> https://amplab.cs.berkeley.edu/jenkins/view/Spark-QA-Compile/ >>>>>> >>> >>>>>> >>> - Patrick >>>>>> >>> >>>>>> >>> On Fri, Jul 3, 2015 at 3:14 PM, Ted Yu <yuzhih...@gmail.com> >>>>>> wrote: >>>>>> >>> > Please take a look at SPARK-8781 >>>>>> >>> > (https://github.com/apache/spark/pull/7193) >>>>>> >>> > >>>>>> >>> > Cheers >>>>>> >>> > >>>>>> >>> > On Fri, Jul 3, 2015 at 3:05 PM, Tarek Auel < >>>>>> tarek.a...@gmail.com> wrote: >>>>>> >>> >> >>>>>> >>> >> I found a solution, there might be a better one. >>>>>> >>> >> >>>>>> >>> >> https://github.com/apache/spark/pull/7217 >>>>>> >>> >> >>>>>> >>> >> On Fri, Jul 3, 2015 at 2:28 PM Robin East < >>>>>> robin.e...@xense.co.uk> >>>>>> >>> >> wrote: >>>>>> >>> >>> >>>>>> >>> >>> Yes me too >>>>>> >>> >>> >>>>>> >>> >>> On 3 Jul 2015, at 22:21, Ted Yu <yuzhih...@gmail.com> wrote: >>>>>> >>> >>> >>>>>> >>> >>> This is what I got (the last line was repeated non-stop): >>>>>> >>> >>> >>>>>> >>> >>> [INFO] Replacing original artifact with shaded artifact. >>>>>> >>> >>> [INFO] Replacing >>>>>> >>> >>> >>>>>> /home/hbase/spark/bagel/target/spark-bagel_2.10-1.5.0-SNAPSHOT.jar >>>>>> >>> >>> with >>>>>> >>> >>> >>>>>> >>> >>> >>>>>> /home/hbase/spark/bagel/target/spark-bagel_2.10-1.5.0-SNAPSHOT-shaded.jar >>>>>> >>> >>> [INFO] Dependency-reduced POM written at: >>>>>> >>> >>> /home/hbase/spark/bagel/dependency-reduced-pom.xml >>>>>> >>> >>> [INFO] Dependency-reduced POM written at: >>>>>> >>> >>> /home/hbase/spark/bagel/dependency-reduced-pom.xml >>>>>> >>> >>> >>>>>> >>> >>> On Fri, Jul 3, 2015 at 1:13 PM, Tarek Auel < >>>>>> tarek.a...@gmail.com> >>>>>> >>> >>> wrote: >>>>>> >>> >>>> >>>>>> >>> >>>> Hi all, >>>>>> >>> >>>> >>>>>> >>> >>>> I am trying to build the master, but it stucks and prints >>>>>> >>> >>>> >>>>>> >>> >>>> [INFO] Dependency-reduced POM written at: >>>>>> >>> >>>> /Users/tarek/test/spark/bagel/dependency-reduced-pom.xml >>>>>> >>> >>>> >>>>>> >>> >>>> build command: mvn -DskipTests clean package >>>>>> >>> >>>> >>>>>> >>> >>>> Do others have the same issue? >>>>>> >>> >>>> >>>>>> >>> >>>> Regards, >>>>>> >>> >>>> Tarek >>>>>> >>> >>> >>>>>> >>> >>> >>>>>> >>> >>> >>>>>> >>> > >>>>>> >>>>>> --------------------------------------------------------------------- >>>>>> To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org >>>>>> For additional commands, e-mail: dev-h...@spark.apache.org >>>>>> >>>>>> >>>>> >>>> >>> > -- class Cocoatomo: name = 'cocoatomo' email_address = 'cocoatom...@gmail.com' twitter_id = '@cocoatomo'
