Answering my own question: "dependency-graph" On Thu, Nov 5, 2015 at 11:44 AM, Marcelo Vanzin <[email protected]> wrote: > Does anyone know how to get something similar to "mvn dependency:tree" from > sbt? > > mvn dependency:tree with hadoop 2.6.0 does not show any instances of guava > 16... > > On Thu, Nov 5, 2015 at 11:37 AM, Ted Yu <[email protected]> wrote: >> build/sbt -Pyarn -Phadoop-2.6 -Phive -Phive-thriftserver >> -Dhadoop.version=2.6.0 -DskipTests assembly >> >> The above command fails on Mac. >> >> build/sbt -Pyarn -Phadoop-2.2 -Phive -Phive-thriftserver -Pkinesis-asl >> -DskipTests assembly >> >> The above command, used by Jenkins, passes. >> That's why the build error wasn't caught. >> >> FYI >> >> On Thu, Nov 5, 2015 at 11:07 AM, Dilip Biswal <[email protected]> wrote: >>> >>> Hello Ted, >>> >>> Thanks for your response. >>> >>> Here is the command i used : >>> >>> build/sbt clean >>> build/sbt -Pyarn -Phadoop-2.6 -Phive -Phive-thriftserver >>> -Dhadoop.version=2.6.0 -DskipTests assembly >>> >>> I am building on CentOS and on master branch. >>> >>> One other thing, i was able to build fine with the above command up until >>> recently. I think i have stared >>> to have problem after SPARK-11073 where the HashCodes import was added. >>> >>> Regards, >>> Dilip Biswal >>> Tel: 408-463-4980 >>> [email protected] >>> >>> >>> >>> From: Ted Yu <[email protected]> >>> To: Dilip Biswal/Oakland/IBM@IBMUS >>> Cc: Jean-Baptiste Onofré <[email protected]>, "[email protected]" >>> <[email protected]> >>> Date: 11/05/2015 10:46 AM >>> Subject: Re: Master build fails ? >>> ________________________________ >>> >>> >>> >>> Dilip: >>> Can you give the command you used ? >>> >>> Which release were you building ? >>> What OS did you build on ? >>> >>> Cheers >>> >>> On Thu, Nov 5, 2015 at 10:21 AM, Dilip Biswal <[email protected]> wrote: >>> Hello, >>> >>> I am getting the same build error about not being able tofind >>> com.google.common.hash.HashCodes. >>> >>> >>> Is there a solution to this ? >>> >>> Regards, >>> Dilip Biswal >>> Tel: 408-463-4980 >>> [email protected] >>> >>> >>> >>> From: Jean-Baptiste Onofré <[email protected]> >>> To: Ted Yu <[email protected]> >>> Cc: "[email protected]" <[email protected]> >>> Date: 11/03/2015 07:20 AM >>> Subject: Re: Master build fails ? >>> ________________________________ >>> >>> >>> >>> Hi Ted, >>> >>> thanks for the update. The build with sbt is in progress on my box. >>> >>> Regards >>> JB >>> >>> On 11/03/2015 03:31 PM, Ted Yu wrote: >>> > Interesting, Sbt builds were not all failing: >>> > >>> > https://amplab.cs.berkeley.edu/jenkins/job/Spark-Master-SBT/ >>> > >>> > FYI >>> > >>> > On Tue, Nov 3, 2015 at 5:58 AM, Jean-Baptiste Onofré <[email protected] >>> > <mailto:[email protected]>> wrote: >>> >>> > >>> > Hi Jacek, >>> > >>> > it works fine with mvn: the problem is with sbt. >>> > >>> > I suspect a different reactor order in sbt compare to mvn. >>> > >>> > Regards >>> > JB >>> > >>> > On 11/03/2015 02:44 PM, Jacek Laskowski wrote: >>> > >>> > Hi, >>> > >>> > Just built the sources using the following command and it worked >>> > fine. >>> > >>> > ➜ spark git:(master) ✗ ./build/mvn -Pyarn -Phadoop-2.6 >>> > -Dhadoop.version=2.7.1 -Dscala-2.11 -Phive -Phive-thriftserver >>> > -DskipTests clean install >>> > ... >>> > [INFO] >>> > >>> > ------------------------------------------------------------------------ >>> > [INFO] BUILD SUCCESS >>> > [INFO] >>> > >>> > ------------------------------------------------------------------------ >>> > [INFO] Total time: 14:15 min >>> > [INFO] Finished at: 2015-11-03T14:40:40+01:00 >>> > [INFO] Final Memory: 438M/1972M >>> > [INFO] >>> > >>> > ------------------------------------------------------------------------ >>> > >>> > ➜ spark git:(master) ✗ java -version >>> > java version "1.8.0_66" >>> > Java(TM) SE Runtime Environment (build 1.8.0_66-b17) >>> > Java HotSpot(TM) 64-Bit Server VM (build 25.66-b17, mixed mode) >>> > >>> > I'm on Mac OS. >>> > >>> > Pozdrawiam, >>> > Jacek >>> > >>> > -- >>> > Jacek Laskowski | >>> http://blog.japila.pl| >>> > http://blog.jaceklaskowski.pl >>> >>> > Follow me at https://twitter.com/jaceklaskowski >>> > Upvote at http://stackoverflow.com/users/1305344/jacek-laskowski >>> >>> > >>> > >>> > On Tue, Nov 3, 2015 at 1:37 PM, Jean-Baptiste Onofré >>> > <[email protected]<mailto:[email protected]>> wrote: >>> > >>> > Thanks for the update, I used mvn to build but without hive >>> > profile. >>> > >>> > Let me try with mvn with the same options as you and sbt >>> > also. >>> > >>> > I keep you posted. >>> > >>> > Regards >>> > JB >>> > >>> > On 11/03/2015 12:55 PM, Jeff Zhang wrote: >>> > >>> > >>> > I found it is due to SPARK-11073. >>> > >>> > Here's the command I used to build >>> > >>> > build/sbt clean compile -Pyarn -Phadoop-2.6 -Phive >>> > -Phive-thriftserver >>> > -Psparkr >>> > >>> > On Tue, Nov 3, 2015 at 7:52 PM, Jean-Baptiste Onofré >>> > <[email protected]<mailto:[email protected]> >>> > <mailto:[email protected]<mailto:[email protected]>>> >>> > wrote: >>> >>> > >>> > Hi Jeff, >>> > >>> > it works for me (with skipping the tests). >>> > >>> > Let me try again, just to be sure. >>> > >>> > Regards >>> > JB >>> > >>> > >>> > On 11/03/2015 11:50 AM, Jeff Zhang wrote: >>> > >>> > Looks like it's due to guava version >>> > conflicts, I see both guava >>> > 14.0.1 >>> > and 16.0.1 under lib_managed/bundles. Anyone >>> > meet this issue too ? >>> > >>> > [error] >>> > >>> > >>> > /Users/jzhang/github/spark_apache/core/src/main/scala/org/apache/spark/SecurityManager.scala:26: >>> > object HashCodes is not a member of package >>> > com.google.common.hash >>> > [error] import >>> > com.google.common.hash.HashCodes >>> > [error] ^ >>> > [info] Resolving >>> > org.apache.commons#commons-math;2.2 ... >>> > [error] >>> > >>> > >>> > /Users/jzhang/github/spark_apache/core/src/main/scala/org/apache/spark/SecurityManager.scala:384: >>> > not found: value HashCodes >>> > [error] val cookie = >>> > HashCodes.fromBytes(secret).toString() >>> > [error] ^ >>> > >>> > >>> > >>> > >>> > -- >>> > Best Regards >>> > >>> > Jeff Zhang >>> > >>> > >>> > -- >>> > Jean-Baptiste Onofré >>> > [email protected]< >>> mailto:[email protected]> >>> > <mailto:[email protected]<mailto:[email protected]>> >>> > http://blog.nanthrax.net >>> > Talend - http://www.talend.com >>> > >>> > >>> > >>> > --------------------------------------------------------------------- >>> > To unsubscribe, e-mail: >>> > [email protected] >>> > <mailto:[email protected]> >>> > <mailto:[email protected] >>> > <mailto:[email protected]>> >>> > For additional commands, e-mail: >>> > >>> > [email protected]<mailto:[email protected]> >>> > <mailto:[email protected] >>> >>> > <mailto:[email protected]>> >>> > >>> > >>> > >>> > >>> > -- >>> > Best Regards >>> > >>> > Jeff Zhang >>> > >>> > >>> > >>> > -- >>> > Jean-Baptiste Onofré >>> > [email protected]<mailto:[email protected]> >>> > http://blog.nanthrax.net >>> > Talend - http://www.talend.com >>> > >>> > >>> > --------------------------------------------------------------------- >>> > To unsubscribe, e-mail: [email protected] >>> > <mailto:[email protected]> >>> > For additional commands, e-mail: [email protected] >>> > <mailto:[email protected]> >>> > >>> > >>> > >>> > --------------------------------------------------------------------- >>> > To unsubscribe, e-mail: [email protected] >>> > <mailto:[email protected]> >>> > For additional commands, e-mail: [email protected] >>> > <mailto:[email protected]> >>> > >>> > >>> > -- >>> > Jean-Baptiste Onofré >>> > [email protected]<mailto:[email protected]> >>> > http://blog.nanthrax.net >>> > Talend - http://www.talend.com >>> > >>> > >>> > --------------------------------------------------------------------- >>> > To unsubscribe, e-mail: [email protected] >>> > <mailto:[email protected]> >>> > For additional commands, e-mail: [email protected] >>> > <mailto:[email protected]> >>> > >>> > >>> >>> -- >>> Jean-Baptiste Onofré >>> [email protected] >>> http://blog.nanthrax.net >>> Talend - http://www.talend.com >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected] >>> For additional commands, e-mail: [email protected] >>> >>> >>> >>> >>> >> > > > > -- > Marcelo
-- Marcelo --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
