Re: Hadoop 2.6 compatibility?
To clarify clarify, Ted's got the right formula. You would use -Phadoop-2.4 to set up the build configuration, and then customize -Dhadoop.version= further to the 2.4+ version you want. On Sat, Dec 20, 2014 at 12:35 AM, Denny Lee denny.g@gmail.com wrote: To clarify, there isn't a Hadoop 2.6 profile per se but you can build using -Dhadoop.version=2.4 which works with Hadoop 2.6. On Fri, Dec 19, 2014 at 12:55 Ted Yu yuzhih...@gmail.com wrote: You can use hadoop-2.4 profile and pass -Dhadoop.version=2.6.0 Cheers On Fri, Dec 19, 2014 at 12:51 PM, sa asuka.s...@gmail.com wrote: Can Spark be built with Hadoop 2.6? All I see instructions up to are for 2.4 and there does not seem to be a hadoop2.6 profile. If it works with Hadoop 2.6, can anyone recommend how to build? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Hadoop-2-6-compatibility-tp20790.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Hadoop 2.6 compatibility?
Can Spark be built with Hadoop 2.6? All I see instructions up to are for 2.4 and there does not seem to be a hadoop2.6 profile. If it works with Hadoop 2.6, can anyone recommend how to build? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Hadoop-2-6-compatibility-tp20790.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Re: Hadoop 2.6 compatibility?
You can use hadoop-2.4 profile and pass -Dhadoop.version=2.6.0 Cheers On Fri, Dec 19, 2014 at 12:51 PM, sa asuka.s...@gmail.com wrote: Can Spark be built with Hadoop 2.6? All I see instructions up to are for 2.4 and there does not seem to be a hadoop2.6 profile. If it works with Hadoop 2.6, can anyone recommend how to build? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Hadoop-2-6-compatibility-tp20790.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Re: Hadoop 2.6 compatibility?
To clarify, there isn't a Hadoop 2.6 profile per se but you can build using -Dhadoop.version=2.4 which works with Hadoop 2.6. On Fri, Dec 19, 2014 at 12:55 Ted Yu yuzhih...@gmail.com wrote: You can use hadoop-2.4 profile and pass -Dhadoop.version=2.6.0 Cheers On Fri, Dec 19, 2014 at 12:51 PM, sa asuka.s...@gmail.com wrote: Can Spark be built with Hadoop 2.6? All I see instructions up to are for 2.4 and there does not seem to be a hadoop2.6 profile. If it works with Hadoop 2.6, can anyone recommend how to build? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Hadoop-2-6-compatibility-tp20790.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Re: Hadoop 2.6 compatibility?
Here is the command I used: mvn package -Pyarn -Dyarn.version=2.6.0 -Phadoop-2.4 -Dhadoop.version=2.6.0 -Phive -DskipTests FYI On Fri, Dec 19, 2014 at 4:35 PM, Denny Lee denny.g@gmail.com wrote: To clarify, there isn't a Hadoop 2.6 profile per se but you can build using -Dhadoop.version=2.4 which works with Hadoop 2.6. On Fri, Dec 19, 2014 at 12:55 Ted Yu yuzhih...@gmail.com wrote: You can use hadoop-2.4 profile and pass -Dhadoop.version=2.6.0 Cheers On Fri, Dec 19, 2014 at 12:51 PM, sa asuka.s...@gmail.com wrote: Can Spark be built with Hadoop 2.6? All I see instructions up to are for 2.4 and there does not seem to be a hadoop2.6 profile. If it works with Hadoop 2.6, can anyone recommend how to build? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Hadoop-2-6-compatibility-tp20790.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Re: Hadoop 2.6 compatibility?
Sorry Ted! I saw profile (-P) but missed the -D. My bad! On Fri, Dec 19, 2014 at 16:46 Ted Yu yuzhih...@gmail.com wrote: Here is the command I used: mvn package -Pyarn -Dyarn.version=2.6.0 -Phadoop-2.4 -Dhadoop.version=2.6.0 -Phive -DskipTests FYI On Fri, Dec 19, 2014 at 4:35 PM, Denny Lee denny.g@gmail.com wrote: To clarify, there isn't a Hadoop 2.6 profile per se but you can build using -Dhadoop.version=2.4 which works with Hadoop 2.6. On Fri, Dec 19, 2014 at 12:55 Ted Yu yuzhih...@gmail.com wrote: You can use hadoop-2.4 profile and pass -Dhadoop.version=2.6.0 Cheers On Fri, Dec 19, 2014 at 12:51 PM, sa asuka.s...@gmail.com wrote: Can Spark be built with Hadoop 2.6? All I see instructions up to are for 2.4 and there does not seem to be a hadoop2.6 profile. If it works with Hadoop 2.6, can anyone recommend how to build? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Hadoop-2-6-compatibility-tp20790.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
